How to use Django QuerySet.union() in ModelAdmin.formfield_for_manytomany()?

Question

Not sure what I am doing wrong here:

I tried to use QuerySet.union(), in Django 2.2.10, to combine two querysets (for the same model) inside ModelAdmin.formfiel

Answer 1

As @tom-carrick pointed out, it appears that a QuerySet returned by QuerySet.union() cannot be filtered. I suppose this is implied by the following excerpt from the documentation:

In addition, only LIMIT, OFFSET, COUNT(*), ORDER BY, and specifying columns (i.e. slicing, count(), order_by(), and values()/values_list()) are allowed on the resulting QuerySet.

If you're using Django 3.0, calling filter() on the result of QuerySet.union() will raise an exception with a pretty clear message:

django.db.utils.NotSupportedError: Calling QuerySet.filter() after union() is not supported.

However, no exception is raised if you're using Django 2.2: In that case it just returns the complete queryset, regardless of the filter arguments. Here's a little test to illustrate that (in Django 2.2):

# using Django 2.2.10
class PublicationTests(TestCase):
    def test_union_filter(self):
        for i in range(2):
            Publication.objects.create()
        queryset_union = Publication.objects.filter(id=1).union(
            Publication.objects.filter(id=2))
        self.assertEqual(2, len(queryset_union))
        for obj in queryset_union.all():
            self.assertIn(obj, queryset_union.filter(id=1))
            self.assertIn(obj, queryset_union.filter())
            self.assertIn(obj, queryset_union.filter(id=0))

So, this must be what happens when we use QuerySet.union() to restrict a queryset in the ModelAdmin: The selection widget works as expected, but when the form is validated, filter() is called on the output of QuerySet.union() (see source for the ModelMultipleChoiceField), and that always returns the complete queryset, regardless of the actual subselection.

Depending on the actual use case, there may be ways around using union(), as explained in tom-carrick's answer.

However, there is at least one way to work around the restrictions imposed by QuerySet.union() in this situation, and that is to create a new queryset from the queryset-union:

Here's a modified version of the ArticleAdmin from the original example:

class ArticleAdmin(admin.ModelAdmin):
    def formfield_for_manytomany(self, db_field, request, **kwargs):
        if db_field.name == 'publications':
            queryset_union = Publication.objects.all().union(
                Publication.objects.all())
            kwargs['queryset'] = Publication.objects.filter(id__in=queryset_union)
        return super().formfield_for_manytomany(db_field, request, **kwargs)

Again, the actual query in this contrived example makes no sense, but that is not important here.

This might not be the most efficient solution in terms of database access.

Answer 2

The problem does seem to be .union(), though I can't figure out why. It seems like a bug, or at least funky behaviour.

Since you don't specify your actual use-case, it's hard to know, but for the example you give you can use the OR operator instead, which will work for that:

class ArticleAdmin(admin.ModelAdmin):
    def formfield_for_manytomany(self, db_field, request, **kwargs):
        if db_field.name == 'publications':
            # the following query makes no sense, but it shows an attempt to
            # combine two separate QuerySets using QuerySet.union()
            kwargs['queryset'] = (
                Publication.objects.filter(id__lt=3)
                | Publication.objects.filter(id__gt=2)
            )
        return super().formfield_for_manytomany(db_field, request, **kwargs)