问题
When using ModelChoiceField or ModelMultipleChoiceField in a Django form, is there a way to pass in a cached set of choices? Currently, if I specify the choices via the queryset parameter, it results in a database hit.
I'd like to cache these choices using memcached and prevent unnecessary hits to the database when displaying a form with such a field.
回答1:
You can override "all" method in QuerySet something like
from django.db import models
class AllMethodCachingQueryset(models.query.QuerySet):
def all(self, get_from_cache=True):
if get_from_cache:
return self
else:
return self._clone()
class AllMethodCachingManager(models.Manager):
def get_query_set(self):
return AllMethodCachingQueryset(self.model, using=self._db)
class YourModel(models.Model):
foo = models.ForeignKey(AnotherModel)
cache_all_method = AllMethodCachingManager()
And then change queryset of field before form using (for exmple when you use formsets)
form_class.base_fields['foo'].queryset = YourModel.cache_all_method.all()
回答2:
The reason that ModelChoiceField in particular creates a hit when generating choices - regardless of whether the QuerySet has been populated previously - lies in this line
for obj in self.queryset.all():
in django.forms.models.ModelChoiceIterator. As the Django documentation on caching of QuerySets highlights,
callable attributes cause DB lookups every time.
So I'd prefer to just use
for obj in self.queryset:
even though I'm not 100% sure about all implications of this (I do know I do not have big plans with the queryset afterwards, so I think I'm fine without the copy .all() creates). I'm tempted to change this in the source code, but since I'm going to forget about it at the next install (and it's bad style to begin with) I ended up writing my custom ModelChoiceField:
class MyModelChoiceIterator(forms.models.ModelChoiceIterator):
"""note that only line with # *** in it is actually changed"""
def __init__(self, field):
forms.models.ModelChoiceIterator.__init__(self, field)
def __iter__(self):
if self.field.empty_label is not None:
yield (u"", self.field.empty_label)
if self.field.cache_choices:
if self.field.choice_cache is None:
self.field.choice_cache = [
self.choice(obj) for obj in self.queryset.all()
]
for choice in self.field.choice_cache:
yield choice
else:
for obj in self.queryset: # ***
yield self.choice(obj)
class MyModelChoiceField(forms.ModelChoiceField):
"""only purpose of this class is to call another ModelChoiceIterator"""
def __init__(*args, **kwargs):
forms.ModelChoiceField.__init__(*args, **kwargs)
def _get_choices(self):
if hasattr(self, '_choices'):
return self._choices
return MyModelChoiceIterator(self)
choices = property(_get_choices, forms.ModelChoiceField._set_choices)
This does not solve the general problem of database caching, but since you're asking about ModelChoiceField in particular and that's exactly what got me thinking about that caching in the first place, thought this might help.
回答3:
Here is a little hack I use with Django 1.10 to cache a queryset in a formset:
qs = my_queryset
# cache the queryset results
cache = [p for p in qs]
# build an iterable class to override the queryset's all() method
class CacheQuerysetAll(object):
def __iter__(self):
return iter(cache)
def _prefetch_related_lookups(self):
return False
qs.all = CacheQuerysetAll
# update the forms field in the formset
for form in formset.forms:
form.fields['my_field'].queryset = qs
回答4:
I also stumbled over this problem while using an InlineFormset in the Django Admin that itself referenced two other Models. A lot of unnecessary queries are generated because, as Nicolas87 explained, ModelChoiceIterator fetches the queryset everytime from scratch.
The following Mixin can be added to admin.ModelAdmin, admin.TabularInline or admin.StackedInline to reduce the number of queries to just the ones needed to fill the cache. The cache is tied to the Request object, so it invalidates on a new request.
class ForeignKeyCacheMixin(object):
def formfield_for_foreignkey(self, db_field, request, **kwargs):
formfield = super(ForeignKeyCacheMixin, self).formfield_for_foreignkey(db_field, **kwargs)
cache = getattr(request, 'db_field_cache', {})
if cache.get(db_field.name):
formfield.choices = cache[db_field.name]
else:
formfield.choices.field.cache_choices = True
formfield.choices.field.choice_cache = [
formfield.choices.choice(obj) for obj in formfield.choices.queryset.all()
]
request.db_field_cache = cache
request.db_field_cache[db_field.name] = formfield.choices
return formfield
回答5:
@jnns I noticed that in your code the queryset is evaluated twice (at least in my Admin inline context), which seems to be an overhead of django admin anyway, even without this mixin (plus one time per inline when you don't have this mixing).
In the case of this mixin, this is due to the fact that formfield.choices has a setter that (to simplify) triggers the re-evaluation of the object's queryset.all()
I propose an improvement which consists of dealing directly with formfield.cache_choices and formfield.choice_cache
Here it is:
class ForeignKeyCacheMixin(object):
def formfield_for_foreignkey(self, db_field, request, **kwargs):
formfield = super(ForeignKeyCacheMixin, self).formfield_for_foreignkey(db_field, **kwargs)
cache = getattr(request, 'db_field_cache', {})
formfield.cache_choices = True
if db_field.name in cache:
formfield.choice_cache = cache[db_field.name]
else:
formfield.choice_cache = [
formfield.choices.choice(obj) for obj in formfield.choices.queryset.all()
]
request.db_field_cache = cache
request.db_field_cache[db_field.name] = formfield.choices
return formfield
回答6:
@lai With Django 2.1.2 I had to change the code in the first if-statement from formfield.choice_cache = cache[db_field.name] to formfield.choices = cache[db_field.name] as in the answer from jnns. In the Django version 2.1.2 if you inherit from admin.TabularInline you can override the method formfield_for_foreignkey(self, db_field, request, **kwargs) directly without the mixin. So the code could look like this:
class MyInline(admin.TabularInline):
model = MyModel
formset = MyModelInlineFormset
extra = 3
def formfield_for_foreignkey(self, db_field, request, **kwargs):
formfield = super().formfield_for_foreignkey(db_field, request, **kwargs)
cache = getattr(request, 'db_field_cache', {})
formfield.cache_choices = True
if db_field.name in cache:
formfield.choices = cache[db_field.name]
else:
formfield.choice_cache = [
formfield.choices.choice(obj) for obj in formfield.choices.queryset.all()
]
request.db_field_cache = cache
request.db_field_cache[db_field.name] = formfield.choices
return formfield
In my case I also had to override get_queryset to get the benefit from select_related like this:
class MyInline(admin.TabularInline):
model = MyModel
formset = MyModelInlineFormset
extra = 3
def formfield_for_foreignkey(self, db_field, request, **kwargs):
formfield = super().formfield_for_foreignkey(db_field, request, **kwargs)
cache = getattr(request, 'db_field_cache', {})
formfield.cache_choices = True
if db_field.name in cache:
formfield.choices = cache[db_field.name]
else:
formfield.choice_cache = [
formfield.choices.choice(obj) for obj in formfield.choices.queryset.all()
]
request.db_field_cache = cache
request.db_field_cache[db_field.name] = formfield.choices
return formfield
def get_queryset(self, request):
return super().get_queryset(request).select_related('my_field')
来源：https://stackoverflow.com/questions/8176200/caching-queryset-choices-for-modelchoicefield-or-modelmultiplechoicefield-in-a-d