Django Selective Dumpdata

Question

Is it possible to selectively filter which records Django\'s dumpdata management command outputs? I have a few models, each with millions of rows, and I only want to dump record

Answer 1

I think django-fixture-magic might be worth a look at.

You'll find some additional background info in Scrubbing your Django database.

Answer 2

This snippet might be helpful for you (it follows relationships and serializes them):

http://djangosnippets.org/snippets/918/

You could use also that management command and override the default managers for whichever models you would like to return custom querysets.

Answer 3

This isn't a simple answer to my question, but I found some interesting docs on Django's built-in natural keys feature, which would allow representing serialized records without the primary key. Unfortunately, it doesn't look like this is fully integrated into dumpdata, and there's an old outstanding ticket to fully rely on natural keys.

It also seems the serializers.serialize() function allows serialization of an arbitrary list of specific model instances.

Presumably, if I implemented a natural_key() method on all my models, and then called serializers.serialize([Users.objects.filter(criteria)]), it should come close to accomplishing what I want. I might have to write a function to crawl all the FK references, and include those in the list of objects passed to serialize().

Answer 4

This is a very old question, but I recently wrote a custom management command to do just that. It looks very similar to the existing dumpdata command except that it takes some extra arguments to define how I want to filter the querysets and it overrides the get_objects function to perform the actual filtering:

def get_objects(dump_attributes, dump_values):
  qs_1 = ModelClass1.objects.filter(**options["filter_options_for_model_class_1"])    
  qs_2 = ModelClass2.objects.filter(**options["filter_options_for_model_class_2"])    
  # ...repeat for as many different model classes you want to dump...
  yield from chain(qs_1, qs_2, ...)