Postgresql Table Partitioning Django Project

Question

I have a Django 1.7 project that uses Postgres 9.3. I have a table that will have rather high volume. The table will have anywhere from 13million to 40million new rows a month.

Answer 1

As long as you use inheritance, and then only connect the parent table to your Django model, the partitions should be entirely transparent to Django. That is, a SELECT on the parent table will cascade down to the partitions, unless the ONLY keyword is explicitly used (where applicable).

Note that partitioning does add complexity in terms of needing to implement a programmatic method of determining when new partitions need to be created, and then creating them -- or doing this manually at certain intervals. Depending on your exact data and business logic, it is quite likely you may also need to implement triggers and rules to determine which partition to, say, INSERT something into (since you wouldn't want to INSERT into the parent table). These, too, should be abstracted from Django, however.

I have found that, depending on the exact circumstances, this may need to be done with your main apps shutdown, lest the new partition creation cause a deadlock.

Also worth considering is whether you need true partitions which get created over time, or if an inheritance model of, say, tables foo and foo_archive would suffice, where foo_archive inherits from foo, and periodically something (e.g. a script) moves older data into foo_archive to keep foo smaller.