Django `bulk_create` with related objects

生来就可爱ヽ(ⅴ<●) 提交于 2020-01-03 16:46:15

问题


I have a Django system that runs billing for thousands of customers on a regular basis. Here are my models:

class Invoice(models.Model):
    balance = models.DecimalField(
        max_digits=6,
        decimal_places=2,
    )

class Transaction(models.Model):
    amount = models.DecimalField(
        max_digits=6,
        decimal_places=2,
    )
    invoice = models.ForeignKey(
        Invoice,
        on_delete=models.CASCADE,
        related_name='invoices',
        null=False
    )

When billing is run, thousands of invoices with tens of transactions each are created using several nested for loops, which triggers an insert for each created record. I could run bulk_create() on the transactions for each individual invoice, but this still results in thousands of calls to bulk_create().

How would one bulk-create thousands of related models so that the relationship is maintained and the database is used in the most efficient way possible?

Notes:

  • I'm looking for a native Django solution that would work on all databases (with the possible exception of SQLite).
  • My system runs billing in a celery task to decouple long-running code from active requests, but I am still concerned with how long it takes to complete a billing cycle.
  • The solution should assume that other requests or running tasks are also reading from and writing to the tables in question.

回答1:


You could bulk_create all the Invoice objects, refresh them from the db, so that they all have ids, create the Transaction objects for all the invoices and then also save them with bulk_create. All of this can be done inside a single transaction.atomic context.

Also, specifically for django 1.10 and postrgres, look at this answer.



来源:https://stackoverflow.com/questions/40789962/django-bulk-create-with-related-objects

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!