Django (?) really slow with large datasets after doing some python profiling

Question

I was comparing an old PHP script of mine versus the newer, fancier Django version and the PHP one, with full spitting out of HTML and all was functioning faster. MUCH faster to

Answer 1

In such a scenario the database is often the bottleneck. Also, using an ORM might result in sub-optimal SQL queries.

As some pointed out it's not possible to tell what the probem really is, just with the information you provided.

I just can give you some general advice:

• If your view is working with related model objects, consider using select_related(). This simple method might speed up the queries generated by the ORM considerably.
• Use the Debug Footer Middleware to see what SQL queries are generated by your views and what time they took to execute.

PS: Just fyi, I had once a fairly simple view which was very slow. After installing the Debug Footer Middleware I saw that around 500! sql queries were executed in that single view. Just using select_related() brought that down to 5 queries and the view performed as expected.