Django (?) really slow with large datasets after doing some python profiling

Question

I was comparing an old PHP script of mine versus the newer, fancier Django version and the PHP one, with full spitting out of HTML and all was functioning faster. MUCH faster to

Answer 1

There is a lot of things to assume about your problem as you don't have any type of code sample.

Here are my assumptions: You are using Django's built-in ORM tools and models (i.e. sales-data = modelobj.objects().all() ) and on the PHP side you are dealing with direct SQL queries and working with a query_set.

Django is doing a lot of type converting and casting to datatypes going from a database query into the ORM/Model object and the associated manager (objects() by default).

In PHP you are controlling the conversions and know exactly how to cast from one data type to another, you are saving some execution time based on that issue alone.

I would recommend trying to move some of that fancy number work into the database, especially if you are doing record-set based processing - databases eat that kind of processing from breakfast. In Django you can send RAW SQL over to the database: http://docs.djangoproject.com/en/dev/topics/db/sql/#topics-db-sql

I hope this at least can get you pointed in the right direction...