I have written a Symfony command to import some data from an API. It works but the problem is my PHP memory usage increases when I insert a big JSON in my database. And my unitO
Try to reset the entity manager with:
$this->getContainer()->get('doctrine')->resetEntityManager();
and then:
$em = $this->getContainer()->get('doctrine')->getEntityManager();
Symfony logs all SQL queries in dev environment, so first you need to disable it
// disable logger
$em->getConnection()->getConfiguration()->setSQLLogger(null);
You may use event listeners on entities, it may also increase memory usage. You can disable them like so
// remove all listeners
foreach ($em->getEventManager()->getListeners() as $event => $listeners) {
foreach ($listeners as $listener) {
$em->getEventManager()->removeEventListener($event, $listener);
}
}
Remove unset
from your code, there is no need for them, as you clear entity manager every step of your loop.
// save and clear
$em->flush();
$em->getUnitOfWork()->clear();
Remember that doctrine can optimize your queries, and improve perfomance if you group queries into one flush
. So the best practice would be to execute flush
once over some parts of your data. For example:
// collect 100 entities and then save them
if (($i % 100) == 0) {
$em->flush();
$em->getUnitOfWork()->clear();
}