reindex Catalog URL Rewrites in magento never ends

Question

I\'ve imported 6K categories and 16K products to magento using a custom import profile. When I\'m trying to reindex everything works except for \'Catalog URL Rewrites\' that kee

Answer 1

My own attempt of reindexing URL rewrites was with 10,000 products and it took me like an Hour and Half to finish up.

this link will help you do it from Command Line , which is preferable than doing it from the same Magento admin panel.

http://www.yireo.com/tutorials/magento/magento-administration/340-magento-14-cronjobs

Make sure you read the part about skipping some basic configuration when facing memory limits , this is the one i usually use.

Answer 2

I was encountering the same problem, the server that I am running this on is a FreeBSD server. With the help of the sample of the above code I was able to get the issue to resolve buy running the following command in the shell.

/usr/local/bin/php /usr/local/www/magento/shell/indexer.php --reindex catalog_url

It runs very quickly and resolved my issue. I then created a cron for this to run every 6 hours on my server.

Answer 3

You need catalog URL rewrites to make your catalog work properly. How long are you letting it run for? There's a really substantial amount of data to be written to it...

If you cannot make it work programmatically, it may be possible to shove the data into the table manually and force the index to feel refreshed? Caveat emptor, I have not tried this.

Answer 4

You can sort of ignore this index if you do not care about pretty, search engine "friendly" URL's. The products will still appear in the catalog but will have their default "Zend Framework" type URL (/catalog/product/view/id/123)

My store has 150,000+ SKUs in two store views. On my development environment it took almost two weeks to complete.

The best way to determine how fast it is running is to look at the core_url_rewrite database table. It appears that the process starts at product ID 1 in store ID 1 and looks to make sure all of its URLs exist and creates the ones that do not yet exist.

For me the reason it took so long was that it had to go through my entire catalog twice to make all the URLs. One thing I did notice was that this process creates a ton of URLs that are completely unnecessary. In our store easily 90% of our products belong to a configurable product so their visibility is set to "Not Visible Individually" so they never would need to have a URL. This index creates those URLs anyways.

Hopefully this will shed some light on to how this URL works. I would keep an eye on that database table so you sorta know how far the process has come. I would also seriously consider running the reindex process for the command line. I have included a link that explains how to do this (disclosure: it's from my blog)

http://overlycaffeinated.com/2011/02/when-reindexing-in-magento-fails-use-the-command-line/

Answer 5

You can reindex using command line it will reduce 75% of load and your magento admin panel working as normal while reindexing.