mysql order by rand() performance issue and solution

Question

i was using order by rand() to generate random rows from database without any issue but i reaalised that as the database size increase this rand() causes heavy load on serve

Answer 1

You are correct, ORDER BY RAND() is not good solution if you are dealing with large datasets. Depending how often it needs to be randomized, what you can do is generate a column with a random number and then update that number at some predefined interval.

You would take that column and use it as your sort index. This works well for a heavy read environment and produces predicable random order for a certain period of time.

Answer 2

First of, all generate a random value from 1 to MAX(id), not 100000000.

Then there are at least a couple of good solutions:

1. Use > not =

   SELECT items FROM tablea where status='0' and id>'$id23' LIMIT 1
   

   Create an index on (status,id,items) to make this an index-only query.

2. Use =, but just try again with a different random value if you don't find a hit. Sometimes it will take several tries, but often it will take only one try. The = should be faster since it can use the primary key. And if it's faster and gets it in one try 90% of the time, that could make up for the other 10% of the time when it takes more than one try. Depends on how many gaps you have in your id values.

Answer 3

Use your DB to find the max value from the table, generate a random number less than or equal to that value, grab the first row in which the id is greater than or equal to your random number. No PHP necessary.

SELECT items
FROM tablea
WHERE status = '0' and
      id >= FLOOR(1 + RAND() * (SELECT MAX(id) FROM tablea))
LIMIT 1

Answer 4

A possible solution is to use limit:

$id23=rand(1,$numberOfRows);

SELECT items FROM tablea where status='0' LIMIT $id23 1

This wont produce any missed rows (but as hek2mgl mentioned) requires knowing the number of rows in the select.