Retrieving only a fixed number of rows in MySQL

Question

I am testing my database design under load and I need to retrieve only a fixed number of rows (5000)

I can specify a LIMIT to achieve this, however it seems that the query builds the result set of all rows that match and then returns only the number of rows specified in the limit. Is that how it is implemented?

Is there a for MySQL to read one row, read another one and basically stop when it retrieves the 5000th matching row?

Answer 1

MySQL is smart in that if you specify a LIMIT 5000 in your query, and it is possible to produce that result without generating the whole result set first, then it will not build the whole result.

For instance, the following query:

SELECT * FROM table ORDER BY column LIMIT 5000

This query will need to scan the whole table unless there is an index on column, in which case it does the smart thing and uses the index to find the rows with the smallest column.

Answer 2

 SELECT * FROM `your_table` LIMIT 0, 5000 

This will display the first 5000 results from the database.

 SELECT * FROM `your_table` LIMIT 1001, 5000 

This will show records from 1001 to 6000 (counting from 0).

Answer 3

@Jarosław Gomułka is right
If you use LIMIT with ORDER BY, MySQL ends the sorting as soon as it has found the first row_count rows of the sorted result, rather than sorting the entire result. If ordering is done by using an index, this is very fast. In either case, after the initial rows have been found, there is no need to sort any remainder of the result set, and MySQL does not do so. if the set is not sorted it terminates the SELECT operation as soon as it's got enough rows to the result set.

Answer 4

Complexity of such query is O(LIMIT) (unless you specify order by).

It means that if 10000000 rows will match your query, and you specify limit equal to 5000, then the complexity will be O(5000).

Answer 5

The exact plan the query optimizer uses depends on your query (what fields are being selected, the LIMIT amount and whether there is an ORDER BY) and your table (keys, indexes, and number of rows in the table). Selecting an unindexed column and/or ordering by a non-key column is going to produce a different execution plan than selecting a column and ordering by the primary key column. The later will not even touch the table, and only process the number of rows specified in your LIMIT.

Answer 6

As I explained in this article, each database defines its own ay of limiting the result set size depends on the database you are using.

While the SQL:2008 specification defines a standard syntax for limiting a SQL query, MySQL 8 does not support it.

Therefore, on MySQL, you need to use the LIMIT clause to restrict the result set to the Top-N records:

SELECT
    title
FROM
    post
ORDER BY
    id DESC
LIMIT 50

Notice that we are using an ORDER BY clause since, otherwise, there is no guarantee which are the first records to be included in the returning result set.