how to query sqlite for certain rows, i.e. dividing it into pages (perl DBI)

Question

sorry for my noob question, I'm currently writing a perl web application with sqlite database behind it. I would like to be able to show in my app query results which might get thousands of rows - these should be split in pages - routing should be like /webapp/N - where N is the page number. what is the correct way to query the sqlite db using DBI, in order to fetch only the relavent rows.

for instance, if I show 25 rows per page so I want to query the db for 1-25 rows in the first page, 26-50 in the second page etc....

Answer 1

Using the LIMIT/OFFSET construction will show pages, but the OFFSET makes the query inefficient, and makes the page contents move off when the data changes.

It is more efficient and consistent if the next page starts the query at the position where the last one ended, like this:

SELECT *
FROM mytable
ORDER BY mycolumn
WHERE mycolumn > :lastvalue
LIMIT 25

This implies that your links are not /webapp?Page=N but /webapp?StartAfter=LastKey.

This is explained in detail on the Scrolling Cursor page.

Answer 2

You should do something like this:

SELECT column FROM table ORDER BY somethingelse LIMIT 0, 25

and when the user clicks on page 2, you should do:

SELECT column FROM table ORDER BY somethingelse LIMIT 25, 50

and so on..

Answer 3

You'd most likely be using the LIMIT and OFFSET keywords, something like this:

$sth->prepare("SELECT foo FROM bar WHERE something LIMIT ? OFFSET ?");
$sth->execute($limit, $offset);
while ( my @row = $sth->fetchrow_array ) { # loop contains 25 items

The $limit and $offset variables would be controlled by the parameters passed to your script by html/cgi/whatever features.

Answer 4

Pagination is one of those problems a lot of CPAN modules have already solved. If you're using straight SQL, you could look at something like DBIx::Pager. You might also want to check out something like Data::Pageset to help you manage creating the links to your various pages. If you're using DBIx::Class (which is an excellent tool) for your SQL queries, then DBIx::Class::ResultSet::Data::Pageset will make this very easy for you.

Essentially handling the SQL is one end of it, but you'll also need to solve various problems in the templating aspect of it. I'd encourage you to have a look at these modules and maybe even poke around CPAN a little bit more to see where somebody else has already done the heavy lifting for you with respect to pagination.