what is best way to paging big Resultset -Java

Question

i am looking for best aproach from perfomance point of view , to show Resultset on webpage partially , lets say by 10 item per page and if user want to see more result, he p

Answer 1

The answer would depend on your users' behavior: how often will the look at page 2, or page 10, or page 100.

If they rarely look at page 2, and never look at page 10 or page 100, then resubmitting the request may be fine.

If they usually look at page 2, often look at page 10, and occasionally look at page 100, then a partial cache will be useful: cache the first 100 (or 200, or 300) results, and only resubmit the query when they go past those results. I would probably store the cache in the user's session, although you have to give that some thought if your application server is clustered.

And if they always page through every result? Partial caches are still the answer, because you don't want to store large chunks of data in-memory.

Answer 2

You usually only get a "page" from the database.

Let's say a query

select * from mytable where column1="a";

would give 1000 records. Then getting a page would be like (mysql):

select * from mytable where column1="a" limit 0, 10;

for page 1 (0 to 10), and page 2 would be retrieved like:

select * from mytable where column1="a" limit 10, 20;

and so forth. If the data is large (1000 records), but not huge ( 1000 000 records), you can also give the entire dataset at once and use javascript to page. That has the added advantage that sorting can be done client-side.

Answer 3

If you can't use a cache-based approach due to memory limitations, use a query-based approach. Adjust the WHERE clause on your search query to explicitly select data based on which page the user has requested. This approach requires that you pass additional context information back and forth on your page requests.

One approach is to fetch pages using logical Row IDs (or primary keys) that delimit the page and identify each row in the result set.

Say you have a very simple table with a numerical sequence of row ids. If you are showing 100 rows per page, and the user has requested page two, you would adjust the WHERE clause as follows:

select col, col2 from my_table where
row_id > 100
and row_id <= 200
order by rownum asc

Answer 4

Since you have "GWT" in your tags, I'll assume your server app is running on Google App Engine (GAE).

• One approach is to have your first query obtain all results, store them in a DB, show the first 20 and then let the next/prev links pull subsets of the stored data out of the DB. You must remember to delete those results from the DB when your user's session times out!

• Another approach is to obtain all results on each page view, but skip through the results until you've hit the desired subset of 20, and output only those.

I think that with GAE underneath, the 2nd approach will work better, unless your query is likely to return more than 1000 results, which GAE won't let you retrieve in one transaction.

• The best approach, if your data and keys lend themselves to it, would be to pull out the correct 20 data items already at query time. But unless your data has continuously ascending integer keys, this may be hard to do.

Answer 5

If you are using JPA (which works quite well on GAE), you can paginate a result set using

Query#setFirstResult(int startPosition)

Query#setMaxResults(int maxResult)

This article might be helpful: Paging large data sets with a LazyList

Answer 6

you can cache/retrieve the records in web layer, backend layer(for example ejb) or the database layer(as last "limit" or row_id statement). which approach you should use depends on your requirement(as said by kdgregory).

most popular one is to cache them at the web layer using the session.