Node.js API with express & mysql - Getting record count, page number, … & provide pagination

问题

I would like to get and provide the total total_record_count, page_number, page_size, total_pages, has_more information in the response and being able to paginate through the result since I LIMIT the query to 100. What's the best way to do this?

That's my current setup:

router.get('/?', function(req, res, next) {
    const sql = "SELECT * from users ";
    const existingParams = ["title", "active"].filter(field => req.query[field]);

    if (existingParams.length) {
        sql += " WHERE ";
        sql += existingParams.map(field => `${field} = ?`).join(" AND ");
        sql += " LIMIT 100 ";
    }

    connection.query(
        sql,
        existingParams.map(field => req.query[field]),
        function (error, results, fields) {
            res.json({"status": 200, "error": null, "total_record_count": 604, "page_number": 1, "page_size": 100, "total_pages": 7, "has_more": true, "records": results});
        }
    );
});

In the url I would like to be able to provide/allow a page parameter 'p' if the the total_record_count exceeds the LIMIT. So the query parameter p specifies which page to return, starting with 1. If the p parameter is omitted, the default value is 1. Sth like:

http://localhost:4001/api/v1/users/?active=0&title=mr&p=1

回答1:

MySQL LIMIT takes 2 values, offset & rowcount. Manipulating these is how you could of course do paging.

eg. If say each page was 10 records long. Page1 = LIMIT 0, 10 Page2 = LIMIT 10, 10 Page3 = LIMIT 20, 10 etc.

IOW: LIMIT (pageNo - 1) * PageSize, PageSize

Now one issue with using limit is that the recordcount is for the resultset, IOW: the limited 10 records.

But what you can do is ask MySQL to store what the recordcount would have been if the LIMIT was not applied. You can retrieve this by prefixing the SQL with SQL_CALC_FOUND_ROWS.

eg. SELECT SQL_CALC_FOUND_ROWS * FROM TABLE WHERE something LIMIT 10, 10

You can then do another query that retrieves this value.

SELECT FOUND_ROWS();

回答2:

For pagination, you need a query something like below in MySQL

SELECT * FROM users LIMIT 0,10

With two arguments, the first argument specifies the offset of the first row to return, and the second specifies the maximum number of rows to return. The offset of the initial row is 0 (not 1).

As you want to have default value as 1st page and 100 result

router.get('/?', function(req, res, next) {
    const sql = "SELECT * from users ";
    const existingParams = ["title", "active"].filter(field => req.query[field]);
    const pageNum = req.query.p || 1;
    const pageSize = req.query.p_size || 100;


    if (existingParams.length) {
        sql += " WHERE ";
        sql += existingParams.map(field => `${field} = ?`).join(" AND ");
    }
    sql += ` LIMIT  ${(pageNum - 1) * PageSize},${PageSize}`;
    ...
});

and for your the second question about providing total row count, you need to run two queries for that. You can utilize SQL_CALC_FOUND_ROWS and FOUND_ROWS() in mysql>4.0. But when we have appropriate indexes for WHERE/ORDER clause in our query, it is much faster to use two separate queries instead of one with SQL_CALC_FOUND_ROWS.(one for data and one for getting all row count).

Now you have to run both the queries parallel for performance so with callback you can utilize this function that I have written for running callback in parallel and wait until all callbacks are completed. Or you can use promisified MySQL library here or you can make your current library promisified using Bluebird.

Like this:

const connection = mysql.createConnection({.....});
global.db  = Bluebird.promisifyAll(connection);
db.queryAsync("SELECT * FROM users")
.then(function(rows){ 
    console.log(rows);
})

and then run the query like this

Promise.all([
    db.queryAsync("SELECT * FROM users WHERE title='ridham' LIMIT 0,10"), // generated by your code
    db.queryAsync("SELECT COUNT(*) FROM users WHERE title='ridham'")
]).then(([data, count]) => {
        // your logic to send response
})

Or you can also run following query, that would also work

SELECT * FROM 'table' JOIN (SELECT COUNT(*) FROM 'table') t2 WHERE title='ridham' LIMIT 0,10")

Also, you can use ORM like Sequilize or Waterline. That would make your like 10x easier for MYSQL at least.

as example in sequilize:

User.findAndCountAll({
    where: {
        title: {
          [Op.like]: 'ridham'
        }
    },
    offset: 10,
    limit: 2
})
.then(result => {
    console.log(result.count);
    console.log(result.rows);
});

来源：https://stackoverflow.com/questions/48944969/node-js-api-with-express-mysql-getting-record-count-page-number-prov

标签

javascript

mysql

sql

node.js

express