AWS has become a fairly ubiquitous hosting option for small companies. But developers typically work on local dev setups outside the host and occasionally need access to the main dev and staging databases located on AWS. How do you do this and still run in a secure environment?
It's ubiquitous on data driven web sites: the result set pager. We've all used them whether we built them from scratch or used one provided by the framework.
Pagers are by nature performance suckers because we're asking the database to re-run the same query for each "page", slicing off just one set of contiguous rows for each page. If your result set is 10,000 rows long but you're only paging through them 10 rows at a time, that's potentially 1,000 database requests to view the entire set.
But it's worse than that because in order to provide those nifty pager controls, like those in the image above, the software has to know how many rows are in the larger result set so it can do the math to populate the navigation for those page numbers. In other words, using the above example the software needs to know that there is a Page 14 to jump to.
A little background first. Internally, garden variety pagers are pretty much the same. They request a fixed number of rows to display per page, like 10. That becomes your LIMIT filter in the database query:
SELECT * FROM people LIMIT 10;
To create the page navigation you need to do some math to generate the OFFSET. For instance, using a page size of 10, the query for a Page 3 display would look like this:
SELECT * FROM people OFFSET 20 LIMIT 10;