Oh, the data table, the workhorse of the enterprise application. This is one of the most common UI elements. Yet even with its ubiquity, there remains a battle on the table scrolling functionality; pagination versus infinite scroll.
But before we get into the battle, let’s set some context. On traditional dashboards, where there are a fixed number of cards, loading data and auto-refreshing is easy. When your data is first presented, the middleware or backend will send the number of data points to plot with a single API call, already aggregated and collapsed into a manageable contained set of data.
As new data comes in, you update the numbers/charts to reflect the updated statistics on a regular interval (keep data fresh every 2 minutes) or continuously (as new data is found on the backend, immediately push and update the UI). While there are technical and logistic considerations – it is clean and easy. All the developer needs to do for an end-user is have a “last updated” timestamp (well, relative timestamp – 5 minutes ago) and animate the update (fade in/out, or animate visualization changing). But the key is that we are (with some rare exceptions) dealing with a much smaller amount of data than the original dataset, so updates are quick and easy.
However, there are many more challenges on grids/lists/tables (in the rest of the article I’ll be referring to these as tables – acknowledging that they are all different UI elements), both in the initial load and with updates. Why? Tables contain all your data. In the big data world, this can be millions if not billions of records. To load that much data, it would take a considerable amount of time to pull out of your backend, render, and be a giant security hole in that you are loading all your data out into your UI – even data that isn’t needed/viewed by the end-user.
This is viewed as the pagination versus infinite scroll battle. Traditionally, often due to technical limitations, pagination was the way to do this. Yet when we consider the end user’s experience, the winner should be infinite scroll in most situations. So let’s look at these two methods.
Normal Usage
In pagination, a subset of data (e.g., 20 records) is shown at one timeThe user can see what “page” they are on, where each page number represents how many subsets they have to view (e.g., page 3 starts at record 41 and goes to record 60). The end-user clicks to go to the next page with tthe following subset of 20 records, and for each click, there is a call to the backend, and the table repaints/updates the data. The end-user sees the load happen. A plus with this approach is that the end-user can “jump” to one specific page of records (e.g., page 16) if they know what they are looking for and where the records occur in the sorted list.
On the other hand, Infinite scroll still loads a subset of records (e.g., 20). Again, as the end-user scrolls through the table, the table itself proactively loads another subset. It appends the data to the end of the table so the scrolling process feels smooth and continuous. The end-user never knows there was an additional backend call so there is no interruption. Unlike pagination, end-users cannot jump to a specific page. They must rely on a filtering mechanism if they know what they are looking for and want to find it.
Both solutions solve the original problem of pulling all of the data from your database into your UI.. While the infinite scroll has fewer clicks and a more seamless experience, most people cite the lack of “page jumping” as a reason to choose pagination.
But here is what most people miss – if your end-user knows what they are looking for,jumping to a random page in an attempt to find that specific record(s) is not directed! The end-user will spend a ton of time going to page 20, then 15, then 18, then 16, then finding it on 17. That is 4+ clicks and a good chunk of time loading data. Filtering is the best option if an end-user knows what they want (or do not want), even loosely. If your table supports filtering, what value does pagination provide since filtering is a more efficient way to find or refine results. At that point, a faster, smoother, and less click-heavy method of infinite scroll is more efficient.
One slight exception is when the data is ranked and not sorted, and the exact order is meaningful. In this one rare exception, a user may want to see the 150th and onward rank player or the 200+ ranked city. So in this one exception, where the order is meaningful and jumps to a specific spot, then pagination could be helpful. That said, a filtering system that would allow you to filter on the rank (e.g., rank greater than 150). This would also get the job done and allow for infinite scroll.
Winner: Infinite Scroll
Performance
Another aspect of tables where people cite pagination as being the better option is on the technical side – the speed and performance. Both tables and infinite scroll load a small subset of data from the backend. Actually, both pagination and infinite scroll call these subsets of data “pages.” So in this regard, pagination and infinite scroll are the same things. The difference is that one appends records while the other replaces records. So, in this aspect of performance it is a wash.
But let’s consider the successive loads of records. When you replace the data (pagination), your browser needs only render the new N records. This keeps the performance hit on the browser at the number of records in the pagination size.
Initially, the infinite scroll would keep appending records, which meant the browser’s render would maintain all the pages. his for sure was a performance issue. But we say was because high-performant tables don’t do this anymore. Instead, the tables will intelligently remove records that are not visible to the end-user; this is called virtual scrolling. This means that only the visible rows (plus a few above and below for scrolling). So if you think about a page size of 100 records, only 20 records are visible on screen. A pagination table would always be rendering 100 records at a time. Still, the infinite scroll table (which only maintains the visible amount of records) would be rendering maybe 25 or 30 records at a time. Now, if a pagination table does this enhancement, then both approaches are equal once again.
The last aspect of performance is the speed of loading a new page. At first glance, both techniques will be the same since they bothloaddata in the UI at the same page size. However, there are some key differences that infinite scroll has over pagination. First, an infinite scroll preloads the next page before you get there, meaning there is no wait time for the next page to load. In contrast, pagination only loads data when you click the page you wish to go to ,which could be the next page or an arbitrary page.
Second, high performance infinite scroll tables (using the virtual scroll) have a virtual table that loads a larger page size (let’s say 3N records), and therefore pushes the normal size table (N records) to the UI. Then it has many more records in memory, meaning the load of the next page of data doesn’t even hit the backend. Thus, all the backend queries happen well in advance of the virtual page being sent.
This last aspect is not quite as noticeable in small scrolling events. Still, if someone is rapidly dragging the scroll bar up/down, having multiple of a page size pre-loaded, larger scrolling events are rapid and seamless. As a result, the end-user rarely sees the lag from a data request to the backend beyond the initial page load.
Winner: Infinite scroll
Keeping Data Fresh
In many systems, the data is constantly getting updated. New events happen, and therefore the data in tables becomes out of date. So we need a way to keep the data in tables fresh.
In pagination, one of two things happen. Both can be valid – depending on what we think a user wants:
Go to page 1 of data, and get new data. This is under the philosophy that the end user really cares about the top ranked data, and if they let a refresh happen, they are reconsidering their entire stack. Think of security alerts – you want to always know the most critical alerts. In this context, you always want to start from the top, b/c a random page isn’t really meaningful, it is just where they happened to be before the refresh.
Stay on the same page of data, but show the new records on that arbitrary page – this makes sense only if the data sort is meaningful – such that page 6 is typically “people w/ names starting with N” so we really want to monitor those people, so page 6 is not arbitrary but has some implicit meaning.
When we consider these two scenarios, almost every situation, we are in case 1. And even if your product happens to fall into use-case 2, that can be resolved thorough filtering.
In other words, if you are looking at a pagination table with some data in it, and the table updates and shifts your records out of the way (scenario 2) – the ones you are looking at – you will be very frustrated because your records just moved away. And you cannot just “append” before page 1 because there is no “page 0” or “page negative one.” This is because page numbers have meaning and order. Making it even worse, every page must be N records (except the last); This means if there is an update with three records, you can add a new page – and that page only has three rows!
Therefore, to support table-based auto-refresh, there must be a warning before an auto-refresh happens (e.g., a toast) that says something like “Hey – your data will refresh in 15 seconds…” with the option to “approve right now,” “skip” – skipping this specific refresh, or “turn off refresh.”
In contrast, if you are using infinite scroll with auto-refresh, records are appended to the table. So the user doesn’t get their data or position shifted. Moreover, the end-user can approve it or scroll to the top when records are updated. If a user already scrolled to the top, we can update and scroll back to the newest records. So cases 1 and 2 are both solved. It couldn’t be simpler.
And yes, with infinite scroll – if there is “new data,” and a user doesn’t refresh, we must stop loading infinite scroll from backend since the data has changed – but we also put a note saying, “Hey, sorry, can’t let you keep scrolling – data has changed.” But that is small.
Winner: Infinite Scroll
Summary
So while the pagination approach is more traditional, with some careful consideration – infinite scroll is less click-heavy, more seamless, and has higher performance. Moreover, when being careful and considered and focusing on the actual end-user interaction with tables, the value add for infinite scroll becomes clear – and we can see that solution in modern web applications from Twitter to Facebook to Google sheets!
