TechnicalPig🐷: Create a Paginated API in 1 minute

This example will use Express to set up the API endpoints and will simulate fetching data from a database. We'll paginate the data by page number, using query parameters to allow the client to specify the page and number of items per page.

Step 1: Create helper function to paginate the data

function paginate(data, page_number, page_size) {
  const start_index = (page_number - 1) * page_size
  const end_index = page_number * page_size
  
  return data.slice(start_index, end_index)
}

We want to break the data into chunks and return only the data requested by the client based on the query params received (see step 2 below)

Paginate slices the array to return only the data for the required page.

Step 2: Create endpoint to get paginated items

app.get('/items', (req, res) => {
    const page = parseInt(req.query.page, 10) || 1;
    const limit = parseInt(req.query.limit, 10) || 10;
    const paginatedData = paginate(data, page, limit);
    res.json({
        page,
        limit,
        total: data.length,
        totalPages: Math.ceil(data.length / limit),
        data: paginatedData,
    });
});

• The /items route handles GET requests. It uses query parameters page and limit to determine which page and how many items per page the client requested.

• With these details, it passes them to the paginate function which breaks down the data accordingly.

• The server responds with JSON containing pagination details and the data for the requested page. It is good practice to include pagination details:

  • total: the total items the data contains.

  • totalPages: the total number of pages you can fetch.

  • data: the paginated data based on the query params provided.

That’s it!

This is how you can create a simple paginated api in 2 simple steps!

Best Practices

When building paginated APIs, there are several best practices to consider that can really enhance the performance, usability, and maintainability of your API. Here are some things you can consider:

1. Use Standard Query Params: For consistency and ease of understanding, use standard query parameter names like page for the page number and limit (or pageSize) for the number of items per page.

2. Provide Total Counts and Pagination Details: Include in your API response not only the current page of data but also metadata that provides context, such as:

   • Total number of records

   • Total number of pages

   • Current page number

   • Number of records per page

   This information is crucial for building effective navigational and display tools on the client side.

3. Implement Efficient Data Fetching: Do not fetch more data than necessary. Use your database's built-in capabilities to fetch only the subset of data that you need to display. For SQL databases, this means using LIMIT and OFFSET clauses. For NoSQL databases like DynamoDB, use the pagination mechanisms provided by the database, such as ExclusiveStartKey and LastEvaluatedKey.

4. Avoid Deep Pagination Issues: As you access higher page numbers, performance tends to degrade as the database must count and skip through large records before it can fetch the data for higher pages. This is data-intensive and slow. Consider alternatives such as:

   • Cursor-based pagination: Using a cursor (often a unique identifier from the last item of the current page) instead of page numbers can be more efficient for large datasets.

   • Search and filtering capabilities: Enabling users to search or filter results can reduce the need to paginate through large datasets.

5. Links for Pagination Controls: Include hyperlinks in your API responses for next, prev, first, and last pages. This HATEOAS (Hypermedia as the Engine of Application State) approach makes it easier for clients to navigate through pages without having to construct URLs manually. Example:

{
  "data": [...], // Array of data items
  "meta": {
    "page": 2,
    "limit": 10,
    "totalPages": 100,
    "totalItems": 1000
  },
  "links": {
    "first": "/api/items?page=1&limit=10",
    "prev": "/api/items?page=1&limit=10",
    "next": "/api/items?page=3&limit=10",
    "last": "/api/items?page=100&limit=10"
  }
}

Summary

These best practices will help you create scalable and user-friendly paginated API!