Source: https://cli.nylas.com/guides/gmail-api-pagination-sync

Guide

# Gmail API Pagination and Sync Without the Hassle

Gmail's REST API requires you to handle pagination tokens, history IDs, and partial sync state yourself. This guide explains how nextPageToken and historyId work, then shows how to skip both with one command. Works with Gmail, Outlook, Exchange, Yahoo, iCloud, and IMAP.

Written by [Prem Keshari](https://cli.nylas.com/authors/prem-keshari) • Senior SRE

Reviewed by [Nick Barraclough](https://cli.nylas.com/authors/nick-barraclough)

Updated April 11, 2026

Verified

 —

CLI

3.1.1

 ·

Gmail

 ·

last tested

April 11, 2026

> **TL;DR:** Gmail's REST API requires you to handle pagination tokens, history IDs, and partial sync state yourself. Or you run one CLI command.

## How Gmail API pagination works

The Gmail API's `messages.list` endpoint returns a maximum of 500 results per request. If your inbox has more messages than that, the response includes a `nextPageToken` — a string you pass back in your next request to get the following page. You loop until `nextPageToken` is absent, which means you've consumed every page.

According to the [Gmail API messages.list documentation](https://developers.google.com/gmail/api/reference/rest/v1/users.messages/list), each call costs 5 quota units, and the default page size is 100 (configurable up to 500 with `maxResults`). Every request requires a valid OAuth2 access token.

Here's what that looks like in Python:

```python
from googleapiclient.discovery import build

service = build("gmail", "v1", credentials=creds)

all_messages = []
page_token = None

while True:
    response = service.users().messages().list(
        userId="me",
        maxResults=500,
        pageToken=page_token,
    ).execute()

    all_messages.extend(response.get("messages", []))
    page_token = response.get("nextPageToken")

    if not page_token:
        break

print(f"Fetched {len(all_messages)} message IDs")
```

That's 18 lines just to collect message IDs. You still need to call `messages.get` on each one to fetch subjects, senders, and bodies — each costing another 5 quota units.

## How Gmail incremental sync works

Full pagination is expensive. If you've already fetched your inbox once, you don't want to re-download everything. Gmail solves this with `history.list` and `historyId`.

Every change in a Gmail mailbox — new messages, deletions, label additions, label removals — gets a monotonically increasing `historyId`. You store the `historyId` from your last sync. On the next run, you call `history.list` with `startHistoryId` to get only what changed. The [Gmail sync guide](https://developers.google.com/gmail/api/guides/sync) recommends this as the primary approach for keeping a local copy in sync.

The `historyTypes` parameter lets you filter by change type: `messageAdded`, `messageDeleted`, `labelAdded`, and `labelRemoved`. Each `history.list` call costs 2 quota units.

```python
def get_changes_since(service, start_history_id):
    """Fetch all mailbox changes since the given historyId."""
    changes = []
    page_token = None

    while True:
        response = service.users().history().list(
            userId="me",
            startHistoryId=start_history_id,
            historyTypes=["messageAdded", "messageDeleted"],
            pageToken=page_token,
        ).execute()

        changes.extend(response.get("history", []))
        page_token = response.get("nextPageToken")

        if not page_token:
            break

    new_history_id = response.get("historyId")
    return changes, new_history_id
```

There's a catch: history IDs expire after roughly 30 days. If your stored `historyId` is too old, `history.list` returns a `404 Not Found` (or sometimes `410 Gone`), and you need to fall back to a full sync. Your code needs to handle both paths.

## The problems with doing it yourself

Pagination and incremental sync sound straightforward in isolation. In production, the edge cases stack up:

- **OAuth2 token management** — Gmail access tokens expire every 3,600 seconds. Your sync loop needs to detect expired tokens, refresh them using the refresh token, and retry the failed request. That's a token refresh callback, error handling, and retry logic.
- **Expired historyId fallback** — When `history.list` returns 404, you need to drop your delta sync and run a full pagination sync instead. Two code paths, both need to work correctly.
- **Rate limiting** — Gmail enforces 250 quota units per user per second. A `messages.list` call costs 5 units, a `messages.get` costs 5 units, and a `history.list` costs 2 units. If you're syncing a large mailbox, you need client-side throttling and exponential backoff on `429 Too Many Requests`.
- **Partial page failures** — A network error mid-pagination means you have half your results. Do you retry from the beginning or from the last page token? You need to track state.
- **OAuth2 setup overhead** — Before writing any code, you need a Google Cloud project, an OAuth consent screen, a client ID and secret, and a redirect URI configured in [console.cloud.google.com](https://console.cloud.google.com/). That's 15-20 minutes of clicking through web forms.

A reliable sync loop with token refresh, pagination, incremental delta via historyId, expired-history fallback, rate limit handling, and error recovery runs 80-120 lines of Python. And that's before you add logging, persistence, or multi-account support.

## List Gmail emails with one command

Nylas CLI handles pagination, OAuth2, and token refresh internally. You don't write a loop. You don't manage tokens. You run one command:

```bash
# List the 50 most recent emails
nylas email list --limit 50 --json
```

```bash
# Filter by subject
nylas email list --subject "invoice" --json
```

```bash
# Filter by sender
nylas email list --from "boss@company.com" --json
```

The CLI paginates through the provider's API behind the scenes, refreshes expired OAuth2 tokens automatically, and returns the results as JSON. No Google Cloud project, no consent screen, no redirect URI.

Install with Homebrew and authenticate once:

```bash
brew install nylas/nylas-cli/nylas
nylas auth login
```

For other install methods, see the [getting started guide](https://cli.nylas.com/guides/getting-started).

## Search and filter

Gmail's API supports a `q` parameter on `messages.list` that accepts the same query syntax as the Gmail search box. With the API, you still need the pagination loop, OAuth2 setup, and token management. With the CLI, you skip all of that:

```bash
# Full-text search
nylas email search "quarterly report" --json
```

```bash
# Unread emails only
nylas email list --unread --json
```

```bash
# Emails from the last 7 days
nylas email list --received-after 2026-03-26 --json
```

Compare that to the API equivalent, which requires building the query string, passing it to `messages.list`, paginating through results, and calling `messages.get` on each message ID to get the actual content. The CLI collapses that into a single line.

## Side-by-side comparison

| Capability | Gmail API (Python) | Nylas CLI |
| --- | --- | --- |
| Pagination | Manual `nextPageToken` loop | Handled internally |
| Incremental sync | `history.list` + `historyId` tracking | Handled internally |
| Authentication | GCP project + OAuth consent screen + token refresh | `nylas auth login` (one time) |
| Token expiration | 3,600s — manual refresh with callback | Automatic refresh |
| Rate limits | 250 units/sec — manual throttling + backoff | Managed internally |
| Error recovery | Handle 404, 410, 429, token errors | Built-in retry logic |
| Search | `q` param + pagination loop | `nylas email search "query"` |
| Setup time | 15-20 min (GCP console) + 80-120 lines code | 2 min install + auth |
| Multi-provider | Gmail only | Gmail, Outlook, Exchange, Yahoo, iCloud, IMAP |

## Frequently asked questions

### What is nextPageToken in the Gmail API?

When you call `messages.list`, the Gmail API returns up to 500 results per page. If more messages exist, the response includes a `nextPageToken` string. You pass that token as the `pageToken` parameter in your next request to fetch the following page. You keep looping until the response no longer contains a `nextPageToken`, which means you've reached the end.

### How does Gmail incremental sync work with historyId?

Every change in a Gmail mailbox — new messages, deletions, label changes — gets a monotonically increasing `historyId`. You store the `historyId` from your last sync, then call `history.list` with `startHistoryId` to get only the changes since then. History IDs expire after roughly 30 days. If your stored ID is too old, the API returns a 404 and you need a full sync fallback.

### Can I list Gmail emails without setting up Google Cloud?

Yes. Nylas CLI handles OAuth2 and provider authentication internally. Run `nylas email list --limit 50 --json` to list your Gmail inbox without creating a Google Cloud project, configuring an OAuth consent screen, or managing access tokens. The CLI works the same way across six providers.

### Does the CLI handle Gmail API rate limits?

Yes. The Gmail API enforces 250 quota units per user per second, and a `messages.list` call costs 5 units. The CLI manages rate limiting, pagination, token refresh, and retry logic internally. You get the results without writing any quota-tracking or backoff code.

## Next steps

- [List Gmail emails from the command line](https://cli.nylas.com/guides/list-gmail-emails) — filter by sender, subject, date, and read status
- [Gmail API incremental sync with historyId and ETags](https://cli.nylas.com/guides/if-match-gmail-api) — deep dive into ETags, If-Match, and 412 handling
- [Send email from the terminal](https://cli.nylas.com/guides/send-email-from-terminal) — covers bash, zsh, and cross-platform workflows
- [Full command reference](https://cli.nylas.com/docs/commands) — every flag and subcommand documented