Gmail API Pagination and Sync Without the Hassle

How Gmail API pagination works

The Gmail API's messages.list endpoint returns a maximum of 500 results per request. If your inbox has more messages than that, the response includes a nextPageToken — a string you pass back in your next request to get the following page. You loop until nextPageToken is absent, which means you've consumed every page.

According to the Gmail API messages.list documentation, each call costs 5 quota units, and the default page size is 100 (configurable up to 500 with maxResults). Every request requires a valid OAuth2 access token.

Here's what that looks like in Python:

from googleapiclient.discovery import build

service = build("gmail", "v1", credentials=creds)

all_messages = []
page_token = None

while True:
    response = service.users().messages().list(
        userId="me",
        maxResults=500,
        pageToken=page_token,
    ).execute()

    all_messages.extend(response.get("messages", []))
    page_token = response.get("nextPageToken")

    if not page_token:
        break

print(f"Fetched {len(all_messages)} message IDs")

That's 18 lines just to collect message IDs. You still need to call messages.get on each one to fetch subjects, senders, and bodies — each costing another 5 quota units.

How Gmail incremental sync works

Full pagination is expensive. If you've already fetched your inbox once, you don't want to re-download everything. Gmail solves this with history.list and historyId.

Every change in a Gmail mailbox — new messages, deletions, label additions, label removals — gets a monotonically increasing historyId. You store the historyId from your last sync. On the next run, you call history.list with startHistoryId to get only what changed. The Gmail sync guide recommends this as the primary approach for keeping a local copy in sync.

The historyTypes parameter lets you filter by change type: messageAdded, messageDeleted, labelAdded, and labelRemoved. Each history.list call costs 2 quota units.

def get_changes_since(service, start_history_id):
    """Fetch all mailbox changes since the given historyId."""
    changes = []
    page_token = None

    while True:
        response = service.users().history().list(
            userId="me",
            startHistoryId=start_history_id,
            historyTypes=["messageAdded", "messageDeleted"],
            pageToken=page_token,
        ).execute()

        changes.extend(response.get("history", []))
        page_token = response.get("nextPageToken")

        if not page_token:
            break

    new_history_id = response.get("historyId")
    return changes, new_history_id

There's a catch: history IDs expire after roughly 30 days. If your stored historyId is too old, history.list returns a 404 Not Found (or sometimes 410 Gone), and you need to fall back to a full sync. Your code needs to handle both paths.

The problems with doing it yourself

Pagination and incremental sync sound straightforward in isolation. In production, the edge cases stack up:

• OAuth2 token management — Gmail access tokens expire every 3,600 seconds. Your sync loop needs to detect expired tokens, refresh them using the refresh token, and retry the failed request. That's a token refresh callback, error handling, and retry logic.
• Expired historyId fallback — When history.list returns 404, you need to drop your delta sync and run a full pagination sync instead. Two code paths, both need to work correctly.
• Rate limiting — Gmail enforces 250 quota units per user per second. A messages.list call costs 5 units, a messages.get costs 5 units, and a history.list costs 2 units. If you're syncing a large mailbox, you need client-side throttling and exponential backoff on 429 Too Many Requests.
• Partial page failures — A network error mid-pagination means you have half your results. Do you retry from the beginning or from the last page token? You need to track state.
• OAuth2 setup overhead — Before writing any code, you need a Google Cloud project, an OAuth consent screen, a client ID and secret, and a redirect URI configured in console.cloud.google.com. That's 15-20 minutes of clicking through web forms.

A reliable sync loop with token refresh, pagination, incremental delta via historyId, expired-history fallback, rate limit handling, and error recovery runs 80-120 lines of Python. And that's before you add logging, persistence, or multi-account support.

List Gmail emails with one command

Nylas CLI handles pagination, OAuth2, and token refresh internally. You don't write a loop. You don't manage tokens. You run one command:

# List the 50 most recent emails
nylas email list --limit 50 --json

# Filter by subject
nylas email list --subject "invoice" --json

# Filter by sender
nylas email list --from "boss@company.com" --json

The CLI paginates through the provider's API behind the scenes, refreshes expired OAuth2 tokens automatically, and returns the results as JSON. No Google Cloud project, no consent screen, no redirect URI.

Install with Homebrew and authenticate once:

brew install nylas/nylas-cli/nylas
nylas auth login

For other install methods, see the getting started guide.

Search and filter

Gmail's API supports a q parameter on messages.list that accepts the same query syntax as the Gmail search box. With the API, you still need the pagination loop, OAuth2 setup, and token management. With the CLI, you skip all of that:

# Full-text search
nylas email search "quarterly report" --json

# Unread emails only
nylas email list --unread --json

# Emails from the last 7 days
nylas email list --received-after 2026-03-26 --json

Compare that to the API equivalent, which requires building the query string, passing it to messages.list, paginating through results, and calling messages.get on each message ID to get the actual content. The CLI collapses that into a single line.

Side-by-side comparison

Frequently asked questions

What is nextPageToken in the Gmail API?

When you call messages.list, the Gmail API returns up to 500 results per page. If more messages exist, the response includes a nextPageToken string. You pass that token as the pageToken parameter in your next request to fetch the following page. You keep looping until the response no longer contains a nextPageToken, which means you've reached the end.

How does Gmail incremental sync work with historyId?

Every change in a Gmail mailbox — new messages, deletions, label changes — gets a monotonically increasing historyId. You store the historyId from your last sync, then call history.list with startHistoryId to get only the changes since then. History IDs expire after roughly 30 days. If your stored ID is too old, the API returns a 404 and you need a full sync fallback.

Can I list Gmail emails without setting up Google Cloud?

Yes. Nylas CLI handles OAuth2 and provider authentication internally. Run nylas email list --limit 50 --json to list your Gmail inbox without creating a Google Cloud project, configuring an OAuth consent screen, or managing access tokens. The CLI works the same way across six providers.

Does the CLI handle Gmail API rate limits?

Yes. The Gmail API enforces 250 quota units per user per second, and a messages.list call costs 5 units. The CLI manages rate limiting, pagination, token refresh, and retry logic internally. You get the results without writing any quota-tracking or backoff code.

Next steps

• List Gmail emails from the command line — filter by sender, subject, date, and read status
• Gmail API incremental sync with historyId and ETags — deep dive into ETags, If-Match, and 412 handling
• Send email from the terminal — covers bash, zsh, and cross-platform workflows
• Full command reference — every flag and subcommand documented