Guide

Audit AI Agent Activity with CLI Logging

AI agents like Claude Code and GitHub Copilot can send emails, create calendar events, and read contacts on your behalf. Audit logging tracks every command they execute — automatically detecting the agent source, logging the grant and API request, and giving you per-session filtering and compliance exports.

Why audit AI agents?

When AI agents run CLI commands autonomously — sending emails, creating events, reading contacts — you need visibility into what they did. Without audit logging, an agent session is a black box. With it, you get:

  • AI agent tracking — see exactly what Claude Code, GitHub Copilot, or custom MCP agents executed, with automatic source detection
  • Per-session filtering — review what an agent did during a specific coding session by time range and source
  • Compliance and security — SOC 2, HIPAA, and internal policies need evidence of who accessed what, including automated agents
  • CI/CD observability — trace pipeline failures back to the exact command, grant, and API request

Quick setup

One command to initialize and enable:

nylas audit init --enable

This creates the log directory, writes a default configuration, and starts recording. Verify it's active:

nylas audit logs status
Audit logging: enabled
Log path:      ~/.nylas/audit/
Retention:     90 days
Max size:      50 MB
Format:        json

You can also enable and disable logging independently:

# Disable temporarily
nylas audit logs disable

# Re-enable
nylas audit logs enable

What gets logged

Every command execution produces a log entry with these fields:

{
  "timestamp": "2026-03-04T14:32:01Z",
  "command": "nylas email send",
  "grant_id": "grant_abc123",
  "invoker": "qasim",
  "source": "claude-code",
  "request_id": "req_xyz789",
  "status": "success",
  "duration_ms": 1240
}
  • timestamp — ISO 8601, always UTC
  • command — the full command (arguments included, sensitive values redacted)
  • grant_id — which Nylas account was used
  • invoker — the user or system that ran the command
  • source — how the command was invoked: terminal, claude-code, github-actions, mcp, etc.
  • request_id — correlates to the Nylas API request for debugging
  • status — success or error
  • duration_ms — round-trip time in milliseconds

Sensitive data like API keys, access tokens, and email bodies are automatically redacted from log entries.

Tracking AI agent activity

When an AI agent like Claude Code or GitHub Copilot runs CLI commands, the audit log automatically detects the agent as the source via environment variables. No configuration needed — the source field is set automatically based on how the CLI was invoked.

# See everything Claude Code ran today
nylas audit logs show --source claude-code --since "2026-03-04"

# See all AI agent activity in the last week
nylas audit logs show --invoker claude-code --since "2026-02-25"

# Review a specific agent session by time window
nylas audit logs show --source mcp --since "2026-03-04T10:00:00" --until "2026-03-04T12:00:00"

# Filter agent errors only
nylas audit logs show --source claude-code --status error

This is especially useful after an AI coding session — review exactly which emails an agent read, which calendar events it created, and whether any API calls failed. Supported agent sources include claude-code, github-copilot, mcp, and any custom agent that sets the NYLAS_CLI_SOURCE environment variable.

Filtering and viewing logs

Beyond agent filtering, nylas audit logs show supports date ranges, commands, status, and more:

# Show the last 20 entries
nylas audit logs show -n 20

# Filter by date range
nylas audit logs show --since "2026-03-01" --until "2026-03-04"

# Filter by command
nylas audit logs show --command "email send"

# Filter by status
nylas audit logs show --status error

# Combine filters
nylas audit logs show --source github-actions --status error --since "2026-03-01"

Add --json to get machine-readable output for piping into other tools:

nylas audit logs show --json -n 100 | jq '.[] | select(.duration_ms > 5000)'

Summary statistics

Get a high-level overview of audit activity:

nylas audit logs summary --days 30
Audit Log Summary (last 30 days)
─────────────────────────────────
Total commands:    1,247
Success rate:      98.6%
Unique invokers:   4

Top commands:
  email send         412  (33.0%)
  email list         298  (23.9%)
  calendar list      187  (15.0%)
  event create        94  (7.5%)
  contacts list       82  (6.6%)

By source:
  terminal           687  (55.1%)
  claude-code        312  (25.0%)
  github-actions     198  (15.9%)
  mcp                 50  (4.0%)

Error breakdown:
  auth_expired         8
  rate_limited         6
  invalid_grant        3

Add --json to pipe the summary into dashboards or alerting systems:

nylas audit logs summary --days 7 --json

Exporting for compliance

Export logs in CSV or JSON for auditors, SIEM systems, or long-term archival:

# CSV export for a specific quarter
nylas audit export --format csv --since "2026-01-01" --until "2026-03-31" --output Q1-audit.csv

# JSON export for SIEM ingestion
nylas audit export --format json --since "2026-03-01" --output march-audit.json

Automate quarterly exports with a simple script:

#!/bin/bash
# quarterly-audit-export.sh
QUARTER_START="2026-01-01"
QUARTER_END="2026-03-31"
OUTPUT="Q1-2026-audit.csv"

nylas audit export \
  --format csv \
  --since "$QUARTER_START" \
  --until "$QUARTER_END" \
  --output "$OUTPUT"

echo "Exported to $OUTPUT"

CI/CD integration

Enable audit logging in GitHub Actions to track every CLI command your pipelines run:

# .github/workflows/sync.yml
name: Calendar Sync
on:
  schedule:
    - cron: "0 9 * * 1"

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - name: Install Nylas CLI
        run: npm install -g nylas-cli

      - name: Enable audit logging
        run: nylas audit init --enable --no-prompt

      - name: Sync calendars
        run: nylas calendar list --grant-id $GRANT_ID

      - name: Export audit log
        run: nylas audit export --format json --output audit.json

      - name: Upload audit artifact
        uses: actions/upload-artifact@v4
        with:
          name: audit-log
          path: audit.json

After the run, filter for pipeline activity:

nylas audit logs show --source github-actions --since "2026-03-04"

Configuration

View and adjust audit settings:

# View current configuration
nylas audit config show

# Set retention period (days)
nylas audit config set retention 180

# Set max log size (MB)
nylas audit config set max_size 100

# Change log directory
nylas audit config set path /var/log/nylas-audit

You can also set these during initialization:

nylas audit init --retention 180 --max-size 100 --path /var/log/nylas-audit --enable

Next steps