Guide

Audit AI Agent Activity (Claude, Copilot, MCP)

Audit logs for AI agent actions across Claude Code, GitHub Copilot, and MCP. Filter by source, export for compliance, and surface the commands agents are running with request IDs and per-session detail.

Written by Caleb Geene Director, Site Reliability Engineering

Reviewed by Hazik

Verified — CLI 3.1.1 · Gmail, Outlook · last tested April 11, 2026

How do you audit AI agent activity?

AI agent audit logging is a single-command setup that records every CLI action an autonomous agent takes. Run nylas audit init --enable and every command that Claude Code, GitHub Copilot, or any MCP agent executes is logged automatically — the agent source, the grant used, the API request ID, and the outcome. Each log entry captures 8 fields including the round-trip duration in milliseconds. No code changes, no wrappers, one command to start.

Why do AI agents need audit logs?

AI agents need audit logs because they run commands autonomously and you need a verifiable record of every action they took. The NIST AI Risk Management Framework's GOVERN function explicitly requires audit trails for autonomous systems, and SOC 2 Type II auditors expect log retention of at least 90 days. Without audit logging, an agent session is a black box — you can't prove what was accessed, changed, or sent. With audit logs, you get:

  • AI agent tracking — see exactly what Claude Code, GitHub Copilot, or custom MCP agents executed, with automatic source detection
  • Per-session filtering — review what an agent did during a specific coding session by time range and source
  • Compliance and security — SOC 2, HIPAA, and internal policies need evidence of who accessed what, including automated agents
  • CI/CD observability — trace pipeline failures back to the exact command, grant, and API request

How do I implement AI agent audit logs?

You implement AI agent audit logs by running a single CLI command that initializes local logging, auto-detects agent sources like Claude Code and GitHub Copilot, and starts recording every command execution with timestamps, grant IDs, and outcomes. Here is how to initialize and enable:

nylas audit init --enable

The init command creates the log directory at ~/.config/nylas/audit.log, writes a default configuration with 90-day retention and a 50 MB size cap, and starts recording immediately. Run nylas audit logs status to verify the system is active and confirm the log path, retention period, and output format:

nylas audit logs status
Audit logging: enabled
Log path:      ~/.config/nylas/audit.log
Retention:     90 days
Max size:      50 MB
Format:        json

Logging can be toggled without losing your configuration or existing log data. Disabling pauses recording but keeps the log file and settings intact. Re-enabling resumes from where you left off, with no gap in the audit trail beyond the disabled window:

# Disable temporarily
nylas audit logs disable

# Re-enable
nylas audit logs enable

What gets logged

Each audit log entry is a structured JSON object containing 8 fields that capture the full context of a CLI command execution. The entry records who ran the command, which agent source triggered it, the Nylas grant used, and the round-trip duration in milliseconds. Sensitive values like API keys and email bodies are automatically redacted before the entry is written to disk.

Here's an example log entry showing a successful email send by Claude Code. Therequest_id field correlates directly to the Nylas API request, so you can trace any entry from the local audit log through to the server-side request:

{
  "timestamp": "2026-03-04T14:32:01Z",
  "command": "nylas email send",
  "grant_id": "grant_abc123",
  "invoker": "qasim",
  "source": "claude-code",
  "request_id": "req_xyz789",
  "status": "success",
  "duration_ms": 1240
}
  • timestamp — ISO 8601, always UTC
  • command — the full command (arguments included, sensitive values redacted)
  • grant_id — which Nylas account was used
  • invoker — the user or system that ran the command
  • source — how the command was invoked: terminal, claude-code, github-actions, mcp, etc.
  • request_id — correlates to the Nylas API request for debugging
  • status — success or error
  • duration_ms — round-trip time in milliseconds

API keys, access tokens, and email bodies are automatically redacted. The log captures only the command structure, flags, and metadata — never the content being sent or read.

How do I view Claude Code audit logs?

Claude Code audit logs are filtered using --source claude-code on thenylas audit logs show command. The audit system auto-detects when Claude Code invokes CLI commands via environment variables — no configuration needed. Thesource field is set automatically based on how the CLI was invoked, covering 4 built-in sources: claude-code, github-copilot,github-actions, and mcp.

The examples below show common filtering patterns. You can filter by source, time range, and status — or combine all three to isolate a specific agent session. Each filter flag narrows the result set, so adding --status error to a source filter shows only the failed commands from that agent:

# See everything Claude Code ran today
nylas audit logs show --source claude-code --since "2026-03-04"

# See all AI agent activity in the last week
nylas audit logs show --invoker claude-code --since "2026-02-25"

# Review a specific agent session by time window
nylas audit logs show --source mcp --since "2026-03-04T10:00:00" --until "2026-03-04T12:00:00"

# Filter agent errors only
nylas audit logs show --source claude-code --status error

After an AI coding session, these filters let you review exactly which emails an agent read, which calendar events it created, and whether any API calls failed. Custom agents that aren't auto-detected can set the NYLAS_CLI_SOURCE environment variable to identify themselves in the audit log.

How do I filter AI agent audit logs?

Filtering AI agent audit logs means narrowing the full log down to specific entries using flags on nylas audit logs show. You can filter by date range, command name, agent source, or status. The command supports 5 filter flags (-n, --since, --until,--command, --status) that stack together to narrow results.

Each filter is additive — combining --source with --status errorshows only failed commands from that specific agent. The -n flag limits output count, which is useful when you want a quick sample rather than the full log:

# Show the last 20 entries
nylas audit logs show -n 20

# Filter by date range
nylas audit logs show --since "2026-03-01" --until "2026-03-04"

# Filter by command
nylas audit logs show --command "email send"

# Filter by status
nylas audit logs show --status error

# Combine filters
nylas audit logs show --source github-actions --status error --since "2026-03-01"

Adding --json outputs each log entry as a JSON object, which lets you pipe results into jq, Python scripts, or monitoring tools. The example below filters for commands that took longer than 5000ms — useful for spotting slow API calls or rate-limited requests:

nylas audit logs show --json -n 100 | jq '.[] | select(.duration_ms > 5000)'

Summary statistics

Summary statistics aggregate your audit log into a single report showing total commands, success rate, top commands by frequency, and a breakdown by agent source. The summary covers any time window you specify — --days 30 for a monthly view or--days 7 for a weekly health check. The output includes the error breakdown by type, which surfaces patterns like recurring auth_expired failures.

Running a 30-day summary typically processes over 1,000 log entries in under 200ms since the log is a local JSON file. The command reads from ~/.config/nylas/audit.logand computes aggregates in memory:

nylas audit logs summary --days 30
Audit Log Summary (last 30 days)
─────────────────────────────────
Total commands:    1,247
Success rate:      98.6%
Unique invokers:   4

Top commands:
  email send         412  (33.0%)
  email list         298  (23.9%)
  calendar list      187  (15.0%)
  event create        94  (7.5%)
  contacts list       82  (6.6%)

By source:
  terminal           687  (55.1%)
  claude-code        312  (25.0%)
  github-actions     198  (15.9%)
  mcp                 50  (4.0%)

Error breakdown:
  auth_expired         8
  rate_limited         6
  invalid_grant        3

Adding --json to the summary command outputs the same aggregates as structured JSON. This is useful for piping into dashboards, Slack alerts, or custom monitoring scripts that track week-over-week changes in agent error rates:

nylas audit logs summary --days 7 --json

How do I export AI agent audit logs for compliance?

You export AI agent audit logs for compliance by running nylas audit exportwith a date range and format flag. The CLI outputs CSV for auditors or JSON for SIEM ingestion into Splunk, Datadog, or any system that accepts structured logs.

# CSV export for a specific quarter
nylas audit export --format csv --since "2026-01-01" --until "2026-03-31" --output Q1-audit.csv

# JSON export for SIEM ingestion
nylas audit export --format json --since "2026-03-01" --output march-audit.json

SOC 2 Type II audits typically require 90 days of continuous audit trail coverage. You can automate quarterly exports with a shell script that runs on a cron schedule or as a CI/CD step at the end of each quarter:

#!/bin/bash
# quarterly-audit-export.sh
QUARTER_START="2026-01-01"
QUARTER_END="2026-03-31"
OUTPUT="Q1-2026-audit.csv"

nylas audit export \
  --format csv \
  --since "$QUARTER_START" \
  --until "$QUARTER_END" \
  --output "$OUTPUT"

echo "Exported to $OUTPUT"

How do I add audit logging to CI/CD pipelines?

CI/CD audit logging means running nylas audit init --enable --no-prompt as an early step in your workflow so every subsequent CLI command in the pipeline is logged automatically. The --no-prompt flag skips interactive confirmation, which is required for non-interactive environments like GitHub Actions runners. You can export the log as a build artifact for post-run review — the exported JSON file is typically under 1 MB for pipelines running fewer than 500 commands.

# .github/workflows/sync.yml
name: Calendar Sync
on:
  schedule:
    - cron: "0 9 * * 1"

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - name: Install Nylas CLI
        run: npm install -g nylas-cli

      - name: Enable audit logging
        run: nylas audit init --enable --no-prompt

      - name: Sync calendars
        run: nylas calendar list --grant-id $GRANT_ID

      - name: Export audit log
        run: nylas audit export --format json --output audit.json

      - name: Upload audit artifact
        uses: actions/upload-artifact@v4
        with:
          name: audit-log
          path: audit.json

After the workflow completes, the audit artifact contains every CLI command the pipeline ran. You can also query pipeline activity directly from your local machine by filtering on the github-actions source, which the CLI sets automatically when it detects the GITHUB_ACTIONS environment variable:

nylas audit logs show --source github-actions --since "2026-03-04"

Configuration

Audit configuration controls 3 settings: retention period (default 90 days), maximum log file size (default 50 MB), and the log directory path (default~/.config/nylas/audit.log). Each setting can be changed individually withnylas audit config set or all at once during initialization. Changes take effect immediately — no restart or re-initialization needed.

The nylas audit config show command displays the current values for all three settings. Use nylas audit config set with the setting name and new value to update any individual setting without affecting the others:

# View current configuration
nylas audit config show

# Set retention period (days)
nylas audit config set retention 180

# Set max log size (MB)
nylas audit config set max_size 100

# Change log directory
nylas audit config set path /var/log/nylas-audit

All three settings can also be passed as flags during nylas audit init, which is useful for provisioning machines with a standard audit configuration. This single command sets a 180-day retention period, a 100 MB size cap, and a custom log path:

nylas audit init --retention 180 --max-size 100 --path /var/log/nylas-audit --enable

How do I monitor AI coding assistant activity?

Monitoring AI coding assistant activity means combining audit logs with summary statistics to catch anomalies early — a spike in error rates, an unexpected agent source, or commands running outside business hours. A healthy baseline is a 98%+ success rate across all sources. If the error rate for any agent source exceeds 5% in a 7-day window, that's typically a sign of expired grants or rate limiting.

Use nylas audit logs summary --days 7 for a weekly health check. The jq examples below show how to count errors from the last 24 hours and group commands by type per agent — both useful for automated alerting:

# Weekly health check
nylas audit logs summary --days 7

# Alert on high error rates
nylas audit logs show --status error --since "$(date -v-1d +%Y-%m-%d)" --json | jq length

# Monitor specific agents
nylas audit logs show --source claude-code --json | jq 'group_by(.command) | map({command: .[0].command, count: length})'

Frequently asked questions

These are the most common questions about AI agent audit logging with the Nylas CLI, covering performance impact, automatic agent detection, data redaction, SIEM integration, and the default 90-day retention policy.

What are AI agent audit logs?
AI agent audit logs are structured records of every action an AI coding assistant takes — commands executed, APIs called, grants used, and outcomes. They provide a verifiable trail for security reviews, compliance audits, and debugging.
Does audit logging affect CLI performance?
No. Logs are written asynchronously to a local file. Typical overhead is under 1ms per command.
What AI agents are detected automatically?
Claude Code, GitHub Copilot, GitHub Actions, and any MCP-based agent. Custom agents can set the NYLAS_CLI_SOURCE environment variable to identify themselves.
Are email bodies or API keys logged?
No. Sensitive data is automatically redacted. Only the command name, flags, grant ID, status, and timing are recorded.
Can I send audit logs to a SIEM?
Yes. Use nylas audit export --format json to generate files for ingestion into Splunk, Datadog, or any SIEM that accepts JSON.
How long are logs retained?
Default is 90 days. Change with nylas audit config set retention 180 for up to a year.

Next steps