Finseo

Cloudflare Server Log Access

Access server logs from Cloudflare's global CDN network for comprehensive bot analytics.

About Cloudflare

Cloudflare is a global content delivery network (CDN) and security company that provides DDoS protection, web application firewall, and performance optimization. Access to raw logs requires an Enterprise plan or creative workarounds.

CDN Provider
Security Platform
Global Network

Overview

Cloudflare processes billions of requests daily through its global network. While they provide excellent analytics dashboards, accessing raw server logs for detailed bot analysis requires specific plan features or alternative approaches.

Key Insight

Cloudflare's edge servers process requests before they reach your origin server. This means Cloudflare logs capture bot traffic that your origin server might never see, providing a more complete picture of AI bot visits.

Enterprise Features

For Enterprise customers, Cloudflare offers comprehensive log access:

Method 1: Logpull API

Programmatically retrieve logs for a specific time range:

  1. Generate API credentials in Cloudflare dashboard
  2. Use the Logpull endpoint with your Zone ID
  3. Specify time range (up to 1 hour per request)
  4. Download logs in JSON or CSV format
curl -X GET "https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/received" \
  -H "X-Auth-Email: your@email.com" \
  -H "X-Auth-Key: your-api-key"

Method 2: Logpush

Automatically push logs to your storage destination:

  • Configure destination (S3, GCS, Azure, etc.)
  • Set up real-time or batched delivery
  • Filter specific fields or events
  • Process logs with your own analytics tools
View Cloudflare Logs Documentation

Alternative Methods

For non-Enterprise plans, consider these alternatives:

Analytics API

Access aggregated data through GraphQL API (limited bot details)

Firewall Events

View bot-related events in Security Analytics

Origin Server Logs

Use your web server's logs (misses CDN-cached requests)

Workers Analytics

Custom logging with Cloudflare Workers (see below)

Cloudflare Workers Solution

Create a custom logging solution using Cloudflare Workers (available on all paid plans):

Implementation Steps:

  1. Create a Worker

    Deploy a Worker that intercepts requests and logs bot user agents

  2. Store Logs

    Use Workers KV, Durable Objects, or external storage

  3. Filter Bot Traffic

    Identify AI bots by User-Agent patterns

  4. Export Data

    Create an endpoint to download logs for Bot Analytics

Important Notes

Rate Limits: Logpull API has rate limits. Plan your requests accordingly

Data Retention: Logs are available for 72 hours (3 days) on Enterprise plans

Log Format: Cloudflare logs include unique fields like edge location and cache status

Privacy: Cloudflare can anonymize IPs in logs for GDPR compliance

Troubleshooting

No access to Enterprise features?

Contact Cloudflare sales to discuss Enterprise plans, or implement the Workers solution for custom logging on any paid plan.

Missing bot traffic in analytics?

Check your Bot Management settings. Some bots might be blocked before logging. Review Firewall Rules and Bot Fight Mode settings.

API authentication issues?

Use API Tokens instead of Global API Key for better security. Ensure your token has the necessary permissions for log access.

Next Steps

Once you have access to Cloudflare logs, you can:

  • Track AI bot visits across your entire CDN-cached content
  • See geographic distribution of AI bot traffic
  • Analyze cache hit rates for bot requests
  • Identify bot behavior patterns and optimize accordingly
Upload to Bot AnalyticsBack to Overview