Configure Authenticated HTTP Input

This step configures a rate-limited HTTP server with an API-key check and CORS for browser-based clients. The wildcard CORS origin is a permissive demonstration setting, not a deployment security baseline.

Goal

Create an HTTP entry point for log ingestion that:

• Authenticates clients using API keys
• Rate limits requests to prevent abuse
• Validates request size and timeouts
• Handles CORS for browser-based logging

Configuration

We'll use the http_server input with authentication and performance tuning settings.

1. Basic HTTP Server

First, let's define the basic server structure listening on port 8080.

input:
  http_server:
    address: '0.0.0.0:8080'
    path: /logs/ingest
    timeout: 10s

2. Add Request Controls

Now we add API key authentication and CORS headers.

# Authentication
auth:
  type: header
  header: 'X-API-Key'
  required_value: '${LOG_API_KEY}'

# Browser access policy (permissive demo setting)
cors:
  enabled: true
  allowed_origins: ['*']
  allowed_methods: ['POST']

3. Performance & Protection

Finally, we add rate limiting and size constraints to protect the pipeline.

# Rate Limiting
rate_limit: '1000/1s'

# Resource Protection
max_request_size: 1048576 # 1MB
max_connections: 1000
keepalive: true

Complete Step 1 Configuration

Combine these into a single configuration file.

production-pipeline-step-1.yaml

# Step 1: Authenticated HTTP Input
input:
  http_server:
    address: '0.0.0.0:8080'
    path: /logs/ingest
    timeout: 10s
    rate_limit: '1000/1s'

    # Request authentication
    auth:
      type: header
      header: 'X-API-Key'
      required_value: '${LOG_API_KEY}'

    cors:
      enabled: true
      allowed_origins: ['*']
      allowed_methods: ['POST']

    # Protection
    max_request_size: 1048576
    max_connections: 1000

pipeline:
  processors:
    # Temporary passthrough for testing
    - log:
        level: INFO
        message: 'Received log event'

output:
  stdout: {}

Deployment & Verification

1. Set the API Key:

   export LOG_API_KEY="secret-key-123"

2. Test Access (Success):

   curl -X POST http://localhost:8080/logs/ingest \
     -H "X-API-Key: secret-key-123" \
     -d '{"message": "hello"}'

   Expected: 200 OK

3. Test Access (Failure - Wrong Key):

   curl -X POST http://localhost:8080/logs/ingest \
     -H "X-API-Key: wrong-key" \
     -d '{"message": "hello"}'

   Expected: 401 Unauthorized

Next Steps

Now that the input checks an API key, validate the data that passes through it.

👉 Step 2: Parse & Validate Logs