Skip to main content

Troubleshooting

Quick Diagnosis​

# Check container status
docker ps | grep parse-logs

# Check recent logs
docker logs parse-logs --tail 50 2>&1 | grep -i error

# Test log parsing
curl -X POST http://localhost:8080/logs \
  -H "Content-Type: application/json" \
  -d '{"level": "INFO", "message": "test"}'

Common Issues​

Format not detected​

Cause: Unknown format or malformed input

# Check what's being received
docker logs parse-logs --tail 20 2>&1 | grep -i "format\|parse"

Fix: Add format detection fallback:

Partial snippet
- mapping: |
    root = this.parse_json().catch({
      "raw": content(),
      "format": "unknown",
      "parse_error": error()
    })

JSON parsing fails on valid JSON​

Cause: Content-Type header missing or wrong

Fix: Parse based on content, not header:

Partial snippet
- mapping: |
    root = if content().has_prefix("{") {
      this.parse_json()
    } else {
      {"raw": content()}
    }

Access log regex not matching​

Cause: Non-standard log format

Fix: Make regex more flexible:

Partial snippet
# Standard combined log format
- mapping: |
    let parts = this.re_find_all_submatch("^(\\S+) \\S+ \\S+ \\[([^]]+)\\] \"([^\"]+)\" (\\d+) (\\d+)")
    root.ip = $parts.index(0).index(1)
    root.timestamp = $parts.index(0).index(2)
    root.request = $parts.index(0).index(3)

Syslog parsing incomplete​

Cause: RFC 3164 vs RFC 5424 format mismatch

Fix: Support both formats:

Partial snippet
Partial snippet: validated RFC 3164 parser
- grok:
    expressions:
      - '<%{POSINT:priority}>%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:hostname} %{DATA:tag}: %{GREEDYDATA:message}'
    named_captures_only: true

Still stuck?​

  1. Add debug logging: logger: {level: DEBUG}
  2. Check the Complete Pipeline for reference config
  3. Review Transform Formats for format conversion