Troubleshooting

Quick Diagnosis

# Check container status
docker ps | grep schema

# Check recent logs
docker logs enforce-schema --tail 50 2>&1 | grep -i error

# Test schema validation
curl -X POST http://localhost:8080/sensors \
  -H "Content-Type: application/json" \
  -d '{"sensor_id": "temp-001", "temperature": 72.5}'

Common Issues

Valid data being rejected

Cause: Schema too strict or type mismatch

# Check validation error messages
docker logs enforce-schema --tail 20 2>&1 | grep validation

Fix: Check for type coercion issues:

# Allow string or number for temperature
"temperature": {
  "oneOf": [
    {"type": "number"},
    {"type": "string", "pattern": "^[0-9.]+$"}
  ]
}

---

Invalid data not being caught

Cause: Schema missing required fields or validation rules

Fix: Add required array and additionalProperties:

"required": ["sensor_id", "temperature", "timestamp"],
"additionalProperties": false

---

DLQ not receiving failed messages

Cause: Switch condition not routing to DLQ

Fix: Ensure error path is configured:

- check: meta("validation_error") != ""
  output:
    kafka: {topic: dlq}
- output:
    kafka: {topic: valid-data}

---

Schema changes breaking pipeline

Cause: Schema updated without version handling

Fix: Support multiple schema versions:

- mapping: |
    root.schema_version = this.schema_version.or("v1")
- switch:
    - check: this.schema_version == "v2"
      processors: [{json_schema: {schema_path: "./schema-v2.json"}}]
    - processors: [{json_schema: {schema_path: "./schema-v1.json"}}]

Still stuck?

1. Add debug logging: logger: {level: DEBUG}
2. Check the Complete Pipeline for reference config
3. Review Remove PII for data cleaning patterns