Troubleshooting

Quick Diagnosis

# Check container status
docker ps | grep schema

# Check recent logs
docker logs enforce-schema --tail 50 2>&1 | grep -i error

# Test schema validation
curl -X POST http://localhost:8080/sensors \
  -H "Content-Type: application/json" \
  -d '{"sensor_id": "temp-001", "temperature": 72.5}'

Common Issues

Valid data being rejected

Cause: Schema too strict or type mismatch

# Check validation error messages
docker logs enforce-schema --tail 20 2>&1 | grep validation

Fix: Check for type coercion issues:

# Allow string or number for temperature
'temperature':
  {
    'oneOf':
      [{ 'type': 'number' }, { 'type': 'string', 'pattern': '^[0-9.]+$' }],
  }

---

Invalid data not being caught

Cause: Schema missing required fields or validation rules

Fix: Add required array and additionalProperties:

"required": ["sensor_id", "temperature", "timestamp"],
"additionalProperties": false

---

DLQ not receiving failed messages

Cause: Switch condition not routing to DLQ

Fix: Ensure error path is configured:

- check: meta("validation_error") != ""
  output:
    kafka: { topic: dlq }
- output:
    kafka: { topic: valid-data }

---

Schema changes breaking pipeline

Cause: Schema updated without version handling

Fix: Support multiple schema versions:

- mapping: |
    root.schema_version = this.schema_version.or("v1")
- switch:
    - check: this.schema_version == "v2"
      processors: [{ json_schema: { schema_path: './schema-v2.json' } }]
    - processors: [{ json_schema: { schema_path: './schema-v1.json' } }]

Still stuck?

1. Add debug logging: logger: {level: DEBUG}
2. Check the Complete Pipeline for reference config
3. Review Remove PII for data cleaning patterns