Skip to main content

Troubleshooting

Quick Diagnosis​

# Check container status
docker ps | grep remove-pii

# Check recent logs
docker logs remove-pii --tail 50 2>&1 | grep -i error

# Test PII removal
curl -X POST http://localhost:8080/events \
  -H "Content-Type: application/json" \
  -d '{"user_name": "John Doe", "email": "[email protected]", "ip": "192.168.1.1"}'

Common Issues​

PII still in output (compliance failure)​

Cause: Field names don't match or nested fields missed

# Check for PII patterns in output
docker logs remove-pii --tail 50 2>&1 | grep -E "@|[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+"

Fix: Add recursive PII detection:

- mapping: |
    root = this.map_each(item -> item.re_replace_all("[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}", "REDACTED"))

Hash values inconsistent​

Cause: Salt not set or varying between instances

Fix: Use consistent salt from environment:

root.email_hash = this.email.hash("sha256", env("HASH_SALT"))

Domain/analytics data lost​

Cause: Deleting entire field instead of extracting useful parts first

Fix: Extract analytics-safe data before removing:

root.email_domain = this.email.split("@").index(1)
root.email_hash = this.email.hash("sha256")
root.email = deleted()

Location generalization too aggressive​

Cause: Removing city/state when only GPS should be removed

Fix: Keep geographic region, remove precise location:

root.location = {
  "city": this.location.city,
  "state": this.location.state,
  "country": this.location.country
  # latitude, longitude, street deleted
}

Still stuck?​

  1. Add debug logging: logger: {level: DEBUG}
  2. Check the Complete Pipeline for reference config
  3. Review Encrypt Data for encryption instead of deletion