Skip to main content

Inspect PII removal

Follow a synthetic purchase record as selected fields are deleted, hashed, pseudonymized, or generalized.

How to use this explorer

Move between stages with the arrows or the numbered stage list. Each stage shows the input it receives on the left and the output it produces on the right, with the lines that changed marked. The configuration that makes the change sits below the comparison, and the final stage shows the complete pipeline.

The 6 stages in order

  1. Step 1: Original Input
    Start with a synthetic purchase record containing payment, identity, network, and location fields.
  2. Step 2: Delete Payment Data
    Delete the full card number and expiry while retaining the last four digits.
  3. Step 3: Hash IP Address
    Replace the source IP address with a keyed, hex-encoded HMAC-SHA-256 value.
  4. Step 4: Hash Email
    Replace the email with a keyed hash and retain its domain as a separate field.
  5. Step 5: Pseudonymize User
    Replace the name with a stable pseudonymous identifier derived from the input.
  6. Step 6: Generalize Location
    Remove precise coordinates while retaining the fixture city and country.

Interactive pipeline

PII removal pipeline

Six-stage curated walkthrough

Stage 1 of 6

Original Input

Start with a synthetic purchase record containing payment, identity, network, and location fields.

  • 0Added
  • 0Updated
  • 0Removed
Copy & download

Input

{"event_id": "evt_20240115_1030","timestamp": "2024-01-15T10:30:00Z","event_type": "purchase","user_name": "Sarah Johnson","email": "[email protected]","ip_address": "192.168.1.100","payment_method": {"type": "credit_card","full_number": "4532-1234-5678-9010","expiry": "12/25","last_four": "9010"},"location": {"latitude": 37.7749,"longitude": -122.4194,"city": "San Francisco","country": "USA"},"purchase_amount": 49.99,"currency": "USD"}

Output

{"event_id": "evt_20240115_1030","timestamp": "2024-01-15T10:30:00Z","event_type": "purchase","user_name": "Sarah Johnson","email": "[email protected]","ip_address": "192.168.1.100","payment_method": {"type": "credit_card","full_number": "4532-1234-5678-9010","expiry": "12/25","last_four": "9010"},"location": {"latitude": 37.7749,"longitude": -122.4194,"city": "San Francisco","country": "USA"},"purchase_amount": 49.99,"currency": "USD"}
Partial snippetremove-pii-complete.yaml#config-input
input:
  http_server:
    address: '0.0.0.0:8080'
    path: /events/ingest
    allowed_verbs: [POST]
    timeout: 30s

# Five field transformations. Review these choices for your own data model.