Skip to main content

Expanso example

Remove PII

Level
intermediate
Evidence
Runs offline
Time
3 min to inspect
Verified
2026-07-18

The problem

Raw event streams often contain more personal data than downstream analytics actually needs.

How Expanso solves it

Delete, hash, pseudonymize, and generalize selected fields in synthetic records.

Next steps

Run and deploy

Run the checked example

Use the checked-in fixture and compare the result with the expected output before changing the pipeline.

Open run instructions

Deploy it

Add the authentication, TLS, storage, network policy, and least-privilege settings required by your target before you deploy this pipeline.

Open deployment requirements

Run your first result​

Remove sensitive fields before forwarding an event by explicitly selecting the fields your destination needs. This first run keeps only the event ID and action from one synthetic event, then prints that smaller record locally.

You need Expanso Edge and the CLI installed and two terminals. No Cloud account or external data service is required. Use only synthetic data for this first check.

In terminal one, start a dedicated local node. Keep it open to see the output:

expanso-edge run --local --no-watch --api-listen 127.0.0.1:19031 --data-dir ./expanso-first-result-data

In terminal two, download and inspect the job, validate its pipeline configuration, then submit it explicitly to the local node:

curl -fsSLo remove-pii.yaml https://examples.expanso.io/files/first-results/remove-pii.yaml
cat remove-pii.yaml
expanso-edge validate remove-pii.yaml
expanso-cli --endpoint http://127.0.0.1:19031 job deploy remove-pii.yaml

Validation checks pipeline syntax and components. Submission alone is not success: confirm the JSON result in terminal one.

The complete job​

The generated input is finite: one batch, then completion.

Download the job file
Complete pipeline
name: first-result-remove-pii
type: pipeline
config:
  input:
    generate:
      count: 1
      interval: 1s
      mapping: |
        root = {"event_id":"demo-001","email":"[email protected]","ip_address":"192.0.2.10","payment_card":"SYNTHETIC-CARD","action":"checkout"}
  pipeline:
    processors:
      - mapping: |
          root.event_id = this.event_id
          root.action = this.action
  output:
    stdout:
      codec: lines

Input​

{
  "event_id": "demo-001",
  "email": "[email protected]",
  "ip_address": "192.0.2.10",
  "payment_card": "SYNTHETIC-CARD",
  "action": "checkout"
}

Expected output​

The local node prints the following JSON record alongside its service logs. JSON key order can differ.

{ "action": "checkout", "event_id": "demo-001" }

This bounded job was executed with Expanso Edge 2.1.21 on September 10, 2026; the output matched this record. This proof covers local synthetic execution only.

If you do not see the result​

  • Connection refused: start the local node; check both ports match.
  • Port occupied: use another unused port in both commands.
  • No output: inspect terminal one for errors.
  • Rerun: change the completed job’s name before submitting again.

When finished, press Ctrl+C in terminal one to stop the local node. Its state remains in ./expanso-first-result-data; use a fresh directory for an independent later session.

What this proves, and what to check next​

This example allowlists only event_id and action. Other input fields, including newly introduced fields, are omitted. It does not detect personal data inside the retained values; validate those fields against your schema and privacy requirements. Regulatory requirements need a separate assessment. The extended guide below covers separate deletion and hashing patterns.

Common questions​

Does hashing make personal data anonymous?​

Not necessarily. Stable hashes can remain linkable, and guessable values can be recovered through enumeration. The extended guide explores hashing and pseudonymization separately; this first run selects two permitted fields.

How do I move from this result to my own data?​

Keep the transformation small, replace the generated fixture with a representative non-sensitive sample, and compare the output against explicit expectations. Then configure your real source and destination, authentication, error handling, and retention policy. Follow the Local Mode Quick Start for local workflow details and the Expanso Cloud Quickstart to manage a deployed node. Cloud deployment and destination receipt require separate verification.

Try another first result​

Explore the extended example​

The extended example explores deletion, salted hashing, pseudonymization, and location generalization. Its fixture and verification scope are separate from this first result.