Step 2: Convert Timezones to UTC

Once you have a standard timestamp object, the next step is to convert it to a consistent timezone. The universal standard for data processing is Coordinated Universal Time (UTC).

The Goal

You will modify your pipeline to take the normalized_timestamp from the previous step and convert it to a new timestamp_utc field, ensuring all your events are comparable on a single timeline.

Implementation

1. Start with the Previous Pipeline: Copy the normalize-timestamps.yaml from Step 1 to a new file named convert-timezones.yaml.

   cp normalize-timestamps.yaml convert-timezones.yaml

2. Add the Timezone Conversion Logic: Open convert-timezones.yaml and add the conversion logic to the bottom of the existing mapping processor.

   Add this to the 'mapping' processor in convert-timezones.yaml

   # --- Logic from Step 1 (Parsing) ---
   # (The existing timestamp parsing logic remains here)
   
   # --- START: New additions for Timezone Conversion ---
   
   # Convert the parsed timestamp to a standard UTC string
   root.timestamp_utc = this.normalized_timestamp.ts_format_iso8601("UTC")
   
   # For auditing, it's good practice to also extract the original timezone offset
   root.original_timezone = this.normalized_timestamp.ts_format("Z")
   
   # --- END: New additions ---

3. Deploy and Test:

   # --- Send a timestamp with a non-UTC timezone ---
   curl -X POST http://localhost:8080/ingest \
     -H "Content-Type: application/json" \
     -d '{"event_id": "C", "timestamp": "2025-10-20T14:23:45-04:00"}'

4. Verify: Check your logs. The output will show:

   • original_timestamp: The original string with the -04:00 offset.
   • timestamp_utc: The converted time, which will be 2025-10-20T18:23:45Z.
   • original_timezone: The extracted -04:00 offset.

You have now successfully standardized all incoming timestamps to UTC, which is a critical step for reliable time-based analysis.