Step 3: Enrich with Time Metadata

The final step in timestamp normalization pre-calculates common time components that an analytics team might otherwise derive in each query. Whether to materialize these fields depends on the downstream query plan and storage model.

The Goal

You will add a time_metadata object to your events that contains useful, pre-calculated fields like the year, month, hour, and day of the week, all extracted from the standardized timestamp_utc field.

Implementation

1. Start with the Previous Pipeline: Copy the convert-timezones.yaml from Step 2 to a new file named enrich-timestamps.yaml.

   cp convert-timezones.yaml enrich-timestamps.yaml

2. Add the Enrichment Logic: Open enrich-timestamps.yaml and add the enrichment logic to the bottom of the existing mapping processor.

   Add this to the 'mapping' processor in enrich-timestamps.yaml

   # --- Logic from previous steps ---
   # (The existing parsing and timezone conversion logic remains here)
   
   # --- START: New additions for Enrichment ---
   
   root.time_metadata = {
     "year": this.timestamp_utc.ts_year(),
     "month": this.timestamp_utc.ts_month(),
     "day": this.timestamp_utc.ts_day(),
     "hour": this.timestamp_utc.ts_hour(),
     "day_of_week": this.timestamp_utc.ts_weekday()
   }
   
   # --- END: New additions ---

3. Deploy and Test:

   # --- Send a test event ---
   curl -X POST http://localhost:8080/ingest \
     -H "Content-Type: application/json" \
     -d '{"event_id": "D", "timestamp": "2025-10-20T18:23:45Z"}'

4. Verify: Check your logs. The output will now contain the new time_metadata object with all the useful time components pre-calculated. Your data is now perfectly formatted for efficient analytics.

   Example Output Snippet:

   {
     "timestamp_utc": "2025-10-20T18:23:45Z",
     "time_metadata": {
       "year": 2025,
       "month": 10,
       "day": 20,
       "hour": 18,
       "day_of_week": 1
     }
   }

You have now built a complete timestamp normalization pipeline that can handle multiple formats, standardize timezones, and enrich the data for analytics.