Step 3: Enrich with Time Metadata

The final step in timestamp normalization is to pre-calculate common time components that your data analytics team would otherwise have to calculate in every single query. This saves them time and saves you expensive query costs.

The Goal

You will add a time_metadata object to your events that contains useful, pre-calculated fields like the year, month, hour, and day of the week, all extracted from the standardized timestamp_utc field.

Implementation

1. Start with the Previous Pipeline: Copy the convert-timezones.yaml from Step 2 to a new file named enrich-timestamps.yaml.

   cp convert-timezones.yaml enrich-timestamps.yaml

2. Add the Enrichment Logic: Open enrich-timestamps.yaml and add the enrichment logic to the bottom of the existing mapping processor.

   Add this to the 'mapping' processor in enrich-timestamps.yaml

   # --- Logic from previous steps ---
   # (The existing parsing and timezone conversion logic remains here)
   
   # --- START: New additions for Enrichment ---
   
   root.time_metadata = {
     "year": this.timestamp_utc.ts_year(),
     "month": this.timestamp_utc.ts_month(),
     "day": this.timestamp_utc.ts_day(),
     "hour": this.timestamp_utc.ts_hour(),
     "day_of_week": this.timestamp_utc.ts_weekday()
   }
   
   # --- END: New additions ---

3. Deploy and Test:

   # --- Send a test event ---
   curl -X POST http://localhost:8080/ingest \
     -H "Content-Type: application/json" \
     -d '{"event_id": "D", "timestamp": "2025-10-20T18:23:45Z"}'

4. Verify: Check your logs. The output will now contain the new time_metadata object with all the useful time components pre-calculated. Your data is now perfectly formatted for efficient analytics.

   Example Output Snippet:

   {
     "timestamp_utc": "2025-10-20T18:23:45Z",
     "time_metadata": {
       "year": 2025,
       "month": 10,
       "day": 20,
       "hour": 18,
       "day_of_week": 1
     }
   }

You have now built a complete timestamp normalization pipeline that can handle multiple formats, standardize timezones, and enrich the data for analytics.