Skip to main content

Step 2: Database Circuit Breakers

Database connections are a common point of failure. If a database becomes unavailable, pipelines can crash or hang indefinitely, exhausting connection pools. In this step, you will protect a pipeline against database failures using a try/catch block.

1. The Fragile Pipeline​

The database-circuit-breaker-foundation.yaml file contains a simple pipeline. It accepts a user_id, queries a PostgreSQL database to get the user's profile, and adds it to the event.

This pipeline is fragile. If the database is down, the sql_select processor will fail, and the entire pipeline will stop.

2. See It Fail​

  1. Start the Mock Database: Make sure the PostgreSQL service is running.

    # (From the services/postgres.yml guide)
    docker compose -f services/postgres.yml up -d

  2. Stop the Database: Now, simulate an outage.

    docker compose -f services/postgres.yml stop

  3. Send a Request:

    curl -X POST http://localhost:8084/user-events \
      -H "Content-Type: application/json" \
      -d '{"user_id": "user_001"}'

    The request will hang and eventually fail. The pipeline is now stuck trying to connect to a dead database.

3. Add the Circuit Breaker​

You will now add a try/catch block to handle the database failure gracefully.

  1. Copy the Foundation:

    cp examples/data-routing/database-circuit-breaker-foundation.yaml database-circuit-breaker.yaml

  2. Modify the Pipeline: Open database-circuit-breaker.yaml and replace the pipeline section with the one below. This wraps the database call in a try block and adds a catch block for error handling.

    Partial snippet
    Insert this into database-circuit-breaker.yaml
    pipeline:
      processors:
        - try:
            - branch:
                processors:
                  - sql_select:
                      driver: postgres
                      dsn: ${DB_CONNECTION_STRING}
                      table: users
                      columns: [user_name, user_tier]
                      where: user_id = $1
                      args_mapping: root = [this.user_id]
                result_map: |
                  root.user_name = this.index(0).user_name
                  root.user_tier = this.index(0).user_tier
            - mapping: |
                root = this
                root.user_profile.name = this.user_name
                root.user_profile.tier = this.user_tier
                root.db_enriched = true
                root.db_status = "success"
        - catch:
            - mapping: |
                root = this
                root.db_enriched = false
                root.db_status = "failed"
                root.fallback_reason = "database_circuit_breaker_open"

4. Deploy and Test Again​

  1. Restart the Database (optional): You can test with the database running first to see the success path.

    docker compose -f services/postgres.yml start

  2. Stop the Database Again:

    docker compose -f services/postgres.yml stop

  3. Send the Same Request:

    curl -X POST http://localhost:8084/user-events \
      -H "Content-Type: application/json" \
      -d '{"user_id": "user_001"}'

    This time, you will get an immediate response. The output will be a JSON object with db_enriched: false and db_status: "failed". The pipeline handled the error gracefully instead of crashing.

You have now protected your pipeline from database failures. Don't forget to restart your postgres container for the next steps.