Setup Environment for Priority Queues

Before building the priority queue system, you'll set up your Kafka environment and configure routing credentials.

Prerequisites

This example requires the following services to be running:

• Kafka

Before you begin, please ensure these services are set up and running according to their respective guides. Additionally, ensure you have completed the Local Development Setup guide for general environment configuration.

Step 1: Configure Example-Specific Variables

After setting up the core services, configure priority queue-specific variables:

# Set edge node identifier for tracking
export NODE_ID="edge-node-01"

# Verify environment variables
echo "Kafka Brokers: $KAFKA_BROKERS"
echo "Node ID: $NODE_ID"

Step 2: Verify Priority Topics

Before proceeding, verify all required priority topics exist and are accessible:

# Run this verification script
for topic in critical high normal low; do
  echo "Testing topic: logs-$topic"
  
  # Test producer
  echo '{"test": "message", "topic": "logs-'$topic'"}' | \
    $KAFKA_HOME/bin/kafka-console-producer.sh \
      --bootstrap-server $KAFKA_BROKERS \
      --topic logs-$topic
  
  # Test consumer (read the test message back)
  timeout 5s $KAFKA_HOME/bin/kafka-console-consumer.sh \
    --bootstrap-server $KAFKA_BROKERS \
    --topic logs-$topic \
    --from-beginning \
    --max-messages 1
  
  echo "✅ Topic logs-$topic is accessible"
  echo
done

Next Steps

With your environment configured, you're ready to implement priority-based routing:

Step 1: Severity Routing

What's Next

• Step 1: Replace shell pipeline with severity-based routing (CRITICAL → immediate, INFO → batched)
• Step 2: Add customer tier prioritization (enterprise → dedicated queue, free → bulk processing)
• Step 3: Implement multi-criteria scoring combining severity, tier, and event type
• Step 4: Add age-based escalation to prevent message starvation