Troubleshooting

Common issues and solutions for messagekit integration.

Events stuck in outbox (not publishing)

Symptoms:

  • Events persist to outbox_events table

  • published flag remains false

  • No events appearing in Kafka topics

Solutions:

  1. Verify Kafka Connect Debezium connector is running:

    curl http://localhost:8083/connectors/outbox-connector/status

  2. Check CDC connector configuration points to outbox table:

    {
  "table.include.list": "public.outbox_events",
  "transforms": "outbox",
  "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter"
}

  3. Review Kafka Connect logs for CDC errors:

    docker logs kafka-connect

  4. Verify database permissions for Debezium user

Duplicate messages consumed

Symptoms:

  • Same event processed multiple times

  • Idempotent consumer not preventing duplicates

Solutions:

  1. Ensure IProcessedMessageStore is configured in consumer:

    from messagekit.infrastructure import IdempotentConsumerBase

class MyConsumer(IdempotentConsumerBase):
    def __init__(self, store: IProcessedMessageStore):
        super().__init__(store)

  2. Verify processed_messages table exists and is populated

  3. Check consumer is using transactional session for store operations

  4. Review Kafka consumer group configuration (enable.auto.commit=false)

Health check failures

Symptoms:

  • /health endpoint returns unhealthy status

  • Outbox health check failing

Solutions:

  1. Check outbox table has unpublished events older than threshold:

    SELECT COUNT(*) FROM outbox_events
WHERE published = false
AND created_at < NOW() - INTERVAL '5 minutes';

  2. Verify Kafka Connect CDC is processing changes

  3. Review FastStream broker health endpoint configuration

  4. Check database connection pool exhaustion

Performance issues

Symptoms:

  • Slow outbox writes

  • High database load

Solutions:

  1. Add database indexes on outbox table (see Setup section)

  2. Tune Debezium CDC polling interval:

    {
  "poll.interval.ms": "500"
}

  3. Increase Kafka Connect worker threads

  4. Review SQLAlchemy connection pool settings

  5. Consider partitioning outbox table by created_at for high volume

Further assistance