Skip to content

fix(queue-amqp): prevent infinite poison-pill loop on message error by supporting requeue: false and DLQ routing #53

Description

@crapougnax

🎯 Goal

Prevent infinite redelivery loops ("poison pill") in @quatrain/queue-amqp by enabling configurable negative acknowledgment (requeue: false) and proper Dead Letter Queue (DLQ) routing when a message handler fails.

📋 Context & Problem Statement

In packages/queue-amqp/src/AmqpQueueAdapter.ts (lines 129–138):

} catch (err) {
   this._logger.error(
      `messageHandler function failed with message: ${
         (err as Error).message
      }`
   )
   // Negative acknowledge the message so it can be requeued or handled by DLQ
   channel.nack(msg)
   process.exit(1)
}
  1. Default requeue: true in amqplib:
    The method signature is channel.nack(message, allUpTo?, requeue?). Omitting requeue defaults to true.
  2. DLQ Failure:
    The inline comment states "Negative acknowledge the message so it can be requeued or handled by DLQ". However, RabbitMQ never dead-letters a message when requeue: true; it only routes messages to a Dead Letter Exchange (DLX) when negatively acknowledged with requeue: false (or when TTL/length limits expire).
  3. Infinite Crash Loop (Poison Pill):
    Calling channel.nack(msg) with requeue: true followed by process.exit(1) causes RabbitMQ to put the failed message back at the head of the queue immediately. When the container restarts in Kubernetes or another worker replica consumes the queue, the exact same deterministically broken message is picked up again, causing infinite CrashLoopBackOff and cluster resource exhaustion.

🛠️ Proposed Solution & Architecture

  1. Configurable Requeue Parameter:
    Add a requeueOnError parameter in QueueListenOptions / AmqpQueueAdapter:
    export interface AmqpListenOptions {
       concurrency?: number
       gpu?: boolean
       requeueOnError?: boolean // Defaults to false or configurable via options / env
    }
  2. Explicit channel.nack(msg, false, requeue):
    Update AmqpQueueAdapter.ts:
    const shouldRequeue = params?.requeueOnError ?? false
    channel.nack(msg, false, shouldRequeue)
    When requeue is false, RabbitMQ will either drop the poison message or route it to the configured Dead Letter Exchange (DLX/DLQ) if dead-lettering is enabled on the queue (x-dead-letter-exchange).
  3. Configurable Exit on Error:
    Rather than unconditionally executing process.exit(1) for every message failure, allow configuring whether the worker process should terminate on handler error or continue processing subsequent messages once the failed message has been safely dead-lettered / nacked.

🧪 Acceptance Criteria

  • channel.nack in AmqpQueueAdapter respects a configurable requeue flag (requeue: false supported).
  • Messages with deterministic handler failures can be routed to RabbitMQ Dead Letter Queues (DLQ) without requeuing.
  • Unit tests cover listen() behavior on message handler failure with both requeue: false and requeue: true.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions