Why a Task Queue
A web request should return fast. When it triggers slow work such as sending email, resizing images, or generating reports, doing that work inline blocks the user and ties up the server. A task queue moves that work onto a background path.
The Three Roles
- Producer enqueues a job describing work to do, then returns immediately.
- Broker durably stores pending jobs and hands them to workers.
- Worker pulls a job, runs the handler, and acknowledges completion.
The broker is the buffer that absorbs bursts. If a spike of requests arrives, jobs pile up safely instead of overloading the workers.
What Travels in a Job
A job is a small message: a type name plus arguments such as an order id. Keep payloads small and pass references, not large blobs. The worker looks up heavy data itself.
Benefits
- Responsiveness because the request returns before work finishes.
- Elasticity because you scale workers independently of web servers.
- Resilience because jobs survive a worker crash and get retried.
Key idea
A task queue trades immediate execution for a fast response and durable, scalable background processing.