# Overview When a system requires a lot of processing and a delay in processing is acceptable, an async job worker pool approach can be used to store the "to-be-processed" work in a [[Message Queue]] that will be picked up by an [[Asynchronous]] job worker pool of a service. ## Diagram ![[Async Job Worker Pool 2024-09-18 07.07.51.excalidraw.svg]] # Key Considerations of Async Job Worker Pool #flashcard - The [[Message Queue]] will hold messages to be processed by the service workers. This pattern requires an asynchronous process, so the upstream component can put the message on the queue and continue processing. If this was a synchronous process, the upstream system would stop processing and wait for a response that the message was processed. This would effectively render the queue useless. - For the message queue, you must determine the [[Message Queue#Message Ordering]]. This will impact how the service workers pull messages off the queue to be processed. The preferred approach will be based on the particular use case, but [[FIFO - Queues]] is the most popular. - There are also some use cases where it is not acceptable to miss a message or to process a message more than once. This is another property of a message queue referred to as [[Message Queue#Delivery Semantics]]. For example, [[AWS SQS]] offers at least once delivery of messages and the workers will respond back to the queue with [[heartbeat]] messages to indicate that they are still processing the job. If no message is received, the message will re-enter the queue. <!--ID: 1751507776438--> # Pros # Cons # Use Cases # Related Topics