Backpressure
When downstream processing cannot keep up, do not let the queue and memory grow forever. Slow or pause the upstream and resume after recovery.
When downstream processing cannot keep up, do not let the queue and memory grow forever. Slow or pause the upstream and resume after recovery.
Without feedback, pending work fills memory.
Problem and key mechanism: When uploads outpace image processing, queue length, unfinished work, and memory rise; when downstream slows or reaches its limit, the system sends that feedback upstream so the producer slows down, pauses, or temporarily stops accepting work.
Judge improvement by recovery evidence: After a simulated peak, queue and memory should stay bounded and publishing should resume when capacity returns. Backpressure does not add capacity or become a fixed rate rule.
Financial report exports also need backpressure: During a bulk report export, data reading may outpace file writing. As the writer nears its capacity limit, it tells the reader to pause and resume after space is available, preventing the export process from exhausting memory as its buffer grows.
Add backpressure to the image-processing path: when downstream processing slows or unfinished work reaches its limit, pause or slow the upstream; resume after recovery. Simulate uploads faster than processing and verify that queue and memory do not grow without bound, with trigger and recovery feedback recorded.