Enforcing strict back pressure #3026
Replies: 1 comment
-
RabbitMQ will stop reading from the socket of connections that send at least one message (one of the Watermark paging ratio controls to what extent classic queues will delay paging in memory messages. Setting it to 1 or higher is dangerous. While setting validation can be tweaked, we usually see people who want to do the opposite: make memory usage more predictable, and a higher watermark for CQs will do the opposite of that. The back pressure mechanism is "strict" by your definition. I'm not sure what you are trying to achieve here. Quorum queues have More or less precise resource caps in highly concurrent distributed system are only possible if you allocate all memory directly and RabbitMQ by definition does not: the Erlang runtime does. Or the kernel cap do it (cgroups) but its reaction is not something you typically want in a stateful data service. |
Beta Was this translation helpful? Give feedback.
-
I'm looking for a way to force strong back pressure to producers.
I'm trying to use flow throttling and avoid paging out. Following https://www.rabbitmq.com/memory.html, it seems that setting a
vm_memory_high_watermark_paging_ratio
to >1
should be enough to avoid paging and usingvm_memory_high_watermark.absolute
should allow to force stopping producer connection when the limit is reached.Unfortunately, trying to set
vm_memory_high_watermark_paging_ratio
to1
fails withvm_memory_high_watermark_paging_ratio invalid, Float is not between 0 and 1
.It seems that this check is here since a long time.
rabbitmq-server/deps/rabbit/priv/schema/rabbit.schema
Line 774 in f49bb00
Is there another way to enforce a strict back pressure. Meaning, if memory threshold is reached, producer connection is totally frozen?
Btw, congrats for the work done on RabbitMQ, it's a great piece of software.
Beta Was this translation helpful? Give feedback.
All reactions