问题
From the doc: https://pulsar.apache.org/docs/en/cookbooks-retention-expiry/#get-the-ttl-configuration-for-a-namespace, it is a little bit confusing about the difference between backlog quotas and TTL.
As I understand so far, a message arrives broker, and broker will find out all subscriptions on that topic, and retrieve their backlog, and put the message to those backlog. If this message is acknowledged by one subscription, it will be removed from its backlog (backlog is per subscription). If the message is not in any backlog (means all subscription acknowledged it), then this message is considered as acknowledged, and then retention policy kicks in, to decide if it needs to be deleted or keep for some time.
If a message is not acknowledged in one backlog for some time, and the backlog quota reaches a size limitation, then backlog retention policy kicks in. So this is more about size than time. And if we use consumer_backlog_eviction, this message will be discarded from the backlog, but question, is that considered acknowledged or not? so the first retention policy kicks in?
And the TTL, if a message is not acknowledged for some time, will it be removed from all backlogs? and then considered as acknowledged and then let the first retention policy handle it?
UPDATE:
to be more precise of this question:
In backlog quotas document, it says:
consumer_backlog_eviction: The broker will begin discarding backlog messages
Discarding means, making it acknowledged? So that the global retention policy can kick in?
producer_request_hold: The broker will hold and not persist produce request payload
Is it saying, that, it will not put new messages into the backlog, but for those new coming messages, are they automatically acknowledged or not (say there is just one subscription at that moment)? And does this block the real producer (I guess not, it is just that the broker won't put new messages into the backlog anymore)
(for TTL) If disk space is a concern, you can set a time to live (TTL) that determines how long unacknowledged messages will be retained.
Again, if TTL is exceeded, it will not "retain" it, means, make it acknowledged? or just throw it away?
回答1:
And if we use consumer_backlog_eviction, this message will be discarded from the backlog, but question, is that considered acknowledged or not? so the first retention policy kicks in?
The message will be acknowledged and marked for deletion. Then the retention policy for acknowledged messages will kick in at some point depending on the configuration.
And the TTL, if a message is not acknowledged for some time, will it be removed from all backlogs? and then considered as acknowledged and then let the first retention policy handle it?
The TTL should be applied to all backlogs and outdated unconsumed messages will be automatically acknowledged. And again the retention policy for acknowledged messages will kick in.
来源：https://stackoverflow.com/questions/60711436/whats-the-difference-between-backlog-quotas-and-ttl