Malachi. Retention. Expirer
(malachi v0.22.7)
View Source
Retention's only production delete path: drop one expired segment from the control plane, then delete its stored bytes on the replicas, and only then.
Why the order and the condition carry the whole module
The control plane is the authority on which segments exist, so its answer decides whether the bytes
under a segment are still owed to a reader. Malachi.Metadata refuses a delete while the topic is
migrating between vnodes, refuses one for a segment that is still the write head, and can answer
neither way when Raft times out. Deleting the files regardless leaves a segment the control plane
still lists with no copy anywhere: Malachi.Cluster.SelfHealing measures every replica short of the
recorded byte_size, finds no intact copy to repair from, and a consumer positioned inside that
segment reads :eof until some later sweep finally gets :ok. The rule lives in
Malachi.Cluster.Retention.delete_replicas?/1, which is pure and says why for each answer.
What it reports, and what it deliberately does not
Three outcomes, three channels, chosen so a cluster in trouble does not drown its own log:
- A refusal is counted, not logged. It is expected and self-correcting, and a vnode split moving
a busy topic refuses every one of its sealed segments on every sweep, which would be a line per
segment per minute for as long as the migration runs.
malachi_retention_expire_failures_totalalready carries it, labeled by the reply. - A replica that did not answer leaves a directory on its disk that no later sweep can name,
because a segment gone from the control plane never comes back from
Malachi.Cluster.Retention.expired/3. That is counted asmalachi_retention_orphan_directories_left_total, the other half of the sweeper's reclaim count: the two together say whetherMalachi.Retention.OrphanSweeperis keeping up, and the first still moves when the sweeper is turned off. - A control-plane call that did not answer at all is logged, because it means the broker on this node is not serving, which is a fact about the node rather than about the segment.
Seams
:delete_command and :delete_replica exist so the refusal and the unreachable replica can be
exercised without a wedged cluster. Their defaults are the real ones.
Summary
Functions
Expires segment through broker, answering what the control plane answered.
Functions
@spec expire(Malachi.Metadata.segment_meta(), GenServer.server(), keyword()) :: term()
Expires segment through broker, answering what the control plane answered.
The answer is the caller's telemetry (Malachi.Cluster.RetentionCoordinator turns it into a label),
so it travels back unchanged whatever this function did with the replicas.
Options
:delete_command-(segment_id -> reply), how the segment is dropped from the control plane (defaultMalachi.BrokerServer.delete_segment/2againstbroker);:delete_replica-(replica, segment_id -> :ok | {:error, term()}), how one replica's stored bytes are removed (defaultMalachi.Cluster.ReplicationServer.delete/2).