Malachi. Consumer. Assignment
(malachi v0.22.7)
View Source
Assigns a topic's ranges across the live members of a consumer group, so each range is consumed by exactly one member and the work spreads in parallel: the consumer-group counterpart of Kafka's partition assignment, over NorthGuard's dynamic ranges.
Reuses Malachi.Cluster.Placement.place/4 (rendezvous/HRW hashing): each range goes to its
top-ranked member (replication_factor: 1). HRW is min-reshuffle, which is exactly the property a
consumer group wants. strong stickiness: a member leaving moves only its own ranges (every
surviving member keeps all of its ranges), and a member joining moves ranges only to it (existing
members only ever lose ranges, never trade them). That matters because each reassignment costs the new
owner a re-read from the group's committed position. Balance is statistical (HRW spreads evenly in
expectation) rather than exactly even: the right trade for NorthGuard, whose hot topics accumulate many
ranges as they split, so the law of large numbers keeps consumers well balanced while churn stays
minimal. The result is deterministic (the same ranges and members yield the same assignment on every
node), so a replicated or failed-over coordinator computes an identical one.
Summary
Functions
Assigns range_ids across members, returning %{member => [range_id]} with each range under exactly
one member (ranges sorted into a canonical order per member). Every member appears in the map (with []
if it got none), so a caller can tell an idle member from an unknown one. [] members => %{}; []
ranges => every member maps to []. Duplicate members or ranges are ignored.
Functions
Assigns range_ids across members, returning %{member => [range_id]} with each range under exactly
one member (ranges sorted into a canonical order per member). Every member appears in the map (with []
if it got none), so a caller can tell an idle member from an unknown one. [] members => %{}; []
ranges => every member maps to []. Duplicate members or ranges are ignored.