Malachi. Broker. Skip
(malachi v0.22.7)
View Source
One stretch of a range's history that a consume read moved a consumer past because it was no longer
stored: expired by retention, or removed by an operator. Malachi.Broker.read_consume/5 returns one
per stretch it stepped over, so the layers above can tell an operator which reader lost data.
The fields:
range_id- the range being consumed.source_range_id- the range whose data was missing: the consumed range itself, or one of its sealed ancestors when the read was still draining the pre-split history.from- the source offset the reader was positioned at when the data was found missing.offsets- how many offsets were stepped over, or:unknownwhen an ancestor has no segment left and its end was not recovered after a restart, so only the fact of the skip is known.origin-:startwhen the reader began without a position (a new group, or a child range after a split, which starts over its ancestors),:cursorwhen it resumed from one it held. Only the second is a reader that fell behind retention.source-:selfor:ancestor.
The count comes from segment boundaries, never from the gaps between record offsets, so a segment
whose offsets are sparse by design is not reported. It counts OFFSETS, not records lost: reading an
ancestor delivers only the child's key slice of it, so an ancestor skip is an upper bound on what this
child missed (span/1 says :upper_bound).
Summary
Functions
:unknown when the size of the skip is not known, :upper_bound for a skip over an ancestor (the
reader would only have received its own key slice of it), :exact otherwise.
Types
@type span() :: :exact | :upper_bound | :unknown
How far offsets can be trusted as a count of what this reader missed.
@type t() :: %Malachi.Broker.Skip{ from: non_neg_integer(), offsets: pos_integer() | :unknown, origin: :start | :cursor, range_id: Malachi.Metadata.range_id(), source: :self | :ancestor, source_range_id: Malachi.Metadata.range_id() }