The incarnation a node resumes at after a restart, kept on disk so a restarted member outranks everything its peers still remember about the member it was.

The problem this solves

A member owns its incarnation, and SWIM's merge is a join on {incarnation, rank} that ignores an equal or lower pair. A node that restarts from scratch therefore announces itself below whatever its peers retained, and every peer ignores the announcement and keeps the old record: the old status and, which is what makes this more than a curiosity, the old attributes.

So a node that comes back on a build advertising fewer capabilities is still remembered as advertising the old set (Malachi.Cluster.Capabilities), and a cluster flag can be switched on over a node that cannot honour it. The same staleness silently feeds rack-aware placement, which reads the same attributes.

An incarnation rises only on a refutation or a deliberate attribute change, so it cannot be derived from anything the node already keeps. It has to be remembered.

How

A single decimal number in malachi.incarnation, at the root of the log data directory beside malachi.format, written the same durable way (Malachi.Storage.FormatMarker): a temporary file with :sync, a rename over the target, then an fsync of the directory, because a rename is a change to the directory that the file's own fsync does not persist.

The number is a ceiling, not the current value. reserve/3 starts the node one above the recorded ceiling and immediately records a ceiling a whole block higher, so the node can raise its incarnation that many times without touching the disk again. Only crossing the block writes, through extend/3. That keeps the write off the refutation path, which is where an fsync would sit in the way of the failure detector.

The wall clock is a floor under all of it

A reservation starts above the recorded ceiling and above the current second (seed/0). The clock is not a source of incarnations, it is a floor: the recorded ceiling still wins whenever it is higher, so a clock moving backwards can never lower a node.

It is there for the boot that has no ceiling to read. A node upgraded from a build that never kept this file has no record, while its peers may remember it well above zero, because the old build raised its incarnation in memory on every refutation and every attribute change. Starting that node at 1 puts it below what they hold, which is the permanent staleness described below, and it is not something an operator can repair by hand: the number needed is whatever the peers happen to remember, which is not visible from the node being started. An incarnation counts refutations, so it reaches tens or hundreds; seconds since 1970 do not. Seeding from the clock therefore clears every value an older build could have reached, and asks nothing of whoever runs the upgrade.

A missing file and a damaged one are not the same

Only a genuinely absent file reads as ceiling 0, which is a node's first boot. A file that exists but cannot be read or parsed is an error, and the node refuses to start on it.

That distinction is the whole safety of this module, because a node that resumes below what its peers remember is never corrected. Its announcement loses the merge and is ignored; the peers keep the old record, including the old attributes; and since the node is answering their pings they have no reason to suspect it, so no refutation is ever provoked to lift it. The staleness is permanent, not a round. Starting from a guess is therefore not a lesser evil than not starting at all.