The one versioning rule shared by every control-plane ra state machine: metadata, lease, ring, users, lockouts, ACLs, cluster flags and storage policies.

Why a command needs a version

A Raft group whose members run different code applies the same log with different apply/3 functions. A command that a newer member understands and an older one skips leaves the replicas holding different states with no error anywhere. ra solves the transport half of this: every machine declares version/0, and under the all upgrade strategy (set explicitly in config/config.exs) a group only switches to version N once every member supports N. The switch is a noop entry in the log, applied as {:machine_version, from, to}, and from then on meta.machine_version in apply/3 is N on every replica, including on a replay after a restart (the effective version rises at the same log index each time). A member whose code is below the effective version stops applying entries until it restarts on newer code: fail-stop, never divergence.

This module is the other half: apply/5 refuses a command whose tag was introduced above the effective version, and a tag it does not know at all, with the state unchanged and no effects. Because the decision reads only meta.machine_version, which is replicated, every member reaches the same refusal. Nothing node-local takes part: not version/0, not the pin, not membership.

The rule for a new command

A command introduced by a release goes into its module's command_versions/0 table at code_version() + 1, and @code_version below rises in the same change. Giving an existing command another field is a new command by this rule: the table is keyed by shape, so the wider tuple is its own entry at the new version, and older members refuse it instead of raising on it or skipping it. With all, the command is then refused identically on every member until the last one runs and advertises the new version. Existing commands stay at the version they were introduced at forever: nothing takes a release cursor, so every command ever written is replayed on restart and must stay appliable. The one exit is a shape that no release has ever emitted into any log: no replay can reach it, so the command leaves the table and a member refuses its tag as unknown, the same refusal on every replica.

The eight machines share one version number. That keeps one pin meaningful for all of them, and bumping it for one machine only costs the others a no-op {:machine_version, n - 1, n}. Version 2 is where Malachi.Cluster.ClusterFlags introduced {:enable_flag, flag} and version 3 is where Malachi.Cluster.PolicyRegistry introduced {:define_policy, name, policy}, and version 4 is where Malachi.Metadata introduced {:bind_topic_policy, topic, name}, and version 5 is where Malachi.Auth.UserRegistry introduced the console roles ({:set_role, username, role}, a five element put_user and {:import_users_with_roles, entries}), and version 6 is where Malachi.Cluster.Policy introduced the retention.segment_max_age_ms field (#197), so each time the others moved with nothing but that no-op.

State that a command fills from the log entry

A field added to the STATE a command produces, with the command's shape unchanged, is not a new command by the rule above, as long as nothing an older member decides reads it. register_segment is the case in point: since #197 a registered segment carries opened_at, filled from meta.system_time, the timestamp the leader wrote into the log entry (Malachi.Cluster.MetadataMachine). Every replica on this code stamps the same value, a replay stamps it again, and a member on older code stores the segment without the field and never reads it, so no member applies the command differently from another in anything either one acts on. The reverse direction is the reader's to handle: a member on this code can read an older leader's state (a consistent query runs at the leader), whose segments lack the key, so Malachi.Cluster.SegmentRoll treats a missing key as unknown and leaves the segment alone rather than raising or rolling it. The same holds for a topic export that carries the field (:insert_topic): a destination on older code stores a key it never reads, so the export format does not move for it. A field an older member would act on, or one carried in the command, still takes a new shape at a new version.

Holding the version during an upgrade

version/0 is min(code_version(), pin), where the pin is MALACHI_RA_MACHINE_VERSION (:ra_machine_version_pin). An operator rolling out a release pins the version in production, so the effective version stays put and rolling the build back stays possible at any moment. Removing the pin with a rolling restart finalizes the upgrade; from then on a binary below the new version stops applying entries, which is the rollback floor. A pin above the code version has no effect.

Moving a topic between vnodes

A vnode split copies a topic's whole state from one Raft group into another with :insert_topic, and the two groups may have different members and different effective versions. The command tag alone cannot say whether the destination understands that state, so the export carries an export_format, and an export above the destination's effective version is refused. Whoever changes the shape of a topic's exported state raises Malachi.Metadata's export format to the version that change ships in.

Seeing a member that stopped

check/3 compares the effective version ra recorded for a local member with the version this node supports. It is pure over its inputs apart from reading ra's counters: the caller keeps the last status, so a member that stays stuck is logged once and reported on every call through the [:malachi, :ra, :machine_version] telemetry event.