Malachi.Cluster.Capabilities (malachi v0.22.7)

View Source

What this binary can do, said out loud to the rest of the cluster, and the pure check that asks whether every node can do a given thing.

Why a node has to say what it supports

Malachi.Cluster.MachineVersion already stops a control-plane command from being applied by some members and skipped by others: ra moves a group to a new machine version only once every member supports it. That covers the Raft log and nothing else. A storage frame, a replication message or a compression codec never passes through a Raft log, so nothing there tells a newer node to hold off while its peers are still old. This module is that missing half: a node advertises a named capability set, and a feature asks whether the whole cluster advertises the one it needs before it is switched on (Malachi.Cluster.ClusterFlags).

Named capabilities rather than one rising integer, because a release order is not a dependency order: a backport reorders integers, and one feature cannot be held back on its own under them.

How it travels

Inside the SWIM membership attributes, under the reserved atom key :capabilities, as a sorted list of atoms. That is deliberate:

  • The gossip payload already carries attributes as the fourth element of every update (Malachi.Cluster.Membership), so no message shape changes. A member on the old binary keeps the key it does not understand and gossips it onward untouched. Widening the update tuple instead would raise inside Membership.merge/2 on that member, which the catch-all of Malachi.UnexpectedMessage does not cover: it only guards the outer shape of a message.
  • The key cannot collide with an operator's own attributes: Malachi.Application.parse_attributes/1 produces only string keys.
  • Atoms rather than an integer bitmask, because a bitmask index is a second version contract alongside the machine version, and a backport that adds a capability out of order breaks reading an old advertisement in silence. Erlang distribution caches an atom per connection, so a repeated name costs a byte or two. If this list ever passes about sixteen entries, measure the payload rather than guess.

One name for both halves

A flag's name is the capability it requires. One registry, one name in the logs, one name on the command line. A flag that one day needs two capabilities turns the registry into a map in a change of its own; the flag's own name keeps working as the key.

Where the merge lives

Malachi.Cluster.Membership.set_attributes/2 replaces the whole attribute map, so any path that sets attributes at runtime would erase the capabilities. attributes/2 is the single function that puts them in, and both paths into a node's own attributes reach it:

What that guarantees is one thing and not another. A caller that says nothing about capabilities cannot erase them, which is the whole failure this closes: an operator changing a rack on a running node would otherwise empty the list on every peer, with nothing to correct it. A caller that does state a list keeps it, so this is not a guarantee that a node advertises only what its build implements. Nothing in production states one, because the list is a compile-time property of the build, and the multinode tests do, because one VM cannot run two builds and they have to play a cluster caught mid-upgrade. Before removing a check elsewhere on the strength of this, read ensure/1.

Summary

Types

attributes()

The membership attributes a member gossips (Malachi.Cluster.Membership).

capability()

A capability, which is also the name of the flag that requires it.

reads()

One read of the membership view: what a node's status and attributes are, or {nil, %{}}.

Functions

advertised()

The capabilities this node advertises to its peers, sorted so the advertisement is stable.

attributes(operator_attributes, capabilities \\ advertised())

This node's membership attributes: the operator's own attributes with this build's capability list merged in under key/0.

ensure(attributes)

attributes with this build's capability list added, when the caller did not state one.

key()

The reserved attribute key this node's capability list is gossiped under.

known()

Every capability this build knows by name, sorted.

of(attributes)

The capabilities attributes advertises, or [] when it advertises none.

resolve(input, known \\ known())

Resolves an operator's input to one of known, or {:error, :unknown_flag}.

supported_by_all(nodes, capability, reads)

Whether every node in nodes advertises capability, answering :ok or naming the ones that do not.

Types

attributes()

@type attributes() :: Malachi.Cluster.Membership.attributes()

The membership attributes a member gossips (Malachi.Cluster.Membership).

capability()

@type capability() :: atom()

A capability, which is also the name of the flag that requires it.

reads()

@type reads() :: (node() -> {Malachi.Cluster.Membership.status() | nil, attributes()})

One read of the membership view: what a node's status and attributes are, or {nil, %{}}.

Functions

advertised()

@spec advertised() :: [capability()]

The capabilities this node advertises to its peers, sorted so the advertisement is stable.

attributes(operator_attributes, capabilities \\ advertised())

@spec attributes(attributes(), [capability()]) :: attributes()

This node's membership attributes: the operator's own attributes with this build's capability list merged in under key/0.

The single place the merge happens, and it overwrites: whatever operator_attributes said about capabilities is replaced by capabilities. An operator key can never shadow the list either, because operator keys are strings.

A caller setting attributes on a running node does not have to come through here, but not because this is applied for it: Malachi.Cluster.MembershipServer.set_attributes/2 goes through ensure/1, which fills the list in only when the caller left it out. See that function for what the difference buys.

ensure(attributes)

@spec ensure(attributes()) :: attributes()

attributes with this build's capability list added, when the caller did not state one.

This is what the runtime path goes through: Malachi.Cluster.MembershipServer.set_attributes/2 replaces the whole attribute map, so an operator changing a rack on a running node would otherwise erase what it advertises, and its peers would keep the emptied list. A live node is never suspected, so nothing raises its incarnation to correct that.

A caller that does state a list keeps it. In production nothing does, because the list is a compile-time property of the build; in the tests it is the seam that stands one server up as though it ran another build, which is the only way a single VM can play a cluster mid-upgrade.

key()

@spec key() :: atom()

The reserved attribute key this node's capability list is gossiped under.

known()

@spec known() :: [capability()]

Every capability this build knows by name, sorted.

This is the registry the operator's input is resolved against and the set a flag name must belong to. It is the same list as advertised/0 today; they are separate functions because a future build may know a name it cannot yet honour.

of(attributes)

@spec of(attributes()) :: [capability()]

The capabilities attributes advertises, or [] when it advertises none.

A member on a build that predates this module has no such key, which reads as advertising nothing. That is the safe answer: it is exactly the member a flag must not be switched on over.

resolve(input, known \\ known())

@spec resolve(String.t(), [capability()]) ::
  {:ok, capability()} | {:error, :unknown_flag}

Resolves an operator's input to one of known, or {:error, :unknown_flag}.

Never String.to_atom/1: the input is a command-line string, and turning arbitrary text into atoms grows a table that is never collected. Matching against known/0 also means an unknown name is refused before anything reaches the Raft log.

supported_by_all(nodes, capability, reads)

@spec supported_by_all([node()], capability(), reads()) ::
  :ok | {:error, {:unsupported, [node()]}}

Whether every node in nodes advertises capability, answering :ok or naming the ones that do not.

reads is the membership view, injected as node -> {status, attributes} so this stays pure over its inputs. A node counts as advertising only when it is :alive and its attributes list the capability. Anything else is refused and named: :suspect, :dead, unknown to the view at all (which answers {nil, %{}}), or simply running a build that does not have the capability.

nodes is the statically configured node set, not the membership's alive set and not a Raft group's members. The alive set silently omits a node that has just joined on an old build, and a Raft group omits any node whose build does not have that group's machine module at all, which during a rolling upgrade is every node not yet upgraded. The configured set is the only population that contains a node the cluster has not heard from, and refusing over it is the safe direction: a stale view can only withhold a capability, never invent one, because attributes travel with their own member's incarnation.

Monotone by construction: adding a node can only move the answer from :ok to a refusal, never the other way.