Malachi.Storage.FormatMarker (malachi v0.22.7)

View Source

The data-directory format marker: a small plain-text file at the root of the log data directory that records which on-disk format the directory holds, so a binary that cannot read that format refuses to start instead of opening it.

Without it, an older release started on data a newer one wrote does not fail. Recovery reads the unknown frame as damage, preserves it, and the node carries on. The marker turns that into a refusal before any process opens a segment.

The file

format=1
written_by=0.12.0
requires=0.12.0

format is the level, written_by the release that wrote the file, and requires the oldest release that understands format. The last one is recorded by the WRITER because only the writer can know it: a binary refusing a marker from its future has no other way to name the release the operator should go back to. Unknown keys are ignored, so a later release may add some.

The parser is strict about everything else. All three keys must be present exactly once and the file must end with a newline, so a marker cut short anywhere reads as invalid rather than as a lower level: format=12 truncated to format=1 must never let a release that only understands 1 start. An invalid marker refuses, which is the conservative answer.

The rule

A binary starts on a directory only when the marker is at or below the highest format it understands (supported_format/0). No marker means either a fresh directory or one written before the marker existed; both get a marker at baseline_format/0, whatever this release reads. The marker only rises, and it rises when a format-changing feature is switched on through the cluster flag, never because a newer binary merely started. raise_to/3 is that mechanism.

Durability

Both writes, creating the marker and raising it, go the same way: the content into a temporary file written with :sync, a rename over the marker, then an fsync of the directory (Malachi.Storage.Directory.sync/1), because the rename is a change to the directory that the file's own fsync does not persist. A crash anywhere along that path leaves either the old marker or the new one, never a mix: the rename is atomic, and a torn temporary file is not the marker. Only after the directory fsync does the call return :ok, which is what lets raise_to/3 promise the new level is durable before the first byte of a new format is written.

A path that TRUSTS an existing marker fsyncs the directory too, rather than assuming an earlier write did: a write whose rename landed and whose fsync failed leaves the marker visible but not durable, and both the startup gate and a repeated raise_to/3 would otherwise take it at face value. The fsync is cheap and idempotent; skipping it is what is expensive.

Summary

Types

marker()

A parsed marker.

observation()

What was found where the marker should be.

parse_error()

Why a marker could not be parsed.

refusal()

Why a directory is refused, with the marker path for the message.

Functions

baseline_format()

The format level a directory without a marker starts at.

decide(arg, supported)

What to do about observation for a binary that reads up to supported. Pure.

enforce(dir, opts \\ [])

The startup gate: reads the marker in dir, writes one when there is none, and answers whether the node may start. Options (for tests): :supported (default supported_format/0) and :version (default this release's version).

exit_status()

The exit status of a refused start (78, EX_CONFIG).

parse(content)

Parses the file's content. Never raises: anything that is not a complete marker is an error.

path(dir)

The marker's path under dir.

raise_to(dir, format, opts \\ [])

Raises the marker in dir to format, durably (see the moduledoc). The marker only rises: the same level is a no-op, a lower one is refused, and so is a level above what this binary reads, since the node would then refuse its own next start. There must already be a marker: enforce/2 writes one on every start, so a directory without one is a directory the startup gate never ran on.

read(dir)

Reads the marker in dir. A missing marker is reported with whether the directory already holds anything else (segments, shard subdirectories), which only changes the log line.

refuse!(reason, halt_fun \\ &System.halt/1)

Refuses the start through Malachi.StartupRefusal, which logs the reason, repeats it on stderr and halts with exit_status/0 through halt_fun. Only the detail is this module's to say.

render(map)

Renders marker as the file's content.

supported_format()

The highest format level this release can read.

write(dir, format, version \\ release_version())

Creates the marker in dir at format, written by release version, durably (see the moduledoc). A leftover temporary file from an earlier crash is overwritten.

Types

marker()

@type marker() :: %{
  format: pos_integer(),
  written_by: String.t(),
  requires: String.t()
}

A parsed marker.

observation()

@type observation() ::
  {:absent, :fresh | :existing} | {:ok, marker()} | {:error, parse_error()}

What was found where the marker should be.

parse_error()

@type parse_error() ::
  :missing_newline
  | :malformed_line
  | {:duplicate_key, String.t()}
  | {:missing_key, String.t()}
  | {:bad_format, String.t()}

Why a marker could not be parsed.

refusal()

@type refusal() ::
  {:too_new, marker(), pos_integer(), Path.t()}
  | {:invalid, parse_error(), Path.t()}
  | {:io, atom(), Path.t()}

Why a directory is refused, with the marker path for the message.

Functions

baseline_format()

@spec baseline_format() :: pos_integer()

The format level a directory without a marker starts at.

decide(arg, supported)

@spec decide(observation(), pos_integer()) ::
  :ok
  | {:write, pos_integer(), :fresh | :existing}
  | {:refuse, {:too_new, marker(), pos_integer()} | {:invalid, parse_error()}}

What to do about observation for a binary that reads up to supported. Pure.

  • {:write, level, kind} - no marker: write one at baseline_format/0 (kind says whether the directory held data already, for the log line, and never changes the level).
  • :ok - the marker is readable by this binary.
  • {:refuse, reason} - the marker is from a newer format, or is not a marker.

enforce(dir, opts \\ [])

@spec enforce(
  Path.t(),
  keyword()
) :: :ok | {:refuse, refusal()}

The startup gate: reads the marker in dir, writes one when there is none, and answers whether the node may start. Options (for tests): :supported (default supported_format/0) and :version (default this release's version).

exit_status()

@spec exit_status() :: non_neg_integer()

The exit status of a refused start (78, EX_CONFIG).

parse(content)

@spec parse(binary()) :: {:ok, marker()} | {:error, parse_error()}

Parses the file's content. Never raises: anything that is not a complete marker is an error.

Examples

iex> Malachi.Storage.FormatMarker.parse("format=2\nwritten_by=0.13.0\nrequires=0.13.0\n")
{:ok, %{format: 2, written_by: "0.13.0", requires: "0.13.0"}}

iex> Malachi.Storage.FormatMarker.parse("format=2\nwritten_by=0.13.0\nrequires=0.1")
{:error, :missing_newline}

path(dir)

@spec path(Path.t()) :: Path.t()

The marker's path under dir.

raise_to(dir, format, opts \\ [])

@spec raise_to(Path.t(), pos_integer(), keyword()) ::
  :ok
  | {:error,
     :no_marker
     | {:lower, pos_integer(), pos_integer()}
     | {:unsupported, pos_integer()}
     | {:no_first_release, pos_integer()}
     | {:invalid, parse_error()}
     | {:io, atom()}}

Raises the marker in dir to format, durably (see the moduledoc). The marker only rises: the same level is a no-op, a lower one is refused, and so is a level above what this binary reads, since the node would then refuse its own next start. There must already be a marker: enforce/2 writes one on every start, so a directory without one is a directory the startup gate never ran on.

Must return :ok BEFORE the first byte of the new format is written.

Options (for tests, since this release reads only format 1 and so has nothing to raise to): :supported (default supported_format/0), :requires (default the recorded first release of format) and :version (default this release's version).

read(dir)

@spec read(Path.t()) :: {:ok, observation()} | {:error, {:io, atom()}}

Reads the marker in dir. A missing marker is reported with whether the directory already holds anything else (segments, shard subdirectories), which only changes the log line.

refuse!(reason, halt_fun \\ &System.halt/1)

@spec refuse!(refusal(), (non_neg_integer() -> any())) :: any()

Refuses the start through Malachi.StartupRefusal, which logs the reason, repeats it on stderr and halts with exit_status/0 through halt_fun. Only the detail is this module's to say.

render(map)

@spec render(marker()) :: String.t()

Renders marker as the file's content.

Examples

iex> Malachi.Storage.FormatMarker.render(%{format: 1, written_by: "0.12.0", requires: "0.12.0"})
"format=1\nwritten_by=0.12.0\nrequires=0.12.0\n"

supported_format()

@spec supported_format() :: pos_integer()

The highest format level this release can read.

write(dir, format, version \\ release_version())

@spec write(Path.t(), pos_integer(), String.t()) :: :ok | {:error, {:io, atom()}}

Creates the marker in dir at format, written by release version, durably (see the moduledoc). A leftover temporary file from an earlier crash is overwritten.