Conventions

The config/conventions package (@conventions) finds an application's configuration files, so the application does not list paths itself. It plugs into the core builder as a source provider. Discovery is pure: it reads the file tree and the working directory that the run gives, never the disk or the process.

Import moonrockz/moon-lander/config/conventions as @conventions. Each mbt check block on this page runs as a test with mise run test.

app_dirs("acme") gives these sources, in merge order:

The environment-variable source (600) of the core comes after them, so a variable wins over every file.

Walk up to the project root

Discovery starts at the working directory of the run and moves to each parent. The first directory that holds a primary file is the project root. Its file becomes the project source, with the role Workspace and the priority 300.

/w/acme.toml          <- found: the project primary
/w/src/
/w/src/nested/        <- the working directory

A primary file has one of three layouts in a directory:

Two primary files in one directory, in one layout or in two, are an error, Ambiguous. Discovery never merges them or picks one, and it does not look further up.

The file names come from the formats that the builder registers. Register only TOML, and discovery never finds acme.yaml, a file that the run could not parse.

///|
test "find the project file from a nested directory" {
  let builder = @config.builder()
    |> @config.working_dir("/w/src/nested")
    |> @config.format(@toml.format())
    |> @conventions.add(@conventions.app_dirs("acme"))
  let tree = @config.FileTree::new(complete=true)
    .with_listing("/w", [
      { name: "acme.toml", kind: File },
      { name: "src", kind: Directory },
    ])
    .with_listing("/w/src", [{ name: "nested", kind: Directory }])
    .with_listing("/w/src/nested", [])
    .with_file("/w/acme.toml", "project.name = \"acme\"")
  guard @config.load(builder, tree) is Done(config) else {
    fail("a complete tree never asks for files")
  }
  @expect.expect(@config.get_text(config, "project.name")).to_equal("acme")
  let project = config
    .reports()
    .iter()
    .find_first(report => report.kind == File(Workspace))
  @expect.expect(project.map(report => report.path)).to_equal(
    Some(Some("/w/acme.toml")),
  )
}

Stop boundary and isolated mode

boundary(dir) stops the walk-up at dir: discovery looks in dir and never above it. A test sets the boundary to its temporary root, so a file above the root never leaks into the test. A working directory outside the boundary is an error, OutsideBoundary.

///|
test "a test root keeps files above it out" {
  let builder = @config.builder()
    |> @config.working_dir("/tmp/case/src")
    |> @config.format(@toml.format())
    |> @conventions.add(
      @conventions.app_dirs("acme") |> @conventions.boundary("/tmp/case"),
    )
  let tree = @config.FileTree::new(complete=true)
    .with_listing("/tmp", [
      { name: "acme.toml", kind: File },
      { name: "case", kind: Directory },
    ])
    .with_listing("/tmp/case", [{ name: "src", kind: Directory }])
    .with_listing("/tmp/case/src", [])
  guard @config.load(builder, tree) is Done(config) else {
    fail("a complete tree never asks for files")
  }
  let project = config
    .reports()
    .iter()
    .find_first(report => report.kind == File(Workspace))
  @expect.expect(project.map(report => report.status)).to_equal(Some(NotFound))
}

Isolated mode comes from DiscoveryOptions(isolated=true), for example from an --isolated flag that a host parses. It skips the files of the user and the machine: the system directory, the global user directory, and the local and user overrides. Each of them reports Skipped. The project file, its environment file and the environment variables stay. The project option selects a workspace member, as Workspaces and members shows.

System and global user directories

The system source and the global user source come from the platform and the variable set of the run, which @config.platform and @config.env_vars give. Discovery never reads the process environment. The host adapter @config/host.load fills both from the running system when the builder holds none. With no platform, the two sources report Skipped and discovery adds a Note warning.

• The file name in each directory is <app>.<ext>.
• An empty variable counts as unset. XDG_CONFIG_HOME counts only when it is absolute, and it has no effect on Windows.
• The user configuration directory and the application home are one global source. Two global files are Ambiguous.
• On Windows, \ becomes / in each variable. An absolute Windows path starts with a drive and a separator (C:/) or with //server/share; C:foo is not absolute. A relative value joins the working directory.

///|
test "the global candidates come from the variable set of the run" {
  let builder = @config.builder()
    |> @config.working_dir("/w")
    |> @config.platform(Linux)
    |> @config.env_vars({ "HOME": "/home/alice", "ACME_HOME": "/opt/acme" })
    |> @config.format(@toml.format())
    |> @conventions.add(@conventions.app_dirs("acme"))
  guard @config.load(builder, @config.FileTree::new(complete=true))
    is Done(config) else {
    fail("a complete tree never asks for files")
  }
  let global = config
    .reports()
    .iter()
    .find_first(report => report.kind == File(Global))
  @expect.expect(global.map(report => report.candidates)).to_equal(
    Some(["/home/alice/.config/acme/acme.toml", "/opt/acme/acme.toml"]),
  )
}

Overrides next to the project file

A local override is for one checkout, and git ignores it. A user override is the preference of one person, so it wins over the local override. Both sit in the directory of the project file:

The extensions come from the registered formats, so acme.user.yaml can override acme.toml. Two overrides of one kind, such as acme.user.toml and acme.user.yaml, are Ambiguous. A legacy file has no overrides, and with no project file there is no override source.

///|
test "the user override wins over the project file" {
  let builder = @config.builder()
    |> @config.working_dir("/w")
    |> @config.format(@toml.format())
    |> @conventions.add(@conventions.app_dirs("acme"))
  let tree = @config.FileTree::new(complete=true)
    .with_listing("/w", [
      { name: "acme.toml", kind: File },
      { name: "acme.user.toml", kind: File },
    ])
    .with_file("/w/acme.toml", "ui.theme = \"shared\"")
    .with_file("/w/acme.user.toml", "ui.theme = \"personal\"")
  guard @config.load(builder, tree) is Done(config) else {
    fail("a complete tree never asks for files")
  }
  @expect.expect(@config.get_text(config, "ui.theme")).to_equal("personal")
}

An explicit project file

DiscoveryOptions(config=Explicit(path)), for example from a --config flag, names the project file. Discovery does not walk up. A relative path joins the working directory, and on Windows \ becomes /. The source has the role Explicit, and a missing file is an error, FileNotFound. When the path has a standard layout, such as configs/acme.toml, its local and user overrides apply. Other files, such as configs/project.yaml, have no overrides. Skip skips the project file.

Environments

An environment file holds the values of one environment, such as Production, next to the project file. Environments are off until the application turns them on with environments(default?). The name of the run comes from the first of these sources that gives one:

With no name, no environment file loads. With environments off, a given environment option adds a Note warning. Discovery never reads the process arguments, so a host parses --environment and gives the value as the option.

The environment file has the priority 375: it wins over the project file, and the local and user overrides win over it. It follows the layout of the project file:

• The name matches exactly, so acme.production.toml is not the file for Production.
• An environment file alone does not mark a project root.
• Two environment files for one project file, such as acme.Production.toml and acme.Production.yaml, are Ambiguous.
• The system file, the global user file, a legacy file and an explicit file outside the standard layouts have no environment file.

A name must not be empty, must not contain /, \, . or a control character, and must not be user or local in any letter case, so an override is never an environment file. A bad name raises InvalidEnvironmentName, which names the source of the name. environment_name gives the name of the run with the same rules, and check_environment_name checks a name from another source.

An application without a host gives the environment-variable source the reserved names <PREFIX>_ENVIRONMENT and <PREFIX>_HOME itself. These variables select files, so they must not become configuration keys:

///|
test "the environment file of the run wins over the project file" {
  let conventions = @conventions.app_dirs("acme")
    |> @conventions.environments(default="Development")
  let vars = { "ACME_ENVIRONMENT": "Production", "ACME_UI__THEME": "dark" }
  let builder = @config.builder()
    |> @config.working_dir("/w")
    |> @config.env_vars(vars)
    |> @config.format(@toml.format())
    |> @conventions.add(conventions)
    |> @config.env(prefix="ACME", reserved=["ACME_ENVIRONMENT", "ACME_HOME"])
  let tree = @config.FileTree::new(complete=true)
    .with_listing("/w", [
      { name: "acme.toml", kind: File },
      { name: "acme.Production.toml", kind: File },
    ])
    .with_file("/w/acme.toml", "log.level = \"info\"")
    .with_file("/w/acme.Production.toml", "log.level = \"warn\"")
  guard @config.load(builder, tree) is Done(config) else {
    fail("a complete tree never asks for files")
  }
  @expect.expect(@conventions.environment_name(conventions, vars)).to_equal(
    Some("Production"),
  )
  @expect.expect(@config.get_text(config, "log.level")).to_equal("warn")
  @expect.expect(@config.get_text(config, "ui.theme")).to_equal("dark")
  // The reserved names give no configuration key.
  @expect.expect(config.get(@config.KeyPath::parse("environment"))).to_equal(
    None,
  )
}

Choose the parts

app_dirs uses every standard part. without(part) removes one, and base with single parts gives full control:

///|
test "remove the system part, or build from single parts" {
  // No system file, for a tool that never installs machine-wide.
  let tool = @conventions.app_dirs("acme") |> @conventions.without(SystemDir)
  // Only the project file in three layouts and a user override.
  let minimal = @conventions.base("acme")
    |> @conventions.layouts
    |> @conventions.user_overrides
  let kinds = (conventions : @conventions.Conventions) => {
    let builder = @config.builder()
      |> @config.working_dir("/w")
      |> @config.platform(Linux)
      |> @config.env_vars({ "HOME": "/home/alice" })
      |> @config.format(@toml.format())
      |> @conventions.add(conventions)
    let tree = @config.FileTree::new(complete=true)
      .with_listing("/w", [{ name: "acme.toml", kind: File }])
      .with_file("/w/acme.toml", "a = 1")
    guard @config.load(builder, tree) is Done(config) else {
      fail("a complete tree never asks for files")
    }
    config.reports().map(report => report.kind.report_name())
  }
  @expect.expect(kinds(tool)).to_equal([
    "global", "project", "local-override", "user-override",
  ])
  @expect.expect(kinds(minimal)).to_equal(["project", "user-override"])
}

Workspaces and members

workspace() turns on workspaces. A project file that sets workspace.members or workspace.default_member is a workspace, and each member directory has its own primary file. The member primary loads after the workspace primary, with the role Member and the priority 350. Then the adjacent files load in pairs, the workspace file first: the environment files, the local overrides, and the user overrides.

/w/acme.toml                        <- workspace primary: members = ["packages/*"]
/w/acme.user.toml                   <- workspace user override
/w/packages/billing/acme.toml       <- member primary
/w/packages/billing/acme.user.toml  <- member user override, which wins
/w/packages/billing/src/            <- the working directory

The walk-up finds the nearest primary first. Then discovery looks from the parent of that root up to the boundary for a workspace that has the root as a member. It passes over a directory with no file, with two files, with a file that does not parse, or with a file that is not a workspace. With no enclosing workspace, the found file is the project primary, and when it is a workspace, discovery selects a member in this order:

With none of them, no member source loads.

A member entry is a path relative to the workspace root, with / between segments. * matches any run of characters in one segment, and ** matches zero or more segments. Other characters, such as ? and [, match themselves. A literal entry, such as members/alpha, names its directory whether or not it exists, so a missing member reports NotFound with its candidates. A wildcard entry gives only the directories that hold a primary or legacy file, and it never enters a hidden directory. An entry that is absolute, starts with a drive, holds \, or holds a .. segment gives a Note warning, and discovery skips it. The workspace root is never its own member.

The members, exclude entries and default member come from the system file, the global user file and the workspace primary, merged. workspace(members?, exclude?, default_member?, project_name?) gives other key paths.

///|
test "the member file wins over the workspace file" {
  let builder = @config.builder()
    |> @config.working_dir("/w/packages/billing/src")
    |> @config.format(@toml.format())
    |> @conventions.add(@conventions.app_dirs("acme") |> @conventions.workspace)
  let tree = @config.FileTree::new(complete=true)
    .with_listing("/w", [
      { name: "acme.toml", kind: File },
      { name: "packages", kind: Directory },
    ])
    .with_listing("/w/packages", [{ name: "billing", kind: Directory }])
    .with_listing("/w/packages/billing", [
      { name: "acme.toml", kind: File },
      { name: "src", kind: Directory },
    ])
    .with_listing("/w/packages/billing/src", [])
    .with_file(
      "/w/acme.toml",
      "workspace.members = [\"packages/*\"]\nui.theme = \"light\"",
    )
    .with_file("/w/packages/billing/acme.toml", "ui.theme = \"dark\"")
  guard @config.load(builder, tree) is Done(config) else {
    fail("a complete tree never asks for files")
  }
  @expect.expect(@config.get_text(config, "ui.theme")).to_equal("dark")
  let roles = config
    .reports()
    .filter(report => report.kind is File(Workspace | Member))
    .map(report => (report.kind, report.priority, report.path))
  @expect.expect(roles).to_equal([
    (File(Workspace), 300, Some("/w/acme.toml")),
    (File(Member), 350, Some("/w/packages/billing/acme.toml")),
  ])
}

DiscoveryOptions(project=selector), for example from a --project flag, selects a project by name or path. . selects the root project. Then the selector matches a member path, such as packages/billing. Then it matches the project name of the root or of a member primary. No match is an error, UnknownProject. Two name matches are an error, AmbiguousProject, and the message asks for a member path. Without the workspace part, a given project option adds a Note warning.

Key filters

Two filters remove keys from layers through the core exclude. A removed value has no origin.

Use member_scope_keys for a key that only the workspace can set, such as an output directory. Use inherited_filter for the tables that describe the root project, so that a member does not inherit them.

///|
test "a member cannot set a workspace key and does not inherit the root project" {
  let conventions = @conventions.app_dirs("acme")
    |> @conventions.workspace
    |> @conventions.member_scope_keys(["workspace.out_dir"])
    |> @conventions.inherited_filter(["project"])
  let builder = @config.builder()
    |> @config.working_dir("/w/app")
    |> @config.format(@toml.format())
    |> @conventions.add(conventions)
  let tree = @config.FileTree::new(complete=true)
    .with_listing("/w", [
      { name: "acme.toml", kind: File },
      { name: "app", kind: Directory },
    ])
    .with_listing("/w/app", [{ name: "acme.toml", kind: File }])
    .with_file(
      "/w/acme.toml",
      "workspace.members = [\"app\"]\nworkspace.out_dir = \"out\"\nproject.name = \"root\"",
    )
    .with_file("/w/app/acme.toml", "workspace.out_dir = \"mine\"")
  guard @config.load(builder, tree) is Done(config) else {
    fail("a complete tree never asks for files")
  }
  @expect.expect(@config.get_text(config, "workspace.out_dir")).to_equal("out")
  @expect.expect(config.get(@config.KeyPath::parse("project.name"))).to_equal(
    None,
  )
  let member_report = config
    .reports()
    .iter()
    .find_first(report => report.kind == File(Member))
  @expect.expect(member_report.map(report => report.warnings.length())).to_equal(
    Some(1),
  )
}

Legacy fallback

legacy(name, format~, decode~) names an older file, for example acme.json. At a location with no primary file, the legacy file is the primary and marks the project root. Its format parses it but is not registered, so .json never becomes a searched extension, and decode turns the parsed value into a layer.

One directory at a time

With an incomplete tree, discovery asks for the listings of one directory per step, nearest first: the directory itself, .<app>/, and .config/<app>/. It stops asking once it finds the root, so it never lists more parents than it needs. A host answers each request and calls @config.load again.

///|
test "discovery asks for the listing of the working directory first" {
  let builder = @config.builder()
    |> @config.working_dir("/w/src")
    |> @config.format(@toml.format())
    |> @conventions.add(@conventions.app_dirs("acme"))
  guard @config.load(builder, @config.FileTree::new()) is Need(requests) else {
    fail("an empty tree needs a listing")
  }
  @expect.expect(requests).to_equal([
    ListDir("/w/src"),
    ListDir("/w/src/.acme"),
    ListDir("/w/src/.config/acme"),
  ])
}

Names