Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 169 additions & 0 deletions .github/scripts/check-toolchain-lanes.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
#!/usr/bin/env bash
# The two lanes that read this collection's C++ with something other than the
# default Linux toolchain. A CI step names the lane:
#
# bash .github/scripts/check-toolchain-lanes.sh llvm
# bash .github/scripts/check-toolchain-lanes.sh editor
#
# THREE COMBINATIONS READ THIS CODE, AND EACH ONE HAS REFUSED SOMETHING THE
# OTHERS ACCEPT. GCC with libstdc++ builds it by default on Linux. clang with
# libc++ builds it under `--toolchain llvm@…`, as on macOS. clang with
# libstdc++'s `std` module is what an editor's clangd uses to read a project
# built with GCC -- and no build ever compiled that combination, so a
# `std::pair<std::string, value>` inside `value` (0.19.1) failed only there:
# every build passed and every build program in an editor showed a module that
# was "not found". docs/plugin-development.md §4 states the rule this enforces.
#
# Environment: MCPP (llvm); MCPP_EDITOR and MCPPLS (editor).
set -euo pipefail

ROOT=$(cd "$(dirname "$0")/../.." && pwd)
LLVM="${MCPP_LLVM:-llvm@22.1.8}"

fail() { echo "FAIL: $*"; exit 1; }

# ── llvm: clang with libc++ ───────────────────────────────────────────────────
#
# FULL BUILDS for the fixtures whose build needs nothing tied to libstdc++, each
# with the criterion its program prints. `all-rules-compile` imports every
# member and calls it, so the whole collection is compiled under libc++ by its
# first line; the others run what the members produce.
#
# CONFIGURE ONLY for the Qt fixtures. Qt's official Linux binaries are built
# against libstdc++ and a libc++ program cannot link them, which is the Qt
# fixtures' business and not this lane's. `--configure-only` still compiles and
# runs the build program -- the member and the library under libc++ -- and that
# is the part this lane is for.
#
# NOT HERE: the fixtures that pin a device toolchain (cuda, hip, sycl,
# multi-rule) already build with llvm@22.1.8 in the consumers job, and the
# deps fixtures' libc++ rows are `vcpkg-libcxx` and `cmake-cache` there.
llvm_full=(
"all-rules-compile|^all-rules-compile ok"
"plugin-logic|"
"embed-consumer|^size=33 "
"embed-module-consumer|^all ok"
"embed-table-consumer|^all ok"
"island-interface|^all ok"
"archive-consumer|^archive-consumer: 'greetings from the archive'"
"spirv-module-consumer|^all ok"
)
llvm_configure_only=(qt-consumer qt-widgets-consumer qt-sdk-consumer qt-import-only)

llvm_lane() {
: "${MCPP:?MCPP names the mcpp under test}"
"$MCPP" toolchain install llvm "${LLVM#llvm@}"
local entry fixture expect results declared passed
for entry in "${llvm_full[@]}"; do
fixture=${entry%%|*}; expect=${entry#*|}
echo "── $fixture under $LLVM"
cd "$ROOT/tests/$fixture"
rm -rf target; mkdir -p target/ci
"$MCPP" build --toolchain "$LLVM" > target/ci/llvm-build.log 2>&1 ||
{ cat target/ci/llvm-build.log; fail "$fixture does not build under $LLVM"; }
grep -q -- '-stdlib=libc++' target/*/*/build.ninja ||
fail "$fixture was not linked against libc++: the lane did not build what it says it built"
if [ "$fixture" = plugin-logic ]; then
# The kit's verdicts, counted against the cases the fixture declares,
# as check-deps-and-qt.sh counts them under the default toolchain.
results=$(find target -path '*plugins-testing/results.txt' | head -1)
[ -n "$results" ] || fail "the kit wrote no results.txt under $LLVM"
declared=$(grep -cE '^ \{ "' build.mcpp)
passed=$(grep -c '^PASS ' "$results")
[ "$declared" -gt 0 ] && [ "$passed" -eq "$declared" ] ||
{ cat "$results"; fail "$passed of $declared plugin-logic cases passed under $LLVM"; }
echo "ok: $passed of $declared plugin-logic cases passed under $LLVM"
continue
fi
"$MCPP" run --toolchain "$LLVM" > target/ci/llvm-run.log 2>&1 ||
{ cat target/ci/llvm-run.log; fail "$fixture built under $LLVM and did not run"; }
grep -qE "$expect" target/ci/llvm-run.log ||
{ cat target/ci/llvm-run.log; fail "$fixture ran under $LLVM without printing '$expect'"; }
echo "ok: $fixture builds and runs under $LLVM"
done
for fixture in "${llvm_configure_only[@]}"; do
echo "── $fixture under $LLVM, configured"
cd "$ROOT/tests/$fixture"
rm -rf target; mkdir -p target/ci
"$MCPP" build --configure-only --toolchain "$LLVM" > target/ci/llvm-configure.log 2>&1 ||
{ cat target/ci/llvm-configure.log; fail "$fixture's build program does not compile or run under $LLVM"; }
echo "ok: $fixture's build program compiles and runs under $LLVM"
done
}

# ── editor: clang with libstdc++, through mcppls ──────────────────────────────
#
# What an editor shows for each fixture's `build.mcpp`, read by mcppls with its
# own clangd over the default (GCC) toolchain's standard library. A module the
# build program imports that clangd cannot compile is an error on the import
# line, so `counts.error` is the criterion; every diagnostic goes to the job
# summary, so a failure names the line without downloading anything.
#
# A FILE THAT WAS NOT READ IS A FAILURE, NOT A PASS. mcppls says so with
# `build-program-context` when the mcpp it runs describes no build programs
# (before 2026.10.11.1); zero errors from a file nobody compiled is the silent
# pass this lane exists to prevent.
editor_lane() {
: "${MCPP_EDITOR:?MCPP_EDITOR names the mcpp mcppls runs}"
: "${MCPPLS:?MCPPLS names the mcppls binary of the payload}"
export PATH="$(dirname "$MCPP_EDITOR"):$PATH"
export MCPPLS_CACHE_DIR="${RUNNER_TEMP:-/tmp}/mcppls-cache"
local summary="${GITHUB_STEP_SUMMARY:-/dev/null}" failed=0 checked=0 dir fixture out
{
echo "### build.mcpp in the editor ($("$MCPPLS" --version | head -1), $("$MCPP_EDITOR" --version | head -1))"
echo
echo "| fixture | errors | warnings | diagnostics |"
echo "|---|---|---|---|"
} >> "$summary"
for dir in "$ROOT"/tests/*/; do
fixture=$(basename "$dir")
[ -f "$dir/build.mcpp" ] || continue
echo "── $fixture"
out="${RUNNER_TEMP:-/tmp}/editor-$fixture.json"
# CONFIGURED FIRST, as a project is before anyone opens it: the
# configuration provisions what the build program's context names --
# the toolchain a fixture pins, the Qt SDK, a shader compiler -- and
# without it the build database describes no build program, so the
# editor finds not even `mcpp` (measured on a fresh runner).
(cd "$dir" && "$MCPP_EDITOR" build --configure-only) > "${out%.json}.configure.log" 2>&1 ||
{ cat "${out%.json}.configure.log"
echo "| $fixture | — | — | \`mcpp build --configure-only\` failed |" >> "$summary"; failed=1; continue; }
# mcppls exits 1 when it reports an error and still writes the report,
# so the report decides; only a run that wrote none is its own failure.
local status=0
(cd "$dir" && "$MCPPLS" diagnostics build.mcpp --format json --timeout 300) > "$out" || status=$?
python3 -c 'import json, sys; json.load(open(sys.argv[1]))' "$out" 2> /dev/null ||
{ echo "| $fixture | — | — | mcppls diagnostics exited $status without a report |" >> "$summary"; failed=1; continue; }
checked=$((checked + 1))
python3 - "$fixture" "$out" "$summary" <<'EOF' || failed=1
import json, sys
fixture, path, summary = sys.argv[1:4]
report = json.load(open(path))
counts = report["counts"]
found = [d for f in report["files"] for d in f["diagnostics"]]
unread = [d for d in found if d.get("code") == "build-program-context"]
incomplete = [f["file"] for f in report["files"] if not f.get("complete", True)]
cells = "<br>".join(
f"{d['severity']} {d['location']['file']}:{d['location']['line']} `{d.get('code', '')}` {d['message'][:240]}".replace("|", "\\|")
for d in found) or "none"
with open(summary, "a") as out:
out.write(f"| {fixture} | {counts['error']} | {counts['warning']} | {cells} |\n")
for d in found:
print(f" {d['severity']} {d['location']['file']}:{d['location']['line']} [{d.get('code', '')}] {d['message']}")
if counts["error"] or unread or incomplete:
why = "errors" if counts["error"] else "not read" if unread else "incomplete"
print(f"FAIL: {fixture}'s build.mcpp in the editor: {why}")
sys.exit(1)
print(f"ok: {fixture}'s build.mcpp has no error in the editor")
EOF
done
[ "$checked" -gt 0 ] || fail "no fixture's build.mcpp was checked"
[ "$failed" = 0 ] || fail "an editor reports errors in a fixture's build.mcpp (the job summary lists them)"
echo "ok: $checked build programs read in the editor without an error"
}

case "${1:-}" in
llvm) llvm_lane ;;
editor) editor_lane ;;
*) echo "usage: $0 llvm|editor"; exit 2 ;;
esac
103 changes: 103 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,15 @@ env:
# what it built. This is how a change to the engine is measured against this
# collection before either is released.
MCPP_SOURCE_REF: ${{ inputs.mcpp_source_ref || '' }}
# THE EDITOR LANE'S TWO PROGRAMS (0.19.1), PINNED. mcppls reads a fixture's
# `build.mcpp` as an editor does, with its own clangd over the GCC
# toolchain's libstdc++ -- the one combination no build compiles. The mcpp it
# runs is the first release that describes build programs in the build
# database (2026.10.11.1); under an older one mcppls reads no build program
# and says so, which the lane counts as a failure. Raising either is a
# one-line change here.
MCPPLS_VERSION: 0.0.15
MCPP_EDITOR_VERSION: 2026.10.11.1
# PINNED, AND WITHOUT IT THE CACHE BELOW CACHED NOTHING.
#
# A released mcpp is self-contained: with no `MCPP_HOME`, `mcpp self env`
Expand Down Expand Up @@ -1741,6 +1750,100 @@ jobs:
- name: a package that enables rules-qt only for its module builds without a report
run: bash .github/scripts/check-deps-and-qt.sh qt-import-only

# ── THE OTHER TWO WAYS THIS CODE IS COMPILED ON LINUX (0.19.1) ─────────────
#
# The consumers job builds with the default toolchain, GCC with libstdc++.
# Two more combinations read the same sources, and each has refused code the
# others accept: clang with libc++ (`--toolchain llvm@…`, macOS's family),
# and clang with libstdc++'s `std` module, which is how an editor's clangd
# reads a project built with GCC. `json::value` held a
# `std::pair<std::string, value>` that only the last one rejects, so every
# build passed while every build program showed a missing module in the
# editor. `.github/scripts/check-toolchain-lanes.sh` states what each lane
# runs; docs/plugin-development.md §4 states the rule.
toolchain-llvm:
name: llvm (linux x86_64, clang + libc++)
runs-on: ubuntu-24.04
env:
MCPP_HOME: /home/runner/.mcpp
timeout-minutes: 120
steps:
- uses: actions/checkout@v4

- name: Cache the mcpp sandbox
uses: actions/cache@v4
with:
path: ~/.mcpp
key: mcpp-sandbox-llvm-v1-${{ runner.os }}-${{ runner.arch }}-${{ env.MCPP_BOOTSTRAP }}-${{ hashFiles('tests/*/mcpp.toml') }}
restore-keys: |
mcpp-sandbox-llvm-v1-${{ runner.os }}-${{ runner.arch }}-${{ env.MCPP_BOOTSTRAP }}-

- name: Fetch the released mcpp
run: |
curl -L -fsS --retry 3 --retry-all-errors -o mcpp.tar.gz \
"https://github.com/mcpp-community/mcpp/releases/download/v${MCPP_BOOTSTRAP}/mcpp-${MCPP_BOOTSTRAP}-linux-x86_64.tar.gz"
tar -xzf mcpp.tar.gz
MCPP="$PWD/mcpp-${MCPP_BOOTSTRAP}-linux-x86_64/bin/mcpp"
"$MCPP" --version
# The bundled xlings, named because `MCPP_HOME` is pinned (see consumers).
export MCPP_VENDORED_XLINGS="$PWD/mcpp-${MCPP_BOOTSTRAP}-linux-x86_64/registry/bin/xlings"
"$MCPP" self config --mirror GLOBAL
echo "MCPP=$MCPP" >> "$GITHUB_ENV"
echo "MCPP_VENDORED_XLINGS=$MCPP_VENDORED_XLINGS" >> "$GITHUB_ENV"

- name: the mcpp under review, built from MCPP_SOURCE_REF when it is set
timeout-minutes: 45
run: bash .github/scripts/mcpp-under-review.sh

- name: the collection builds and runs under clang with libc++
run: bash .github/scripts/check-toolchain-lanes.sh llvm

toolchain-editor:
name: editor (linux x86_64, clangd + libstdc++)
runs-on: ubuntu-24.04
env:
MCPP_HOME: /home/runner/.mcpp
# Each fixture is configured before it is read, which provisions what its
# build program names (the device toolchains, the Qt SDK); a cold cache
# downloads all of them.
timeout-minutes: 120
steps:
- uses: actions/checkout@v4

- name: Cache the mcpp sandbox
uses: actions/cache@v4
with:
path: ~/.mcpp
key: mcpp-sandbox-editor-v1-${{ runner.os }}-${{ runner.arch }}-${{ env.MCPP_EDITOR_VERSION }}-${{ hashFiles('tests/*/mcpp.toml') }}
restore-keys: |
mcpp-sandbox-editor-v1-${{ runner.os }}-${{ runner.arch }}-${{ env.MCPP_EDITOR_VERSION }}-

- name: Fetch the mcpp the editor runs
run: |
curl -L -fsS --retry 3 --retry-all-errors -o mcpp.tar.gz \
"https://github.com/mcpp-community/mcpp/releases/download/v${MCPP_EDITOR_VERSION}/mcpp-${MCPP_EDITOR_VERSION}-linux-x86_64.tar.gz"
tar -xzf mcpp.tar.gz
MCPP_EDITOR="$PWD/mcpp-${MCPP_EDITOR_VERSION}-linux-x86_64/bin/mcpp"
export MCPP_VENDORED_XLINGS="$PWD/mcpp-${MCPP_EDITOR_VERSION}-linux-x86_64/registry/bin/xlings"
"$MCPP_EDITOR" --version
"$MCPP_EDITOR" self config --mirror GLOBAL
echo "MCPP_EDITOR=$MCPP_EDITOR" >> "$GITHUB_ENV"
echo "MCPP_VENDORED_XLINGS=$MCPP_VENDORED_XLINGS" >> "$GITHUB_ENV"

- name: Fetch the mcppls release payload, checked against its SHA256SUMS
run: |
base="https://github.com/Sunrisepeak/mcpp-language-server/releases/download/v${MCPPLS_VERSION}"
curl -L -fsS --retry 3 --retry-all-errors -o payload-linux-x64.tar.gz "$base/payload-linux-x64.tar.gz"
curl -L -fsS --retry 3 --retry-all-errors -o SHA256SUMS "$base/SHA256SUMS"
grep ' payload-linux-x64.tar.gz$' SHA256SUMS | sha256sum -c -
mkdir -p "$RUNNER_TEMP/mcppls" && tar -xzf payload-linux-x64.tar.gz -C "$RUNNER_TEMP/mcppls"
MCPPLS="$RUNNER_TEMP/mcppls/payload/bin/mcppls"
"$MCPPLS" --version
echo "MCPPLS=$MCPPLS" >> "$GITHUB_ENV"

- name: every fixture's build.mcpp reads without an error in the editor
run: bash .github/scripts/check-toolchain-lanes.sh editor

# ── THE SAME RULE ON THE OTHER TWO PLATFORMS ────────────────────────────────
#
# `rules-spirv` is the only member whose payload this ecosystem publishes for
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ module name the member declares, and configures it there.

```toml
[build-dependencies.mcpp]
plugins = { version = "0.19.0", features = ["rules-spirv"], host-module = true }
plugins = { version = "0.19.1", features = ["rules-spirv"], host-module = true }
```

```cpp
Expand Down Expand Up @@ -120,7 +120,11 @@ docs/ one page per member or group of members

A member is one module file, one feature in `mcpp.toml`, a consumer under
`tests/` that CI builds and asserts on, a row in the table above with its mcpp
floor, and a version bump. The conventions a member follows -- the same for a
floor, and a version bump. Its code is the C++23 that GCC with libstdc++, clang
with libc++ and an editor's clangd with libstdc++ all accept, and CI builds or
reads every fixture under each of the three
([docs/plugin-development.md](docs/plugin-development.md#4-the-c-a-plugin-is-written-in)).
The conventions a member follows -- the same for a
third-party or a project's own plugin -- are in
[docs/plugin-development.md](docs/plugin-development.md); the division between
what the engine owns and what the member owns is in
Expand Down
23 changes: 21 additions & 2 deletions docs/plugin-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ mcpp-index refuses such a package.

```toml
[build-dependencies.mcpp]
plugins = { version = "0.19.0", features = ["plugins-core"], host-module = true, reexport = true }
plugins = { version = "0.19.1", features = ["plugins-core"], host-module = true, reexport = true }
```

`reexport = true` is needed when the plugin's consumers import L2 modules in
Expand Down Expand Up @@ -124,7 +124,26 @@ what it keeps, since when, its retirement date (six months later), its
replacement and the note it prints once per build.
`.github/scripts/check-compat-retirement.sh` fails once a date has passed.

## 4. A new plugin
## 4. The C++ a plugin is written in

A plugin is C++23 that three combinations of compiler and standard library accept, because each of them compiles it for some user:

| combination | who compiles the plugin with it | CI |
|---|---|---|
| GCC 16 with libstdc++ | `mcpp build` on Linux by default | `consumers` |
| clang (LLVM 22) with libc++ | `mcpp build --toolchain llvm@22.1.8` on Linux, and every build on macOS | `toolchain-llvm`, `rules-cross-platform` |
| clang with libstdc++'s `std` module | an editor's clangd (mcppls) reading a project built with GCC | `toolchain-editor` |

MSVC is a fourth for the members that claim Windows, checked by `rules-cross-platform` and `deps-managed-msvc`. The third combination is never used by a build, so only the editor lane sees what it rejects; it runs `mcppls diagnostics build.mcpp` on every fixture and fails on any error.

Known differences, each one met in this package:

- **A type that holds a container of itself does not use `std::pair` or `std::tuple` for the element.** Under clang with libstdc++'s `std` module, `std::vector<std::pair<std::string, value>>` inside `value` instantiates the pair while `value` is incomplete and fails with "incomplete type used in type trait expression"; GCC and clang with libc++ accept it. The element is a plain struct, and a member function that walks the container is defined after both types are complete (`mcpp::plugins::json::member`, 0.19.1).
- **No compiler's leniency is relied on.** An extension, a missing `typename`, or a template accepted only because one compiler instantiates it later is a failure in another lane.
- **A build program does not use `std::println`.** On macOS 14 the build program links the system libc++, which lacks a symbol it needs; `std::format` into a string and `std::cout` work everywhere.
- **A module does not instantiate `std::filesystem::path`'s iterator.** Under MSVC 14.52 every importer that then touches `std::filesystem` fails inside the STL; `mcpp::plugins::names::components` splits a path as a string instead.

## 5. A new plugin

A plugin in this package consists of:
- a module file under `rules/`, `tools/`, `dist/` or `deps/`;
Expand Down
2 changes: 1 addition & 1 deletion mcpp.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "plugins"
namespace = "mcpp"
version = "0.19.0"
version = "0.19.1"
# The first mcpp release that states where each tool and payload comes from
# (protocol 15, mcpp#755): `xpkg_source`, `xpkg_program`, `xpkg_request`,
# `mcpp::decision`, `mcpp::toolchain`, `mcpp::phase`, and the
Expand Down
Loading
Loading