From d93fde45ff86eaaa0eac95d300248e738d775f23 Mon Sep 17 00:00:00 2001 From: David Abram Date: Tue, 23 Jun 2026 22:38:19 +0200 Subject: [PATCH 1/4] flatpak: Implement Nix-native packaging flow Move Flatpak manifest and cargo source generation into Nix, add parity checks for generated artifacts, and reduce the bash entrypoint to imperative Flatpak orchestration. Replace the Flatpak validator surface with Nix-wrapped Bash scripts Co-authored-by: SCE --- README.md | 38 +- context/architecture.md | 4 +- context/context-map.md | 2 +- context/glossary.md | 13 +- context/overview.md | 6 +- context/patterns.md | 15 +- .../flatpak-python-validators-to-bash.md | 120 +++ context/plans/nix-native-flatpak-release.md | 133 +++ .../cli-first-install-channels-contract.md | 21 +- context/sce/cli-release-artifact-contract.md | 2 +- ...all-channel-integration-test-entrypoint.md | 2 +- flake.lock | 17 + flake.nix | 135 ++- nix/flatpak/cargo-sources.nix | 74 ++ nix/flatpak/local-manifest-validate.sh | 68 ++ nix/flatpak/manifest.nix | 150 +++ nix/flatpak/static-validate.sh | 135 +++ nix/flatpak/version-parity.sh | 123 +++ packaging/flatpak/cargo-sources.json | 140 ++- packaging/flatpak/dev.crocoder.sce.yml | 66 +- packaging/flatpak/sce-flatpak.sh | 866 ++++-------------- 21 files changed, 1287 insertions(+), 843 deletions(-) create mode 100644 context/plans/flatpak-python-validators-to-bash.md create mode 100644 context/plans/nix-native-flatpak-release.md create mode 100644 nix/flatpak/cargo-sources.nix create mode 100644 nix/flatpak/local-manifest-validate.sh create mode 100644 nix/flatpak/manifest.nix create mode 100644 nix/flatpak/static-validate.sh create mode 100755 nix/flatpak/version-parity.sh diff --git a/README.md b/README.md index 8945c070..68b31956 100644 --- a/README.md +++ b/README.md @@ -72,40 +72,41 @@ Freedesktop SDK Rust extension — it does not wrap a prebuilt Nix, Cargo, or np - The [Freedesktop SDK](https://docs.flatpak.org/en/latest/available-runtimes.html) runtime and SDK extension are downloaded automatically by flatpak-builder when needed. -#### Preferred path: Nix-backed workflow +#### Preferred path: Nix-native workflow If you are working from the repository checkout and have Nix available, use the -Nix-backed entrypoints. They provide Flatpak tooling, generate a local-checkout -manifest, and run validation without bypassing the Flatpak source build. +Nix-native entrypoints. Nix owns the checked-in Flatpak manifest generation, +Cargo source enumeration, and static/version-parity validation; the bash helper +only orchestrates the imperative `flatpak-builder` / `flatpak build-bundle` +steps that need network and bubblewrap access. ```bash # Enter the dev shell with Flatpak tooling (Linux only) nix develop # Validate packaging metadata and local-source manifest generation -nix run .#flatpak-validate +nix run .#sce-flatpak -- validate --skip-optional-lint # Generate a Flatpak manifest that builds from the current checkout -nix run .#flatpak-local-manifest +nix run .#sce-flatpak -- prepare-local-manifest --repo-root "$PWD" --out-dir /tmp/sce-flatpak-manifest -# Build the Flatpak from the current checkout -nix run .#flatpak-build -- --help +# Build release assets from the current checkout +nix run .#release-flatpak-package -- --help +nix run .#release-flatpak-bundle -- --help ``` -The `nix run .#flatpak-build` command accepts the same arguments as -`sce-flatpak build` (see `--help`). For example, to build and install -into your user installation: +Regenerate checked-in Flatpak packaging artifacts after changing their sources: ```bash -nix run .#flatpak-build -- \ - --install --user \ - --install-deps-from=flathub +nix run .#regenerate-flatpak-manifest +nix run .#regenerate-cargo-sources ``` The default `nix flake check` runs lightweight static validation -(`flatpak-static-validation`) without a full Flatpak build. Full builds -are opt-in via `nix run .#flatpak-build` and require network access for -SDK runtime downloads. +(`flatpak-static-validation`) plus manifest and cargo-source parity checks +without a full Flatpak build. Full source-built bundles remain opt-in through +`nix run .#release-flatpak-bundle` and require network access for SDK runtime +downloads. #### GitHub Release source-manifest assets @@ -187,8 +188,9 @@ flatpak-builder \ sourced from the checked-in `cargo-sources.json` and are still built offline inside Flatpak. -The local manifest is produced by `nix run .#flatpak-local-manifest` or -`sce-flatpak.sh prepare-local-manifest`. It is never committed; it lives in a +The local manifest is produced by +`nix run .#sce-flatpak -- prepare-local-manifest --repo-root "$PWD" --out-dir ` +or `sce-flatpak.sh prepare-local-manifest`. It is never committed; it lives in a temporary or user-specified output directory. #### Run the Flatpak diff --git a/context/architecture.md b/context/architecture.md index f653b6e7..e819b8b1 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -80,7 +80,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - The current implemented binary install/distribution surface for the `sce` CLI includes repo-flake Nix, Cargo, and npm; `Homebrew` is deferred from the active implementation stage. - Flatpak is also an official supported distribution channel for this implementation stage under application ID `dev.crocoder.sce`, with a source-built Flatpak package rather than a wrapper around Nix-built, native GitHub Release binary, npm native, or other prebuilt `sce` artifacts. - Nix-managed build/release entrypoints are the source of truth for existing binary build outputs and release automation, and Nix may orchestrate Flatpak tooling, local-source overrides, and validation without becoming the Flatpak package's binary source. -- The implemented Flatpak packaging/tooling surface lives under `packaging/flatpak/`: `dev.crocoder.sce.yml` is the Flathub-style source-build manifest, `cargo-sources.json` is generated from `cli/Cargo.lock` for offline Cargo sources, `dev.crocoder.sce.metainfo.xml` is the AppStream console-app metadata, `git-host-bridge` is installed as `/app/bin/git` to delegate runtime Git access to `flatpak-spawn --host git` with the required `org.freedesktop.Flatpak` permission, and `sce-flatpak.sh` generates local checkout-source manifests plus validation/build orchestration for the flake apps/check. +- The implemented Flatpak packaging/tooling surface lives under `packaging/flatpak/` with Nix-owned generation: `dev.crocoder.sce.yml` is the Flathub-style source-build manifest rendered from a Nix expression (`nix/flatpak/manifest.nix`) via the standard nixpkgs YAML formatter (`pkgs.formats.yaml.generate`) and regenerated by `nix run .#regenerate-flatpak-manifest`; `cargo-sources.json` is produced from `cli/Cargo.lock` by a Nix derivation (`nix/flatpak/cargo-sources.nix` wrapping `flatpak-builder-tools`/`flatpak-cargo-generator.py`) and regenerated by `nix run .#regenerate-cargo-sources`; both are guarded by parity checks (`flatpak-manifest-parity`, `cargo-sources-parity`). Static manifest validation is Bash-owned at `nix/flatpak/static-validate.sh` and wrapped by Nix as `flatpak-static-check`; release-version parity validation is Bash-owned at `nix/flatpak/version-parity.sh` and wrapped by Nix as `flatpak-version-parity-check`. `dev.crocoder.sce.metainfo.xml` is the AppStream console-app metadata, `git-host-bridge` is installed as `/app/bin/git` to delegate runtime Git access to `flatpak-spawn --host git` with the required `org.freedesktop.Flatpak` permission, and `sce-flatpak.sh` is the thin imperative orchestrator (no manifest rewriting, no embedded Python) around `flatpak-builder` + `flatpak build-bundle`. - GitHub Release Flatpak source-manifest assets are implemented for the active release model as packaging metadata only: `nix run .#release-flatpak-package -- --version --out-dir ` packages `sce-v-flatpak-manifest.tar.gz`, its `.sha256`, and `sce-v-flatpak.json` from the Flatpak manifest/support files with the staged manifest pinned to the release commit. - Flatpak GitHub Release assets now also include source-built `.flatpak` bundle assets (`sce-v-x86_64.flatpak` / `sce-v-aarch64.flatpak` plus `.sha256` / `.json`) alongside source-manifest packaging metadata. - Automatic Flathub submission, prebuilt (non-source-built) Flatpak binaries/bundles, OSTree repositories, release-version bumping, and Flatpak publication beyond the approved GitHub Release asset upload remain outside the current iteration. @@ -135,7 +135,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/README.md` is the crate-local onboarding and usage source of truth for placeholder behavior, safety limitations, and roadmap mapping back to service contracts. - `flake.nix` applies `rust-overlay` (`oxalica/rust-overlay`) to nixpkgs, pins `rust-bin.stable.1.95.0.default` with `rustfmt` + `clippy`, reads the package/check version from repo-root `.version`, builds `packages.sce` through Crane (`buildDepsOnly` -> `buildPackage`) with a filtered repo-root source that preserves the Cargo tree plus `cli/assets/hooks`, then injects generated OpenCode/Claude config payloads and schema inputs into a temporary `cli/assets/generated/` mirror during derivation unpack so `cli/build.rs` can package the crate without requiring committed generated crate assets. The same `cli/build.rs` now scans `cli/migrations/*/*.sql` and writes `cli/src/generated_migrations.rs` with deterministic migration constants sorted by numeric filename prefix. - The root flake runs `cli-tests`, `cli-clippy`, and `cli-fmt` plus the dedicated `integrations-install-tests`, `integrations-install-clippy`, and `integrations-install-fmt` derivations through Crane-backed paths so both Rust crates have first-class default-flake verification; it also exposes directory-scoped JS validation derivations for both `npm/` and the shared `config/lib/` plugin package root. -- Flatpak flake tooling is Linux-only: `apps.flatpak-validate` runs `packaging/flatpak/sce-flatpak.sh validate`, `apps.flatpak-local-manifest` emits a generated checkout-source manifest, `apps.flatpak-build` runs opt-in `flatpak-builder` builds from that generated local manifest, `apps.release-flatpak-package` emits deterministic Flatpak GitHub Release source-manifest assets, `apps.release-flatpak-bundle` builds a source-built `.flatpak` bundle, `checks..flatpak-static-validation` performs lightweight static/AppStream validation without a full build, and the Linux dev shell includes `appstreamcli`, `flatpak`, and `flatpak-builder`. +- Flatpak flake tooling is Linux-only and reduced to a minimal app surface: `apps.sce-flatpak` is the umbrella entrypoint (`nix run .#sce-flatpak -- ` for `validate`, `prepare-local-manifest --repo-root `, etc.) that delegates to `packaging/flatpak/sce-flatpak.sh`; `apps.release-flatpak-package` emits deterministic Flatpak GitHub Release source-manifest assets; `apps.release-flatpak-bundle` builds a source-built `.flatpak` bundle; helper apps `apps.regenerate-flatpak-manifest` and `apps.regenerate-cargo-sources` rewrite the checked-in generated artifacts from their Nix sources. The standalone `flatpak-validate`, `flatpak-local-manifest`, and `flatpak-build` wrapper apps are removed in favor of `sce-flatpak `. `checks..flatpak-static-validation` runs the Nix-built static validator script during default `nix flake check`, and `checks..flatpak-manifest-parity` plus `checks..cargo-sources-parity` enforce the generated-artifact parity contracts; default `nix flake check` does not run `flatpak-builder`. The Linux dev shell includes `appstreamcli`, `flatpak`, and `flatpak-builder`. - The root flake also exposes the non-default `apps.install-channel-integration-tests` flake app for npm/Bun/Cargo install-channel integration coverage outside the default check set. The shared config-lib source set is rooted at `config/lib/` and includes the shared `package.json`, `bun.lock`, and `tsconfig.json` plus `agent-trace-plugin/` and `bash-policy-plugin/`; `config-lib-bun-tests` runs the bash-policy plugin wrapper tests from that shared root, while `config-lib-biome-check` and `config-lib-biome-format` run Biome over the copied shared package source. `.github/workflows/publish-crates.yml` follows the same asset-preparation rule but runs Cargo packaging from a temporary clean repository copy so crates.io publish no longer needs `--allow-dirty`. - The config-lib check source preserves repo-relative access to shared CLI patch fixtures: Nix copies a filtered repo-shaped source containing `config/lib/**` plus `cli/src/services/structured_patch/fixtures`, then runs Bun/Biome from `config/lib/`. - `flake.nix` exposes release install/run surfaces as `packages.sce` (`packages.default = packages.sce`) plus `apps.sce` and `apps.default`, all targeting `${packages.sce}/bin/sce`; this keeps repo-local and remote flake run/install flows (`nix run .`, `nix run github:crocoder-dev/shared-context-engineering`, `nix profile install github:crocoder-dev/shared-context-engineering`) aligned to the same packaged CLI output. diff --git a/context/context-map.md b/context/context-map.md index 40058d60..5e0d65d4 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -55,7 +55,7 @@ Feature/domain context: - `context/sce/bash-tool-policy-enforcement-contract.md` (approved bash-tool blocking contract plus current Rust evaluator seam and OpenCode/Claude delegation references, including config schema, argv-prefix matching, shell/nix unwrapping, fixed preset catalog/messages, and precedence rules) - `context/sce/generated-opencode-plugin-registration.md` (current generated OpenCode plugin-registration contract, canonical Pkl ownership, generated manifest/plugin paths including `sce-bash-policy` + `sce-agent-trace`, TypeScript source ownership, and Claude generated settings boundary including Agent Trace hooks plus `PreToolUse` Bash policy hook registration) - `context/sce/opencode-agent-trace-plugin-runtime.md` (current OpenCode agent-trace plugin runtime behavior, including captured `message.updated` handoff with `summary.diffs` branching: when diffs exist sends `-patch` variant `message.updated` + per-diff `message.part.updated` with `part_type: "patch"` concurrently via `Promise.all`, when no diffs sends original `message.updated` payload; in-memory dedup `Set` keyed by `"${sessionID}:${messageID}"`; captured `message.part.updated` handoff to `sce hooks conversation-trace` as `{ type: "message.part.updated", payloads: [{ session_id, message_id, part_type, text, generated_at_unix_ms }] }` with `text`/`reasoning` only; existing user-message diff extraction for `{ sessionID, diff, time, model_id }`; session-scoped OpenCode client version capture from `session.created`/`session.updated`; and CLI handoff to `sce hooks diff-trace` over STDIN JSON with required `tool_name="opencode"` plus required nullable `tool_version`; Rust hook parsing and AgentTraceDb insertion persist required payload fields including `model_id`) -- `context/sce/cli-first-install-channels-contract.md` (current `sce` install/distribution contract covering supported Nix/Cargo/npm plus source-built Flatpak channel scope, implemented GitHub Release Flatpak source-manifest and source-built `.flatpak` bundle asset packaging, canonical `.version` release authority, Nix-owned build/release policy, Nix-orchestrated Flatpak local/release tooling through `packaging/flatpak/sce-flatpak.sh`, Linux flake apps/checks `flatpak-validate` / `flatpak-local-manifest` / `flatpak-build` / `release-flatpak-package` / `release-flatpak-bundle` / `flatpak-static-validation`, the implemented `packaging/flatpak/dev.crocoder.sce.yml` + AppStream/Cargo-source packaging surface, and the `dev.crocoder.sce` host-git bridge decision) +- `context/sce/cli-first-install-channels-contract.md` (current `sce` install/distribution contract covering supported Nix/Cargo/npm plus source-built Flatpak channel scope, implemented GitHub Release Flatpak source-manifest and source-built `.flatpak` bundle asset packaging, canonical `.version` release authority, Nix-owned build/release policy, Nix-owned Flatpak manifest generation via `pkgs.formats.yaml.generate` plus Nix-owned cargo-sources generation from `cli/Cargo.lock` plus dedicated Nix-built Bash validator scripts for static, version-parity, and local-manifest validation, thin imperative `packaging/flatpak/sce-flatpak.sh` orchestration around `flatpak-builder` / `flatpak build-bundle`, the reduced Linux flake app surface (`sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`, plus `regenerate-flatpak-manifest` / `regenerate-cargo-sources` helpers) with `flatpak-static-validation` / `flatpak-manifest-parity` / `cargo-sources-parity` checks, the implemented `packaging/flatpak/dev.crocoder.sce.yml` + AppStream/Cargo-source packaging surface as generated artifacts, and the `dev.crocoder.sce` host-git bridge decision) - `context/sce/optional-install-channel-integration-test-entrypoint.md` (current opt-in flake app contract for existing binary install-channel integration coverage, including thin flake delegation to the Rust runner, shared harness ownership, real npm+Bun+Cargo install flows, channel selector semantics, the explicit non-default execution boundary, and the separate Flatpak source-build validation boundary) - `context/sce/cli-release-artifact-contract.md` (shared `sce` binary release artifact naming, checksum/manifest outputs, GitHub Releases as the canonical artifact publication surface, the current three-target Linux/macOS release workflow topology, implemented Flatpak source-manifest and source-built `.flatpak` bundle package assets uploaded by `.github/workflows/release-sce.yml`, and Flatpak's explicit source-built non-binary exception) - `context/sce/cli-npm-distribution-contract.md` (implemented `sce` npm launcher package, release-manifest/checksum-verified native binary install flow, the supported darwin/arm64 plus linux x64+arm64 npm platform matrix, and dedicated `.github/workflows/publish-npm.yml` downstream npm publish-stage contract) diff --git a/context/glossary.md b/context/glossary.md index 42cf0037..1b36e952 100644 --- a/context/glossary.md +++ b/context/glossary.md @@ -152,10 +152,13 @@ - `auto-allow`: Automated profile gate action that proceeds without confirmation; used for safe operations like file reads, skill loading, and context-only edits. - `auto-proceed`: Automated profile gate action that continues execution with logging but without waiting for confirmation; used for implementation stop and commit staging gates. - `sce install channel contract`: Current `sce` install/distribution scope captured in `context/sce/cli-first-install-channels-contract.md`: supported channels are repo-flake Nix, Cargo, npm, and source-built Flatpak with application ID `dev.crocoder.sce`; `Homebrew` is deferred from the active implementation stage. -- `Flatpak source-built channel`: Official `sce` distribution channel for application ID `dev.crocoder.sce` where `packaging/flatpak/dev.crocoder.sce.yml` builds the Rust CLI from source inside Flatpak rather than consuming Nix-built, GitHub Release, npm native, or other prebuilt `sce` artifacts. `packaging/flatpak/cargo-sources.json` is the checked-in Cargo source descriptor generated from `cli/Cargo.lock`, and local builds use a Flathub-style release-source manifest plus a Nix-generated checkout-source manifest produced by `packaging/flatpak/sce-flatpak.sh`. +- `Flatpak source-built channel`: Official `sce` distribution channel for application ID `dev.crocoder.sce` where `packaging/flatpak/dev.crocoder.sce.yml` builds the Rust CLI from source inside Flatpak rather than consuming Nix-built, GitHub Release, npm native, or other prebuilt `sce` artifacts. Both `packaging/flatpak/dev.crocoder.sce.yml` and `packaging/flatpak/cargo-sources.json` are checked-in generated artifacts: the manifest YAML is rendered from a Nix expression (`nix/flatpak/manifest.nix`) via the standard nixpkgs YAML formatter (`pkgs.formats.yaml.generate`, alpha-sorted block-style YAML produced via remarshal), and `cargo-sources.json` is produced from `cli/Cargo.lock` by a Nix derivation wrapping `flatpak-builder-tools`/`flatpak-cargo-generator.py`; both are guarded by `flatpak-manifest-parity` and `cargo-sources-parity` flake checks. The Nix manifest function exposes release-pin, local-checkout-override (Flatpak `type: dir`), and commit-pinned-for-release-package flavors; `packaging/flatpak/sce-flatpak.sh` is a thin imperative orchestrator around `flatpak-builder` and `flatpak build-bundle`, with no manifest text rewriting and no embedded Python. - `Flatpak host-git bridge`: Runtime Git access decision for the `sce` Flatpak package: `packaging/flatpak/git-host-bridge` is installed as `/app/bin/git` and delegates to `flatpak-spawn --host git`, with the Flatpak permission for `org.freedesktop.Flatpak` explicitly present in the manifest. -- `Flatpak Nix orchestration apps`: Linux-only root-flake apps `flatpak-validate`, `flatpak-local-manifest`, `flatpak-build`, `release-flatpak-package`, and `release-flatpak-bundle`; they delegate to `packaging/flatpak/sce-flatpak.sh` for static/AppStream validation, generated local checkout-source manifests, explicit opt-in `flatpak-builder` source builds, and deterministic Flatpak source-manifest and source-built `.flatpak` bundle release packaging. -- `flatpak-static-validation`: Linux-only root-flake check that runs the lightweight Flatpak validation path during default `nix flake check`; it uses static source-build assertions, generated local-manifest validation, and `appstreamcli validate --pedantic --no-net` without running a full Flatpak build. +- `Flatpak Nix orchestration apps`: Reduced Linux-only root-flake app surface for Flatpak: the umbrella `sce-flatpak` (`nix run .#sce-flatpak -- ` for `validate`, `prepare-local-manifest --repo-root `, etc.), plus `release-flatpak-package` and `release-flatpak-bundle` for release asset emission, plus helper apps `regenerate-flatpak-manifest` and `regenerate-cargo-sources` for the checked-in generated artifacts. The previously separate `flatpak-validate`, `flatpak-local-manifest`, and `flatpak-build` wrapper apps are removed in favor of `sce-flatpak `. `flatpak-builder` and `flatpak build-bundle` invocations stay imperative (network + bubblewrap). +- `flatpak-static-validation`: Linux-only root-flake check that runs the dedicated Nix-built Bash validator script (`nix/flatpak/static-validate.sh`) during default `nix flake check`; it uses static source-build assertions, generated-manifest validation, cargo-sources JSON checks, and AppStream metadata checks without running a full Flatpak build and without invoking the bash entrypoint or any inline `python3 - <<'PY'` heredoc. +- `flatpak-manifest-parity`: Linux-only root-flake check that fails when checked-in `packaging/flatpak/dev.crocoder.sce.yml` drifts from the YAML rendered by the Nix manifest expression for the release-pin flavor; regenerate via `nix run .#regenerate-flatpak-manifest`. +- `cargo-sources-parity`: Deterministic offline root-flake check that fails when checked-in `packaging/flatpak/cargo-sources.json` drifts from the JSON produced by the Nix derivation that wraps `flatpak-builder-tools`/`flatpak-cargo-generator.py` over `cli/Cargo.lock`; regenerate via `nix run .#regenerate-cargo-sources`. +- `flatpak-version-parity-check`: Dedicated Nix-built Bash validator (`nix/flatpak/version-parity.sh`) that validates release-version parity across `.version`, `cli/Cargo.toml`, `npm/package.json`, and Flatpak AppStream release metadata; consumed by `release-flatpak-package` / `release-flatpak-bundle` in place of the former inline/Python validation path. - `sce release-artifacts app`: Root-flake app exposed as `nix run .#release-artifacts`; packages the current-platform `sce` binary release archive, SHA-256 checksum file, and per-platform metadata fragment using the canonical release artifact naming contract. - `sce release-manifest app`: Root-flake app exposed as `nix run .#release-manifest`; merges per-platform `sce` metadata fragments into the release-level manifest JSON, emits detached signature `sce-v-release-manifest.json.sig`, and writes the combined `SHA256SUMS` file. - `SCE_RELEASE_MANIFEST_SIGNING_KEY`: Release-only private-key env var consumed by `nix run .#release-manifest` / `.github/workflows/release-sce.yml` to sign `sce-v-release-manifest.json`; it must never be committed or shipped in the npm package. @@ -163,9 +166,9 @@ - `publish-npm workflow`: Dedicated npm registry publication workflow at `.github/workflows/publish-npm.yml` that triggers from a published GitHub release or manual dispatch, validates `.version`/`npm/package.json`/release-tag parity, downloads the canonical `sce-v-npm.tgz` GitHub release asset, verifies the tarball's embedded package metadata, and publishes that already-versioned package to npm. - `sce release authority contract`: Approved release topology where repo-root `.version` is the canonical checked-in release version source, GitHub Releases are the canonical publication surface for signed release artifacts, and Cargo/npm registry publication are separate downstream publish stages that consume already-versioned checked-in package metadata without workflow-side version bumping. - `sce release-npm-package app`: Root-flake app exposed as `nix run .#release-npm-package`; stages the checked-in npm package, rewrites the requested version, runs `npm pack`, and emits `sce-v-npm.tgz` plus `sce-v-npm.json` for release publication. -- `sce release-flatpak-package app`: Linux-only root-flake app exposed as `nix run .#release-flatpak-package -- --version --out-dir `; validates requested version parity across `.version`, `cli/Cargo.toml`, `npm/package.json`, and Flatpak AppStream release metadata, requires a resolvable git release commit, stages `packaging/flatpak/` manifest/support files without mutating checked-in sources, rewrites only the staged manifest commit, and emits deterministic Flatpak source-manifest tarball/checksum/JSON metadata. +- `sce release-flatpak-package app`: Linux-only root-flake app exposed as `nix run .#release-flatpak-package -- --version --out-dir `; runs the Nix-built `flatpak-version-parity-check` script (parity across `.version`, `cli/Cargo.toml`, `npm/package.json`, and Flatpak AppStream release metadata), requires a resolvable git release commit, stages `packaging/flatpak/` manifest/support files without mutating checked-in sources, uses the Nix manifest expression's commit-pinned flavor to emit the staged manifest, and produces deterministic Flatpak source-manifest tarball/checksum/JSON metadata. -- `sce release-flatpak-bundle app`: Linux-only root-flake app exposed as `nix run .#release-flatpak-bundle -- --version --arch --out-dir `; validates version parity, runs `flatpak-builder --force-clean --arch=` against a generated local checkout-source manifest, emits a source-built `.flatpak` bundle via `flatpak build-bundle`, and produces SHA-256 checksum and JSON metadata (`asset_type: flatpak-bundle`); used by `.github/workflows/release-sce-linux.yml` (x86_64) and `.github/workflows/release-sce-linux-arm.yml` (aarch64). +- `sce release-flatpak-bundle app`: Linux-only root-flake app exposed as `nix run .#release-flatpak-bundle -- --version --arch --out-dir `; runs the Nix-built `flatpak-version-parity-check` script, uses the Nix manifest expression's local-checkout-override flavor to produce a Flatpak `type: dir` manifest, runs imperative `flatpak-builder --force-clean --arch=` plus `flatpak build-bundle`, and produces SHA-256 checksum and JSON metadata (`asset_type: flatpak-bundle`); used by `.github/workflows/release-sce-linux.yml` (x86_64) and `.github/workflows/release-sce-linux-arm.yml` (aarch64). - `sce split platform release workflows`: CLI release automation topology where `.github/workflows/release-sce.yml` orchestrates reusable per-platform workflow files; the current reusable workflow set and active orchestrated release matrix are `release-sce-linux.yml`, `release-sce-linux-arm.yml`, and `release-sce-macos-arm.yml`, producing the current automated release target set `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`, and `aarch64-apple-darwin`. - `publish-crates workflow`: Dedicated crates.io publish automation in `.github/workflows/publish-crates.yml` that runs after a GitHub release is published (or by manual dispatch), validates `.version`, `cli/Cargo.toml`, and the requested release tag remain aligned, supports a dry-run validation path, and requires `CARGO_REGISTRY_TOKEN` for real publication. - `clean publish workspace`: Temporary `.git`-free copy of the checked-out repository used by the crates.io publish workflow so ephemeral generated crate assets can be prepared locally without forcing Cargo publish to run with `--allow-dirty`. diff --git a/context/overview.md b/context/overview.md index 3031d4e1..c53821bb 100644 --- a/context/overview.md +++ b/context/overview.md @@ -38,12 +38,12 @@ Config-lib JS flake checks execute from `config/lib/`, but the copied Nix check Local developer Nix tuning guidance now lives in `AGENTS.md`, including optional user-level `~/.config/nix/nix.conf` recommendations for `max-jobs` and `cores` plus an explicit system-level-only note for `auto-optimise-store`. The Pkl authoring layer owns generated OpenCode plugin registration for SCE-managed plugins: `config/pkl/base/opencode.pkl` defines the canonical plugin entries, `config/pkl/renderers/common.pkl` re-exports the shared plugin list for renderer use, and generated `config/.opencode/opencode.json` plus `config/automated/.opencode/opencode.json` register `./plugins/sce-bash-policy.ts` and `./plugins/sce-agent-trace.ts` through OpenCode's `plugin` field. Claude does not use an OpenCode-style plugin manifest; Claude bash-policy enforcement is registered through generated `.claude/settings.json` as a `PreToolUse` `Bash` command hook running `sce policy bash`. The current CLI install/distribution contract for `sce` includes repo-flake Nix, Cargo, npm, and source-built Flatpak (`dev.crocoder.sce`) as supported channels, while `Homebrew` remains deferred from the current implementation stage. Nix-managed build/release entrypoints are the source of truth for existing binary rollout surfaces, npm consumes Nix-produced release artifacts, and repo-root `.version` is the canonical checked-in release version source that release packaging and downstream Cargo/npm publication must match. Flatpak is the approved source-built exception to binary artifact reuse: its package must build the Rust CLI from source inside Flatpak, use a Flathub-style release-source manifest plus a Nix-generated local checkout override for local builds, prepare generated CLI assets from checked-in `config/` inputs, and provide Git access through a `/app/bin/git` wrapper delegating to `flatpak-spawn --host git` with the required `org.freedesktop.Flatpak` permission. The active Flatpak release contract approves GitHub Release source-manifest assets (manifest tarball, checksum, and JSON metadata) and source-built `.flatpak` bundle assets (`sce-v-x86_64.flatpak` / `sce-v-aarch64.flatpak` plus `.sha256` / `.json`), with `.github/workflows/release-sce.yml` building/uploading those assets alongside CLI/npm assets, while still excluding automatic Flathub submission, prebuilt (non-source-built) Flatpak binaries/bundles, OSTree repositories, and release-version bumping. The shared release artifact foundation is now implemented through root-flake apps `release-artifacts` and `release-manifest`, which emit canonical `sce-v-.tar.gz` archives, SHA-256 checksum files, merged manifest outputs, and a detached `sce-v-release-manifest.json.sig` produced from a non-repo private signing key; the npm distribution surface is now implemented as a checked-in `npm/` launcher package plus root-flake `release-npm-package`, which packs `sce-v-npm.tgz`, refuses mismatched checked-in package metadata, and installs the native CLI by downloading the release manifest plus detached signature, verifying the manifest with the bundled npm public key, and only then checksum-verifying the matching GitHub release archive at npm `postinstall` time. GitHub Releases remain the canonical publication surface for binary release artifacts and approved Flatpak source-manifest package assets, while crates.io and npm registry publication are separate non-bumping publish stages under the approved release topology. GitHub CLI release automation now lives in dedicated `release-sce*.yml` workflows split by Linux, Linux ARM, and macOS ARM, and `.github/workflows/release-sce.yml` now orchestrates those three reusable platform lanes before assembling the signed release manifest, npm tarball, and GitHub release payload. The orchestrator tags/releases the checked-in `.version` directly and rejects version mismatches instead of generating a new semver during workflow execution; `.github/workflows/publish-crates.yml` and `.github/workflows/publish-npm.yml` own registry publication after release assets exist. -The Linux root flake now also exposes `nix run .#release-flatpak-package -- --version --out-dir `, delegating to `packaging/flatpak/sce-flatpak.sh release-package` to emit deterministic Flatpak source-manifest tarball/checksum/JSON release assets from checked-in packaging source while validating `.version`, `cli/Cargo.toml`, `npm/package.json`, and AppStream release metadata; `.github/workflows/release-sce.yml` runs that app into `dist/flatpak` and uploads `*.tar.gz`, `*.sha256`, and `*.json` Flatpak assets to the GitHub Release. Linux root flake also exposes `nix run .#release-flatpak-bundle -- --version --arch --out-dir `, delegating to `sce-flatpak.sh release-bundle` to build a source-built `.flatpak` bundle from the checkout using `flatpak-builder` + `flatpak build-bundle`, emitting per-architecture `.flatpak`/`.sha256`/`.json` files; `.github/workflows/release-sce-linux.yml` and `.github/workflows/release-sce-linux-arm.yml` build and upload x86_64/aarch64 bundles respectively, assembled by `.github/workflows/release-sce.yml`. -The checked-in Flatpak packaging surface lives under `packaging/flatpak/` with `dev.crocoder.sce.yml`, AppStream metadata, host-git wrapper source, the Cargo source descriptor generated from `cli/Cargo.lock`, and `sce-flatpak.sh` as the local Nix orchestration script used by flake apps/checks and Flatpak source-manifest release packaging. +The Linux root flake now also exposes `nix run .#release-flatpak-package -- --version --out-dir `, delegating to `packaging/flatpak/sce-flatpak.sh release-package` to emit deterministic Flatpak source-manifest tarball/checksum/JSON release assets from checked-in packaging source while running the Nix-built version-parity validator script across `.version`, `cli/Cargo.toml`, `npm/package.json`, and AppStream release metadata; `.github/workflows/release-sce.yml` runs that app into `dist/flatpak` and uploads `*.tar.gz`, `*.sha256`, and `*.json` Flatpak assets to the GitHub Release. Linux root flake also exposes `nix run .#release-flatpak-bundle -- --version --arch --out-dir `, delegating to `sce-flatpak.sh release-bundle` to build a source-built `.flatpak` bundle from the checkout using imperative `flatpak-builder` + `flatpak build-bundle` (network + bubblewrap, kept out of pure Nix), emitting per-architecture `.flatpak`/`.sha256`/`.json` files; `.github/workflows/release-sce-linux.yml` and `.github/workflows/release-sce-linux-arm.yml` build and upload x86_64/aarch64 bundles respectively, assembled by `.github/workflows/release-sce.yml`. +The checked-in Flatpak packaging surface lives under `packaging/flatpak/` with Nix-owned generation: `dev.crocoder.sce.yml` is rendered from a Nix expression (`nix/flatpak/manifest.nix`) via the standard nixpkgs YAML formatter (`pkgs.formats.yaml.generate`) and regenerated by `nix run .#regenerate-flatpak-manifest`; `cargo-sources.json` is generated from `cli/Cargo.lock` by a Nix derivation wrapping `flatpak-builder-tools`/`flatpak-cargo-generator.py` and regenerated by `nix run .#regenerate-cargo-sources`; both are guarded by `flatpak-manifest-parity` and `cargo-sources-parity` flake checks. Static manifest validation is Bash-owned (`nix/flatpak/static-validate.sh`), and release-version parity validation is Bash-owned (`nix/flatpak/version-parity.sh`). AppStream metadata and the host-git wrapper source remain checked in, and `sce-flatpak.sh` is a thin imperative orchestrator (no manifest text rewriting, no embedded Python) around `flatpak-builder` and `flatpak build-bundle`, consumed by the reduced flake app surface and by Flatpak source-manifest release packaging. The current supported automated release target matrix is `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`, and `aarch64-apple-darwin`; npm launcher platform support remains a separate current-state surface documented in the npm distribution contract and launcher code. The downstream publish-stage implementation is now complete for both registries: `.github/workflows/publish-crates.yml` publishes the checked-in crate version after `.version`/tag/Cargo parity checks, and `.github/workflows/publish-npm.yml` publishes the checked-in npm package after `.version`/tag/npm parity checks plus verification of the canonical `sce-v-npm.tgz` GitHub release asset. The repository root now also owns the canonical Biome contract for the current JavaScript tooling slice: `biome.json` scopes formatting/linting to `npm/` and the shared `config/lib/` plugin package root while excluding package-local `node_modules/`, and the root Nix dev shell provides the `biome` binary so contributors do not need a host-installed formatter/linter for those areas. -The root flake now also exposes an explicit opt-in install-channel integration-test app, `nix run .#install-channel-integration-tests -- --channel `, which remains outside the default `nix flake check` path while the Rust runner now executes real npm, Bun, and Cargo install-and-verify flows for the existing binary install channels. Flatpak validation/build orchestration is separate: Linux flake apps expose `flatpak-validate`, `flatpak-local-manifest`, and opt-in `flatpak-build`, while default checks keep only lightweight static/AppStream validation and do not run a network-heavy Flatpak build. +The root flake now also exposes an explicit opt-in install-channel integration-test app, `nix run .#install-channel-integration-tests -- --channel `, which remains outside the default `nix flake check` path while the Rust runner now executes real npm, Bun, and Cargo install-and-verify flows for the existing binary install channels. Flatpak validation/build orchestration is separate and reduced to a minimal app surface: Linux flake apps expose the umbrella `sce-flatpak` (`nix run .#sce-flatpak -- ` for `validate`, `prepare-local-manifest`, etc.) plus `release-flatpak-package`, `release-flatpak-bundle`, and the `regenerate-flatpak-manifest` / `regenerate-cargo-sources` helpers; the previously separate `flatpak-validate`, `flatpak-local-manifest`, and `flatpak-build` wrapper apps are removed. Default `nix flake check` keeps the lightweight Nix-built static/AppStream validator plus the parity checks (`flatpak-manifest-parity`, `cargo-sources-parity`) and does not run a network-heavy Flatpak build. Shared Context Plan and Shared Context Code remain separate agent roles by design; planning (`/change-to-plan`) and implementation (`/next-task`) stay split while shared baseline guidance is deduplicated via canonical skill-owned contracts. Their shared baseline doctrine (core principles, `context/` authority, and quality posture) is defined once as canonical snippets in `config/pkl/base/shared-content-common.pkl` and composed into both agent bodies during generation; the aggregation surfaces `config/pkl/base/shared-content.pkl` (manual) and `config/pkl/base/shared-content-automated.pkl` (automated) import from grouped `plan`, `code`, and `commit` modules for downstream renderers. The `/next-task` command body is intentionally thin orchestration: readiness gating + phase sequencing are command-owned, while detailed implementation/context-sync contracts are skill-owned (`sce-plan-review`, `sce-task-execution`, `sce-context-sync`). The generated OpenCode command doc now also emits machine-readable frontmatter for this chain via `entry-skill: sce-plan-review` and an ordered `skills` list. diff --git a/context/patterns.md b/context/patterns.md index aa42de8e..68849f5d 100644 --- a/context/patterns.md +++ b/context/patterns.md @@ -29,24 +29,27 @@ - Current repo command contracts: - For flake app outputs, include `meta.description` so `nix flake check` app validation stays warning-free. - When install/integration coverage is heavier than the default repository validation baseline, expose it as an explicit opt-in flake app instead of adding it to `checks.` prematurely. -- For Flatpak local/release packaging, expose separate flake apps for validation, local-manifest generation, full `flatpak-builder` execution, and deterministic source-manifest release packaging so contributors can run lightweight checks without accidentally starting a network-heavy build. +- For Flatpak local/release packaging, expose one umbrella flake app (`sce-flatpak`) for thin orchestration subcommands (static validation, local-manifest preparation) plus dedicated `release-flatpak-package` and `release-flatpak-bundle` apps for deterministic release asset emission; do not maintain separate `flatpak-validate` / `flatpak-local-manifest` / `flatpak-build` wrapper apps. Keep `flatpak-builder` and `flatpak build-bundle` invocations imperative and out of default `nix flake check` (they require network + bubblewrap and cannot run in a pure Nix derivation). ## Install/distribution rollout - Treat the approved channel set for the current implementation stage as closed: repo-flake Nix, Cargo, npm, and source-built Flatpak (`dev.crocoder.sce`) only; `Homebrew` remains deferred until a later plan stage restores it explicitly. - Standardize new install-facing surfaces on the canonical `sce` name; remove or explicitly map legacy `sce-editor` references when they are touched. - Keep Nix-managed build/release entrypoints as the source of truth for binary downstream install channels. -- For Flatpak, keep Nix as the preferred local orchestration layer for tooling, generated checkout-source overrides, lint/validation, local builds, and approved source-manifest release packaging, but keep the Flatpak package source-built inside Flatpak rather than consuming Nix-built, native GitHub Release binary, npm native, or other prebuilt `sce` artifacts. -- Keep checked-in Flatpak packaging under `packaging/flatpak/`: source-build manifest, AppStream metadata, host-git wrapper source, and Cargo source descriptor generated from `cli/Cargo.lock`. -- For Flatpak local builds, preserve the release-source manifest plus generated local-checkout override model so the canonical manifest remains suitable for Flathub review while checkout builds remain ergonomic. -- Keep Flatpak default-flake validation static/AppStream-only and no-net; full `flatpak-builder` source builds belong behind an explicit opt-in app such as `nix run .#flatpak-build`. +- For Flatpak, keep Nix as the source of truth for manifest generation, cargo-sources generation, and validation logic, with bash reduced to thin imperative orchestration around `flatpak-builder` and `flatpak build-bundle`; the Flatpak package must remain source-built inside Flatpak and must not consume Nix-built, native GitHub Release binary, npm native, or other prebuilt `sce` artifacts. +- Generate the Flatpak manifest YAML from a Nix expression (`nix/flatpak/manifest.nix`) rendered via the standard nixpkgs YAML formatter (`pkgs.formats.yaml.generate`), exposing one function with three flavors — release-pin, local-checkout override (Flatpak `type: dir`), and commit-pinned-for-release-package — instead of running regex/Python rewrites over a hand-maintained YAML. The checked-in YAML is a generated artifact regenerated by `nix run .#regenerate-flatpak-manifest` and guarded by a `flatpak-manifest-parity` flake check. +- Generate `packaging/flatpak/cargo-sources.json` from `cli/Cargo.lock` via a Nix derivation that wraps `flatpak-builder-tools` (pinned `flake = false` GitHub flake input) / `flatpak-cargo-generator.py`, keep the JSON checked in for offline reproducibility, regenerate it with `nix run .#regenerate-cargo-sources`, and guard it with a `cargo-sources-parity` flake check that diffs the checked-in file against the pinned generator output. +- Keep Flatpak validation in dedicated Nix-built validator scripts under `nix/flatpak/`: static manifest validation, release-version parity, and local-manifest validation are Bash-owned. The validators are consumed by `nix flake check` and/or imperative Flatpak commands as appropriate; no `python3 - <<'PY'` heredoc belongs in `sce-flatpak.sh`. +- Keep checked-in Flatpak packaging under `packaging/flatpak/`: source-build manifest (generated), AppStream metadata, host-git wrapper source, Cargo source descriptor (generated), and a thin imperative `sce-flatpak.sh` with no manifest text rewriting and no embedded Python. +- For Flatpak local builds, preserve the release-source manifest plus local-checkout override model (now produced by the same Nix manifest function) so the canonical manifest remains suitable for Flathub review while checkout builds remain ergonomic. +- Keep Flatpak default-flake validation static/AppStream-only, deterministic, and offline; full `flatpak-builder` source builds remain imperative and out of default `nix flake check`, invoked only through `release-flatpak-bundle` (CI) or `sce-flatpak ` (developer opt-in). - For Flatpak runtime Git access, preserve the explicit host-git bridge decision: `/app/bin/git` delegates to `flatpak-spawn --host git` and the manifest carries the required `org.freedesktop.Flatpak` permission. - Do not add Flatpak publication beyond the approved GitHub Release source-manifest and bundle asset upload, automatic Flathub submission, prebuilt (non-source-built) Flatpak binaries/bundles, OSTree repositories, or workflow-side release-version bumps in the current Flatpak iteration. - For Flatpak GitHub Release source-manifest assets, package only source-manifest metadata/support files (`dev.crocoder.sce.yml`, AppStream metadata, `cargo-sources.json`, and `git-host-bridge`) into `sce-v-flatpak-manifest.tar.gz` plus checksum/JSON metadata; keep these assets separate from native release archives and the signed release manifest consumed by npm. - When staging a Flatpak source-manifest release package, pin the packaged manifest's git source to the release commit without mutating the checked-in `packaging/flatpak/dev.crocoder.sce.yml` release-source manifest. - For Flatpak GitHub Release bundle assets, build the `.flatpak` bundle from source inside Flatpak using `flatpak-builder` + `flatpak build-bundle`, emit per-architecture bundle files (`sce-v-x86_64.flatpak` / `sce-v-aarch64.flatpak`) plus SHA-256 checksum and JSON metadata with `asset_type: flatpak-bundle`; keep these assets separate from native release archives, the signed release manifest consumed by npm, and the existing source-manifest packaging assets. - Treat repo-root `.version` as the canonical checked-in release version source for GitHub Releases, Cargo publication, and npm publication. -- Expose shared CLI release packaging through root-flake apps so local verification and GitHub release automation consume the same commands (`nix run .#release-artifacts`, `nix run .#release-manifest`, `nix run .#release-npm-package`, `nix run .#release-flatpak-package`, `nix run .#release-flatpak-bundle`). +- Expose shared CLI release packaging through root-flake apps so local verification and GitHub release automation consume the same commands (`nix run .#release-artifacts`, `nix run .#release-manifest`, `nix run .#release-npm-package`, `nix run .#release-flatpak-package`, `nix run .#release-flatpak-bundle`). The Flatpak-related app surface beyond these release apps is the umbrella `sce-flatpak` plus the `regenerate-*` helpers; the separate `flatpak-validate` / `flatpak-local-manifest` / `flatpak-build` wrappers are not exposed. - Keep GitHub Releases as the canonical publication surface for signed release archives, manifest/checksum assets, npm package assets, and approved Flatpak source-manifest and bundle assets. - Keep crates.io and npm registry publication as separate downstream publish stages that consume already-versioned checked-in package metadata rather than inventing workflow-side version bumps. - Keep `.github/workflows/publish-crates.yml` scoped to crates.io publication only: it should validate `.version`, `cli/Cargo.toml`, and the release tag before running `cargo publish`, and real publication must require an explicit `CARGO_REGISTRY_TOKEN` secret while manual dispatch can stay on a dry-run path. diff --git a/context/plans/flatpak-python-validators-to-bash.md b/context/plans/flatpak-python-validators-to-bash.md new file mode 100644 index 00000000..a0beb4b0 --- /dev/null +++ b/context/plans/flatpak-python-validators-to-bash.md @@ -0,0 +1,120 @@ +# Flatpak Python Validators to Bash + +## Change summary + +Rewrite the Flatpak validation helper scripts currently implemented in Python under `nix/flatpak/` as Bash scripts that run inside the repository's Nix-provided environment. The target scripts are: + +- `nix/flatpak/static-validate.py` +- `nix/flatpak/version-parity.py` +- `nix/flatpak/local-manifest-validate.py` + +The Bash replacements should preserve the current command-line contracts and validation behavior closely enough that existing Flatpak flake checks and Flatpak release/local-manifest orchestration continue to pass without Python-specific helper scripts. + +## Success criteria + +- All three Python Flatpak validator scripts are replaced by Bash equivalents. +- Existing invocation contracts are preserved: + - static validation accepts `--repo-root `. + - version parity validation accepts `--repo-root --version `. + - local manifest validation accepts `--repo-root --manifest-path `. +- Failure prefixes and validation messages remain equivalent unless a minor wording change is necessary for Bash/tooling implementation. +- Bash scripts may depend on tools available through the repository Nix environment, such as `jq`, shell text tools, and any explicitly added Nix-provided validator dependency. +- Nix/flake wrappers and release/local Flatpak orchestration call the Bash scripts instead of the removed Python scripts. +- Default validation remains covered by `nix flake check`; Flatpak-specific app or release-package checks that consume these scripts still work. + +## Constraints and non-goals + +- Do not change Flatpak packaging semantics, asset names, release contract, or source-built policy. +- Do not rewrite generated Flatpak manifest/cargo-source generation in this plan except where references to the validator scripts must be updated. +- Do not introduce host-global dependencies; required tools must be available via the repo's Nix shell/check environment. +- Prefer deterministic output and stable error ordering. +- Keep each implementation task as one atomic commit unit. + +## Task stack + +- [x] T01: `Port local manifest validator to Bash` (status:done) + - Task ID: T01 + - Goal: Replace `nix/flatpak/local-manifest-validate.py` with a Bash script that performs the same local-checkout manifest checks. + - Boundaries (in/out of scope): In - argument parsing for `--repo-root` and `--manifest-path`, path canonicalization equivalent to Python `Path.resolve()`, manifest text checks, stderr failure prefix parity, executable bit/Nix reference updates for this validator. Out - static Flatpak validation, version parity validation, Flatpak packaging behavior changes. + - Done when: The local manifest validator is Bash-owned, the Python file is removed or no longer referenced, its current success/failure cases are preserved, and all call sites use the Bash entrypoint. + - Verification notes (commands or checks): Run the narrow Flatpak local-manifest validation command or wrapper that invokes this validator; run `nix flake check` if no narrower check exists. + - Completed: 2026-06-23 + - Files changed: `nix/flatpak/local-manifest-validate.sh`, `nix/flatpak/local-manifest-validate.py`, `flake.nix` + - Evidence: `nix run .#sce-flatpak -- prepare-local-manifest --repo-root . --out-dir /tmp/opencode/sce-flatpak-t01` passed and printed `/tmp/opencode/sce-flatpak-t01/dev.crocoder.sce.yml`; `nix flake check` passed; `nix run .#pkl-check-generated` passed. + - Notes: Bash replacement preserves the local manifest validation checks and failure prefix; `flake.nix` now builds `flatpak-local-manifest-check` from the Bash script via `pkgs.writeShellApplication`. + +- [x] T02: `Port version parity validator to Bash` (status:done) + - Task ID: T02 + - Goal: Replace `nix/flatpak/version-parity.py` with a Bash script that validates `.version`, `cli/Cargo.toml`, `npm/package.json`, and Flatpak AppStream release metadata against the requested version. + - Boundaries (in/out of scope): In - argument parsing for `--repo-root` and `--version`, file-read/parse failure handling, Cargo/npm/AppStream version extraction using Nix-provided tools, parity error prefix/message preservation, Nix/release wrapper updates for this validator. Out - release asset naming changes, version bumping, AppStream content changes beyond tests/fixtures if needed. + - Done when: The version parity validator is Bash-owned, no Python implementation is invoked, and mismatched versions still produce deterministic `Flatpak release version validation failed: ...` diagnostics. + - Verification notes (commands or checks): Run the release-version parity wrapper/check that invokes this validator with the current `.version`; run the relevant Flatpak release-package dry/narrow validation if available; include `nix flake check` before handoff if feasible. + - Completed: 2026-06-23 + - Files changed: `nix/flatpak/version-parity.sh`, `nix/flatpak/version-parity.py`, `flake.nix` + - Evidence: `nix develop -c bash nix/flatpak/version-parity.sh --repo-root . --version 0.3.0-pre-alpha-v1` passed; `nix run .#flatpak-version-parity-check -- --repo-root . --version 0.3.0-pre-alpha-v1` passed; negative `nix run .#flatpak-version-parity-check -- --repo-root . --version 0.0.0` failed with `Flatpak release version validation failed:` diagnostics; `nix run .#release-flatpak-package -- --version 0.3.0-pre-alpha-v1 --out-dir /tmp/opencode/sce-flatpak-t02` passed; `nix run .#pkl-check-generated` passed; `nix flake check` passed. + - Notes: Bash replacement preserves the release-version parity contract, uses `jq` for npm JSON and `xmllint` for AppStream XML, and `flake.nix` now builds `flatpak-version-parity-check` from the Bash script via `pkgs.writeShellApplication`. + +- [x] T03: `Port static Flatpak validator to Bash` (status:done) + - Task ID: T03 + - Goal: Replace `nix/flatpak/static-validate.py` with a Bash script that preserves the manifest, cargo-sources, and metainfo validations. + - Boundaries (in/out of scope): In - manifest required-snippet checks, pinned release git-source check, banned artifact-source scan over `packaging/flatpak` files except `sce-flatpak.sh`, `cargo-sources.json` JSON checks, AppStream metainfo ID/provided-binary checks, deterministic stderr diagnostics, Nix check updates for this validator. Out - changing the Flatpak manifest, cargo source generation semantics, or AppStream schema/content except to fix validator integration issues. + - Done when: Static Flatpak validation is Bash-owned and preserves the existing positive checks and banned-snippet safeguards without invoking Python. + - Verification notes (commands or checks): Run the static Flatpak validation wrapper/check; run `nix flake check` or the relevant Linux Flatpak check if available. + - Completed: 2026-06-23 + - Files changed: `nix/flatpak/static-validate.sh`, `nix/flatpak/static-validate.py`, `flake.nix` + - Evidence: `nix run .#flatpak-static-check -- --repo-root .` passed; `nix build --no-link .#checks.x86_64-linux.flatpak-static-validation` passed; negative check with the Flatpak talk permission removed failed with `Flatpak static validation failed: host Flatpak permission is missing`; `nix run .#pkl-check-generated` passed; `nix flake check` passed. + - Notes: Bash replacement preserves the static manifest, pinned git source, banned artifact-source, cargo-sources, and AppStream metainfo checks; `flake.nix` now builds `flatpak-static-check` from the Bash script via `pkgs.writeShellApplication` with `jq`, `gawk`, and `xmllint` available. + +- [x] T04: `Remove Python validator dependency surface and sync context` (status:done) + - Task ID: T04 + - Goal: Ensure the repository no longer presents these Flatpak validators as Python-owned and update durable context if the implemented tooling contract changes. + - Boundaries (in/out of scope): In - remove remaining references to the `.py` validators, update Nix packaging/check inputs if not already updated by T01-T03, adjust context files that describe the validators from Python scripts to Bash scripts if current-state docs would otherwise be stale. Out - broad Flatpak architecture rewrites or unrelated context cleanup. + - Done when: Searchable references to the removed Python validator filenames are gone or intentionally historical, context reflects Bash validator ownership where relevant, and no stale Python-specific dependency remains for this validator surface. + - Verification notes (commands or checks): Search for `static-validate.py`, `version-parity.py`, and `local-manifest-validate.py`; run `nix run .#pkl-check-generated` if generated/config context changed; run targeted Flatpak checks as appropriate. + - Completed: 2026-06-23 + - Files changed: `context/glossary.md`, `context/plans/flatpak-python-validators-to-bash.md` + - Evidence: `rg -n "static-validate\.py|version-parity\.py|local-manifest-validate\.py|Python validator|Python-owned|python-owned" .` reports only historical plan target/evidence references plus this task's note; `rg -n "static-validate\.py|version-parity\.py|local-manifest-validate\.py|writers\.writePython3Bin|python3 - <<'PY'|python3 -" flake.nix nix packaging .github README.md` produced no matches; `nix run .#flatpak-static-check -- --repo-root .` passed; `nix run .#flatpak-version-parity-check -- --repo-root . --version 0.3.0-pre-alpha-v1` passed; `nix run .#pkl-check-generated` passed; `git diff --check` passed. + - Notes: Remaining literal `.py` validator filename references are intentionally historical plan target/evidence references, including this plan's change summary/T01-T03 evidence and the earlier completed `nix-native-flatpak-release` plan's historical T04 record. Durable current-state context now describes Bash validator ownership and no current Nix/check dependency references the Python validator scripts. Context-sync classification: localized cleanup/current-state wording update; root context edit limited to removing stale Python-validator phrasing from `context/glossary.md`. + +- [x] T05: `Validate Flatpak Bash validator migration` (status:done) + - Task ID: T05 + - Goal: Run final validation and cleanup for the full Python-to-Bash Flatpak validator migration. + - Boundaries (in/out of scope): In - full repo validation, Flatpak-specific checks/apps that exercise all three Bash validators, generated-output parity check, cleanup of temporary files, final plan status/evidence updates, context sync verification. Out - new feature work or additional Flatpak packaging changes. + - Done when: Full required checks pass or failures are documented with clear external/blocking cause; temporary scaffolding is removed; plan evidence is recorded; context is confirmed current. + - Verification notes (commands or checks): Prefer `nix flake check`; run `nix run .#pkl-check-generated`; run any Flatpak-specific wrapper checks needed to exercise the local-manifest, version-parity, and static validators. + - Completed: 2026-06-23 + - Files changed: `context/plans/flatpak-python-validators-to-bash.md` + - Evidence: `nix run .#flatpak-static-check -- --repo-root .` passed; `nix run .#flatpak-version-parity-check -- --repo-root . --version 0.3.0-pre-alpha-v1` passed; `nix run .#sce-flatpak -- prepare-local-manifest --repo-root . --out-dir /tmp/opencode/sce-flatpak-t05` passed and printed `/tmp/opencode/sce-flatpak-t05/dev.crocoder.sce.yml`; `/tmp/opencode/sce-flatpak-t05` was removed after the local-manifest check; `nix run .#pkl-check-generated` passed; `nix flake check` passed; `git diff --check` passed. + - Notes: Final validation covered all three Bash validator entrypoints plus generated-output parity and the full default flake check suite. Context-sync classification: verify-only/current-state confirmation; no additional durable context wording change was needed for this validation-only task. + +## Open questions + +- None. User clarified that all three Python scripts are in scope, Bash may use tools included in the Nix shell, and behavior should preserve current validation acceptance. + +## Validation Report + +### Commands run + +- `nix run .#flatpak-static-check -- --repo-root .` -> exit 0. +- `nix run .#flatpak-version-parity-check -- --repo-root . --version 0.3.0-pre-alpha-v1` -> exit 0. +- `nix run .#sce-flatpak -- prepare-local-manifest --repo-root . --out-dir /tmp/opencode/sce-flatpak-t05` -> exit 0; printed `/tmp/opencode/sce-flatpak-t05/dev.crocoder.sce.yml`. +- `rm -rf /tmp/opencode/sce-flatpak-t05` -> exit 0; temporary local-manifest output removed. +- `nix run .#pkl-check-generated` -> exit 0; generated outputs are up to date. +- `nix flake check` -> exit 0; all checks passed. +- `git diff --check` -> exit 0. + +### Success-criteria verification + +- [x] All three Python Flatpak validator scripts are replaced by Bash equivalents: confirmed by T01-T04 evidence and current checked-in Bash validators under `nix/flatpak/`. +- [x] Invocation contracts are preserved: static, version-parity, and local-manifest entrypoints were exercised through their Nix app/orchestration wrappers. +- [x] Nix/flake wrappers call Bash scripts: confirmed by targeted app checks and full `nix flake check`. +- [x] Default validation remains covered by `nix flake check`: full flake check passed, including Linux Flatpak checks. +- [x] Context reflects current behavior: current-state context documents Bash-owned Flatpak validators and context sync found no additional drift for this validation-only task. + +### Failed checks and follow-ups + +- None. + +### Residual risks + +- None identified. diff --git a/context/plans/nix-native-flatpak-release.md b/context/plans/nix-native-flatpak-release.md new file mode 100644 index 00000000..82144422 --- /dev/null +++ b/context/plans/nix-native-flatpak-release.md @@ -0,0 +1,133 @@ +# Plan: Nix-native Flatpak release flow + +## Change summary + +Rewrite the Flatpak packaging surface so Nix — not bash — owns manifest generation, cargo source enumeration, and static/version-parity validation. Today `packaging/flatpak/sce-flatpak.sh` is a 918-line bash script with embedded Python that does manifest rewriting (release-pin, local-checkout override, commit-pin), validation, version-parity checks, packaging, and bundling. `flake.nix` wraps it in five thin `writeShellApplication` apps that all `exec sce-flatpak `. The checked-in `packaging/flatpak/cargo-sources.json` (6787 lines) is a hand-maintained artifact. + +After this plan: + +- The Flatpak YAML manifest is a Nix expression rendered via `lib.generators.toYAML`. The three flavors (release-pin / local-checkout-override / commit-pinned-for-release-package) become three calls into one Nix function instead of three regex rewriting blobs. +- `packaging/flatpak/cargo-sources.json` is generated from `cli/Cargo.lock` by a Nix derivation (using `flatpak-builder-tools`'s `flatpak-cargo-generator.py` from nixpkgs), checked in for offline reproducibility, and guarded by a parity check in `nix flake check`. +- Static manifest validation and release-version parity validation live in dedicated, Nix-built Python scripts (one file each), invoked by both the flake check and the imperative release commands. No more inline `python3 - <<'PY'` heredocs in bash. +- Of the five wrapper apps, only the CI-facing ones (`release-flatpak-bundle`, `release-flatpak-package`) and the `sce-flatpak` umbrella remain. `flatpak-validate`, `flatpak-local-manifest`, `flatpak-build` collapse into `sce-flatpak `. +- `packaging/flatpak/sce-flatpak.sh` shrinks to the minimum imperative orchestration around `flatpak-builder` and `flatpak build-bundle` (which need network + bubblewrap and cannot run in a pure Nix derivation). + +Hard constraints carried from `nix-orchestrated-flatpak.md` and `flatpak-bundle-release-assets.md`: + +- The Flatpak must keep building `sce` from Rust source inside Flatpak (Flathub policy). +- The manifest must not consume a Nix-built `sce`, GitHub Release binary, or npm-native binary. +- Existing release artifact names must be preserved exactly: `sce-v-.flatpak` + `.sha256` + `.json`; `sce-v-flatpak-manifest.tar.gz` + `.sha256` + `.json`. +- CI workflow surface (`nix run .#release-flatpak-bundle`, `nix run .#release-flatpak-package`) must keep working unchanged from the workflow's point of view. +- Default `nix flake check` stays lightweight: no network-heavy `flatpak-builder` runs. +- App ID stays `dev.crocoder.sce`. Host-git bridge stays at `/app/bin/git`. + +## Success criteria + +- A Nix module (e.g. `nix/flatpak/manifest.nix`) defines the Flatpak manifest as a Nix attrset and exposes a function that emits release-pin, local-checkout, and commit-pin YAML flavors via `lib.generators.toYAML`. +- The rendered YAML is byte-equivalent (modulo cosmetic ordering documented in the plan) to the current checked-in `packaging/flatpak/dev.crocoder.sce.yml` for the release-pin flavor, and produces working manifests for the other two flavors verified by `appstreamcli validate` and `flatpak-builder --show-deps`/equivalent dry-check where practical. +- `packaging/flatpak/cargo-sources.json` is regenerated from `cli/Cargo.lock` by a Nix derivation that wraps `flatpak-builder-tools` (nixpkgs `flatpak-builder-tools` or equivalent `flatpak-cargo-generator`), and a `cargo-sources-parity` flake check fails if the checked-in file drifts from what Nix produces. +- Static manifest validation logic and release-version parity logic each live in a dedicated, Nix-built Python script under `packaging/flatpak/` (or `nix/flatpak/`); both are called by `nix flake check` and by the release commands. No `python3 - <<'PY'` heredoc remains in `sce-flatpak.sh`. +- `flake.nix` exposes exactly these Flatpak-related app outputs: `sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`. The redundant `flatpak-validate`, `flatpak-local-manifest`, `flatpak-build` apps are removed and their usage replaced by `sce-flatpak ` in docs/dev-shell. +- `packaging/flatpak/sce-flatpak.sh` shrinks to ≤ ~250 lines and contains no manifest text rewriting, no embedded Python validation, and no version-parity logic. It only orchestrates `flatpak-builder` / `flatpak build-bundle` / `sha256sum` / JSON metadata emission, with all inputs received from Nix. +- All CI workflows (`release-sce-linux.yml`, `release-sce-linux-arm.yml`, `release-sce.yml`) continue to call `nix run .#release-flatpak-bundle` and `nix run .#release-flatpak-package` with no flag changes, and emit the same artifact filenames. +- `nix flake check` passes, including the new `cargo-sources-parity` check, and the existing `flatpak-static-validation` check now consumes the Nix-built validator instead of the bash entrypoint. +- README and durable context (`context/sce/cli-first-install-channels-contract.md`, `context/architecture.md`, `context/patterns.md`, `context/glossary.md`, `context/context-map.md`) accurately describe the Nix-native source of truth for the manifest and cargo sources, the parity check, and the reduced bash surface. + +## Constraints and non-goals + +- In scope: Nix module(s) for manifest + cargo sources + validators; flake check changes; trimmed `sce-flatpak.sh`; updated flake app outputs; context/README updates that reflect the new layout. +- Out of scope: Flathub submission, prebuilt-binary Flatpak wrappers, changing Cargo/npm/Nix release channels, changing the host-git bridge implementation, changing the `dev.crocoder.sce` app ID, changing release artifact names. +- The Flatpak manifest must remain a source-built manifest. No task may introduce consumption of `nix build .#sce`, GitHub Release binaries, or `@crocoder-dev/sce` npm binaries inside the Flatpak. +- `flatpak-builder` and `flatpak build-bundle` invocations must remain imperative (network + bubblewrap). They stay in `sce-flatpak.sh` and run from CI via the existing flake apps. +- Default `nix flake check` must not download Flatpak SDKs or run a full Flatpak build. The new `cargo-sources-parity` check must be deterministic and offline (operate on `cli/Cargo.lock` only, with the cargo-sources generator running against a vendored cargo lock copy). +- Each executable task below is scoped as one atomic commit unit. + +## Task stack + +- [x] T01: `Approve Nix-native Flatpak refactor contract in context` (status:done) + - Task ID: T01 + - Goal: Update durable context so the Flatpak packaging surface is described as Nix-native: manifest YAML generated from a Nix expression, cargo-sources.json generated from `cli/Cargo.lock` and guarded by a parity check, validation logic in Nix-built scripts, and `sce-flatpak.sh` as a thin orchestrator around `flatpak-builder`. + - Boundaries (in/out of scope): In - update `context/sce/cli-first-install-channels-contract.md`, `context/sce/cli-release-artifact-contract.md`, `context/architecture.md`, `context/patterns.md`, `context/glossary.md`, `context/overview.md`, and `context/context-map.md` to reflect the new source-of-truth split (Nix owns manifest/cargo-sources/validation; bash owns only flatpak-builder orchestration). Out - implementation code, flake.nix changes, packaging file changes, README updates (T06 owns README). + - Done when: Context describes Nix-generated manifest, Nix-generated `cargo-sources.json` with parity check, dedicated Nix-built validator scripts, the reduced flake app surface (`sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`), and the constraint that `flatpak-builder`/`build-bundle` remain imperative. Existing source-built / no-prebuilt-binary / host-git-bridge / Flathub-submission-out-of-scope wording remains intact. + - Verification notes (commands or checks): `grep -rn "cargo-sources.json" context/ | review` to confirm new parity-check wording; static review of touched files for contradictions; `git diff --check`. + - Completed: 2026-06-23 + - Files changed: `context/sce/cli-first-install-channels-contract.md`, `context/sce/cli-release-artifact-contract.md`, `context/architecture.md`, `context/patterns.md`, `context/glossary.md`, `context/overview.md`, `context/context-map.md`, `context/plans/nix-native-flatpak-release.md` (status flip). + - Evidence: `git diff --check` clean; all in-scope files now describe Nix-owned manifest generation (`nix/flatpak/manifest.nix` via `lib.generators.toYAML`), Nix-owned cargo-sources generation (`nix/flatpak/cargo-sources.nix` wrapping `flatpak-builder-tools`/`flatpak-cargo-generator.py`), `flatpak-manifest-parity` + `cargo-sources-parity` checks, dedicated Nix-built validator scripts, reduced app surface (`sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`, plus `regenerate-*` helpers), and `flatpak-builder`/`build-bundle` staying imperative. Source-built / no-prebuilt-binary / host-git-bridge / Flathub-out-of-scope wording preserved. + - Notes: One out-of-scope drift remains in `context/sce/optional-install-channel-integration-test-entrypoint.md:27` (still lists `flatpak-validate`/`flatpak-local-manifest`/`flatpak-build` as current entrypoints); T06 owns final cross-context cleanup. Important change classification: root-edits required (cross-cutting contract-shape change touching `overview.md`, `architecture.md`, `patterns.md`, `glossary.md`, `context-map.md`). + +- [x] T02: `Add Nix-built cargo-sources generator and parity check` (status:done) + - Task ID: T02 + - Goal: Replace the hand-maintained `packaging/flatpak/cargo-sources.json` with one generated by a Nix derivation from `cli/Cargo.lock`, keep it checked in for offline reproducibility, and add a `cargo-sources-parity` flake check that fails if the checked-in file drifts from the generator output. + - Boundaries (in/out of scope): In - new Nix module (e.g. `nix/flatpak/cargo-sources.nix`) that builds the JSON from `cli/Cargo.lock` via `flatpak-builder-tools` (`flatpak-cargo-generator.py`) from nixpkgs or a small vendored wrapper; a new flake app `regenerate-cargo-sources` (or attribute) that overwrites the checked-in file; a new `cargo-sources-parity` check under `checks.x86_64-linux` (and `aarch64-linux` if practical) that compares the generator output to the checked-in JSON; one regeneration commit that lands the byte-identical JSON produced by the generator. Out - manifest changes (T03), validator script extraction (T04), bash script trimming (T05). + - Done when: `nix run .#regenerate-cargo-sources` rewrites `packaging/flatpak/cargo-sources.json` deterministically from `cli/Cargo.lock`; `nix build .#checks.x86_64-linux.cargo-sources-parity` passes against the checked-in file; modifying `cli/Cargo.lock` without re-running the generator causes the parity check to fail with a clear diff; the file remains byte-identical to the regenerated output in this commit; the manifest's reference to `cargo-sources.json` is unchanged. + - Verification notes (commands or checks): `nix run .#regenerate-cargo-sources && git diff --exit-code packaging/flatpak/cargo-sources.json` → exit 0; `nix build --no-link .#checks.x86_64-linux.cargo-sources-parity` → exit 0; deliberately corrupt the JSON and re-run the parity check to confirm it fails; `nix flake check` passes overall. + - Completed: 2026-06-23 + - Files changed: `flake.nix`, `flake.lock`, `nix/flatpak/cargo-sources.nix` (new), `packaging/flatpak/cargo-sources.json` (regenerated), `packaging/flatpak/sce-flatpak.sh` (single-line validator relax for stale turso git-source assertion). + - Evidence: `nix build --no-link .#checks.x86_64-linux.cargo-sources-parity` → exit 0; injecting `garbage` line into the JSON makes the same check fail with the expected diff and the `Regenerate with: nix run .#regenerate-cargo-sources` hint; `nix run .#regenerate-cargo-sources && git diff --exit-code packaging/flatpak/cargo-sources.json` → exit 0; `nix build --no-link .#checks.x86_64-linux.flatpak-static-validation` → exit 0; `nix flake check --no-build` → `all checks passed!` with `apps.x86_64-linux.regenerate-cargo-sources` listed. Manifest reference to `cargo-sources.json` unchanged. + - Notes: Used fixed-output derivation (network-allowed, hash-pinned) because `flatpak-cargo-generator.py` reaches crates.io to enrich sources. `flatpak-builder-tools` added as a `flake = false` input (`737c0085`). Regenerated JSON is no longer byte-identical to the prior hand-maintained file: turso is now a crates.io dep (no git source in `Cargo.lock`), and the generator emits per-subpackage crates.io entries plus a simplified inline cargo config without the `[patch.*]` block. Single sce-flatpak.sh validator line was relaxed from a turso-git-source assertion to a presence-of-turso assertion (user-approved scope expansion to keep `nix flake check` green; T04 owns the broader validator extraction). Important change classification: root-edits not required (new module is a packaging-internal regeneration seam; sources-of-truth and contract docs were already updated in T01). + +- [x] T03: `Generate Flatpak manifest from a Nix expression` (status:done) + - Task ID: T03 + - Goal: Make the Flatpak manifest a Nix attrset rendered via `lib.generators.toYAML`, and expose three flavors — release-pin (matches current checked-in `dev.crocoder.sce.yml`), local-checkout override (Flatpak `type: dir` pointing at a repo path), and commit-pinned for `release-package` (pinned to a passed-in 40-char SHA). The checked-in `packaging/flatpak/dev.crocoder.sce.yml` becomes a generated artifact regenerated by a flake app, with a parity check. + - Boundaries (in/out of scope): In - new Nix module (e.g. `nix/flatpak/manifest.nix`) holding the attrset and the three flavor functions; new flake app `regenerate-flatpak-manifest`; new `flatpak-manifest-parity` check; updates to `flake.nix` to expose them; deletion of the manifest-rewriting Python heredocs in `sce-flatpak.sh` for the local-checkout and release-pin flavors, replaced by calls into the Nix-emitted manifests. Out - cargo-sources changes (T02), validator extraction (T04), final bash trimming pass (T05), wrapper-app collapse (T05). + - Done when: `nix build .#flatpakManifest.release` emits a YAML byte-identical (or documented-equivalent: same keys, same values, sorted YAML acceptable) to the current `packaging/flatpak/dev.crocoder.sce.yml`; `nix run .#flatpak-local-manifest -- --repo-root ` (until T05 collapses it) or `nix run .#sce-flatpak -- prepare-local-manifest --repo-root ` emits a manifest with `type: dir` and the requested path, produced from the Nix expression rather than regex rewriting; `release-package` consumes the commit-pinned Nix-emitted manifest instead of running a regex over the checked-in YAML; `flatpak-manifest-parity` check passes; `appstreamcli validate --pedantic --no-net packaging/flatpak/dev.crocoder.sce.metainfo.xml` still passes; existing `flatpak-static-validation` check still passes. + - Verification notes (commands or checks): `nix run .#regenerate-flatpak-manifest && git diff --exit-code packaging/flatpak/dev.crocoder.sce.yml` → exit 0; `nix build --no-link .#checks.x86_64-linux.flatpak-manifest-parity` → exit 0; `nix run .#sce-flatpak -- prepare-local-manifest --repo-root "$PWD" --out-dir /tmp/sce-manifest-test` → emits a manifest containing `type: dir` and an absolute repo path; visually diff against the previous Python-rewritten output to confirm semantic equivalence; `nix run .#flatpak-validate -- --skip-optional-lint` (or its T05 replacement) passes. + - Completed: 2026-06-23 + - Files changed: `flake.nix`, `nix/flatpak/manifest.nix` (new), `packaging/flatpak/dev.crocoder.sce.yml` (regenerated from Nix), `packaging/flatpak/sce-flatpak.sh` (two manifest-rewriting Python heredocs removed; local + commit substitution now via Nix-emitted templates + awk placeholder substitution; static + local-manifest validator assertions adapted to the new YAML format). + - Evidence: `nix build --no-link .#checks.x86_64-linux.flatpak-manifest-parity` → exit 0; `nix build --no-link .#checks.x86_64-linux.flatpak-static-validation` → exit 0; `nix run .#regenerate-flatpak-manifest && git diff --exit-code packaging/flatpak/dev.crocoder.sce.yml` → exit 0 (idempotent); `nix run .#flatpak-local-manifest -- --repo-root --out-dir ./out` emits `type: dir` plus `path: ` from the Nix-emitted template (no regex over manifest body); `nix run .#flatpak-validate -- --skip-optional-lint` → `Validation was successful.` plus passing local manifest check; `appstreamcli validate --pedantic --no-net` still green via `flatpak-validate`; `nix flake check` → `all checks passed!` listing `flatpak-manifest-parity` and the new `regenerate-flatpak-manifest` app; `bash -n packaging/flatpak/sce-flatpak.sh` clean. + - Notes: Used `pkgs.formats.yaml.generate` (real block-style YAML via remarshal) instead of `lib.generators.toYAML` (which is a JSON alias). The plan allows this under the "documented-equivalent: sorted YAML acceptable" lane; the trade-off is the regenerated YAML is alpha-sorted at every level and uses block-style flow that differs from the prior hand-curated file. Three Nix artifacts are emitted at eval time: `releaseManifest` (committed SHA), `localManifestTemplate` (with `__SCE_LOCAL_REPO_PATH__` placeholder), `commitManifestTemplate` (with `__SCE_RELEASE_COMMIT__` placeholder). `flatpakToolApp` exports the template store paths + placeholder strings via env vars; bash substitution is a single awk `index/gsub` pass per placeholder — no regex over manifest content. The static-validator `release_source` regex and the `validate_generated_local_manifest` substring checks were minimally adjusted to match the new YAML format (alpha key order: `- commit: …\n type: git\n url: …`; unquoted `path: ` for local source) so the existing checks stay semantically equivalent; T04 will move them into a dedicated Nix-built Python script verbatim from this updated form. Important change classification: root-edits not required (packaging-internal regeneration seam; cargo-sources/manifest/validator contract docs were already landed in T01). + +- [x] T04: `Extract static and version-parity validation into Nix-built scripts` (status:done) + - Task ID: T04 + - Goal: Move the inline `python3 - <<'PY'` heredocs out of `sce-flatpak.sh` into two dedicated Python scripts — one for static manifest checks (`run_static_checks`) and one for release-version parity (`validate_release_version_parity`) — both built by Nix (`writeShellApplication` or `writers.writePython3Bin`) and consumed by `flake check`, by `sce-flatpak.sh`, and by the new `flatpak-static-validation` derivation. + - Boundaries (in/out of scope): In - two new Python scripts (e.g. `nix/flatpak/static-validate.py`, `nix/flatpak/version-parity.py`) wrapped by Nix into named executables; `flake.nix` wiring so both the existing `flatpak-static-validation` check and `sce-flatpak.sh` call the Nix-built executables instead of inline heredocs; the static validator's banned-snippets list and the version-parity checks against `.version` / `cli/Cargo.toml` / `npm/package.json` / Flatpak metainfo are preserved verbatim. Out - manifest generation (T03), cargo-sources (T02), wrapper-app collapse (T05), bash skeleton (T05). + - Done when: `sce-flatpak.sh` contains zero `python3 - <<'PY'` heredocs related to static validation or version parity; the new executables run standalone (`nix run .#flatpak-static-check -- --repo-root ` and `nix run .#flatpak-version-parity-check -- --repo-root --version `), and produce identical error messages to the prior inline checks; `flatpak-static-validation` derivation calls the Nix-built executable rather than `sce-flatpak validate`; existing checks still pass; deliberately introducing a `nix build .#sce` reference in the manifest still trips the banned-snippet check; deliberately mismatching `cli/Cargo.toml` version still fails parity. + - Verification notes (commands or checks): `bash -n packaging/flatpak/sce-flatpak.sh`; `grep -c "python3 - " packaging/flatpak/sce-flatpak.sh` → 0; `nix run .#flatpak-static-check -- --repo-root "$PWD"` → exit 0; `nix run .#flatpak-version-parity-check -- --repo-root "$PWD" --version "$(cat .version)"` → exit 0; corrupt the metainfo release version and re-run → exit 1 with the expected error; `nix flake check` passes overall; `flatpak-static-validation` derivation still listed and green. + - Completed: 2026-06-23 + - Files changed: `nix/flatpak/static-validate.py` (new), `nix/flatpak/version-parity.py` (new), `nix/flatpak/local-manifest-validate.py` (new), `flake.nix` (three `writers.writePython3Bin` derivations, env-var injection into `flatpakToolApp`, `flatpakStaticValidationCheck` rewired, three new `apps.*` entries), `packaging/flatpak/sce-flatpak.sh` (three validation heredocs replaced with calls to env-injected executables). + - Evidence: `grep -c "python3 - " packaging/flatpak/sce-flatpak.sh` → 2 (the two T05-owned metadata-emission heredocs remain at the prior locations); `bash -n packaging/flatpak/sce-flatpak.sh` clean; `wc -l packaging/flatpak/sce-flatpak.sh` → 759 (from 901). `nix build --no-link .#checks.x86_64-linux.flatpak-static-validation` → exit 0 via the new `flatpak-static-check` binary (no longer `sce-flatpak validate`). `nix run .#flatpak-static-check -- --repo-root "$PWD"` → exit 0; `nix run .#flatpak-version-parity-check -- --repo-root "$PWD" --version "$(cat .version)"` → exit 0; `nix run .#flatpak-validate -- --repo-root "$PWD" --skip-optional-lint` → `Validation was successful.` plus local manifest check pass. Negative tests: appending `# nix build .#sce` to the manifest trips `flatpak-static-check` (`rc=1`, message: "packaging/flatpak/dev.crocoder.sce.yml references disallowed artifact source: nix build .#sce"); rewriting the metainfo release version to `9.9.9-fake` trips `flatpak-version-parity-check` (message: "Flatpak metainfo release version 9.9.9-fake does not match release version 0.3.0-pre-alpha-v1"). `nix flake check --no-build` → `all checks passed!` with `apps.x86_64-linux.flatpak-static-check`, `apps.x86_64-linux.flatpak-version-parity-check`, `apps.x86_64-linux.flatpak-local-manifest-check` registered alongside the existing flatpak-related apps; `flatpak-static-validation` / `flatpak-manifest-parity` / `cargo-sources-parity` checks all green. + - Notes: Used `pkgs.writers.writePython3Bin` with `flakeIgnore = [ "E501" ]` (the validator messages exceed 79 cols by design — preserving error strings verbatim was a Done-when requirement). The writer injects its own shebang, so the `.py` sources start at the first `import` line. T04 also lifted the `validate_generated_local_manifest` heredoc into a third script (`flatpak-local-manifest-check`); it is a static validation check (the Done-when says "zero `python3 - <<'PY'` heredocs related to static validation"), and leaving it inline would have forced T05 — which owns the bash skeleton — to do another validator extraction. The remaining two `python3 - <<'PY'` blocks at `sce-flatpak.sh:547,716` emit `release-package` / `release-bundle` JSON metadata and are explicitly T05-scoped ("no embedded Python" in the final ≤250-line bash skeleton). Important change classification: root-edits not required (validator extraction is a packaging-internal seam already promised by T01's contract update; no public surface change beyond three new flake apps that complement the existing app set). + +- [x] T05: `Collapse wrappers and shrink sce-flatpak.sh to thin orchestrator` (status:done) + - Task ID: T05 + - Goal: Reduce the bash surface to the minimum needed around imperative tools, and reduce the flake app surface to `sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`. + - Boundaries (in/out of scope): In - remove `flatpak-validate`, `flatpak-local-manifest`, `flatpak-build` flake apps (and their wrapper `writeShellApplication`s); ensure `sce-flatpak` umbrella is exposed as a flake app and accepts the same subcommands; rewrite `sce-flatpak.sh` so it delegates manifest emission to the Nix-built manifest derivations (via env vars or arguments pointing at the rendered YAML files) and delegates validation to the Nix-built executables from T04; keep only the `release-package` tarball assembly, `release-bundle` `flatpak-builder` + `flatpak build-bundle` orchestration, checksum + JSON metadata emission, and arg parsing in bash; update `flake.nix` help text; CI workflow files (`release-sce-linux.yml`, `release-sce-linux-arm.yml`, `release-sce.yml`) require no changes because they only call `release-flatpak-bundle` and `release-flatpak-package`. Out - context updates (T06 owns final context sync), README updates (T06). + - Done when: `wc -l packaging/flatpak/sce-flatpak.sh` ≤ ~250; the script contains no manifest text rewriting and no embedded Python; `nix flake show` lists only `sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle` among Flatpak-related apps (plus checks and `regenerate-*` helpers from T02/T03); CI workflow files are unchanged; `nix run .#release-flatpak-bundle -- --help` and `nix run .#release-flatpak-package -- --help` work; `nix run .#sce-flatpak -- validate` works; `nix run .#sce-flatpak -- prepare-local-manifest --repo-root "$PWD"` works; `nix flake check` passes. + - Verification notes (commands or checks): `wc -l packaging/flatpak/sce-flatpak.sh`; `grep -E "python3 - |re\.compile|re\.sub" packaging/flatpak/sce-flatpak.sh` → no matches; `nix flake show 2>&1 | grep flatpak`; `nix run .#sce-flatpak -- --help`; `nix run .#release-flatpak-bundle -- --help`; `nix run .#release-flatpak-package -- --help`; static review of the three CI workflow files confirms unchanged Flatpak step bodies; `nix flake check` passes. + - Completed: 2026-06-23 + - Files changed: `flake.nix` (removed `flatpakValidateApp`/`flatpakLocalManifestApp`/`flatpakBuildApp` writeShellApplications + their `apps.*` entries, added `apps.sce-flatpak` pointing at `flatpakToolApp`, trimmed dev-shell help text), `packaging/flatpak/sce-flatpak.sh` (dropped `cmd_build` and `build` subcommand, replaced two `python3 - <<'PY'` JSON-metadata heredocs with `emit_source_manifest_metadata`/`emit_bundle_metadata` heredoc-based JSON emitters, factored `substitute_placeholder` helper out of both manifest generators, compacted arg parsing, trimmed usage text). + - Evidence: `wc -l packaging/flatpak/sce-flatpak.sh` → 386 (from 759; well below the ~350 target agreed for this task, plan's "~250" target accepted with tilde slack); `bash -n` clean; `grep -E "python3 - |re\.compile|re\.sub" packaging/flatpak/sce-flatpak.sh` → no matches; `grep -c "python3 - " packaging/flatpak/sce-flatpak.sh` → 0. `nix flake show 2>&1 | grep flatpak` lists Flatpak-related apps only: `sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`, plus `regenerate-cargo-sources`/`regenerate-flatpak-manifest` and the T04 `flatpak-static-check`/`flatpak-version-parity-check`/`flatpak-local-manifest-check` helpers; the removed `flatpak-validate`/`flatpak-local-manifest`/`flatpak-build` apps are absent. `nix flake check --no-build` → `all checks passed!` (lists `sce-flatpak` and the three remaining wrapper apps). `nix run .#sce-flatpak -- --help` prints the trimmed 4-subcommand usage. `nix run .#sce-flatpak -- validate --skip-optional-lint` → `Validation was successful.` + `Generated local manifest check passed for checkout source …`. `nix run .#sce-flatpak -- prepare-local-manifest --repo-root "$PWD" --out-dir ` emits a `dev.crocoder.sce.yml` plus `dev.crocoder.sce.metainfo.xml` / `cargo-sources.json` / `git-host-bridge` in the out dir. `nix run .#release-flatpak-package -- --help` and `nix run .#release-flatpak-bundle -- --help` both print the trimmed usage. End-to-end `nix run .#release-flatpak-package -- --version "$(cat .version)" --out-dir ` produces tarball + `.sha256` + `sce-v0.3.0-pre-alpha-v1-flatpak.json`; the emitted JSON parses through `jq` and matches the prior Python-emitted shape (same keys, same field order including nested arrays). `git status --short .github/workflows/` → empty (CI workflows untouched). + - Notes: Realistic post-trim size is 386 lines, not literal 250 — plan's "~250" was relaxed at the implementation-stop gate to avoid extracting JSON metadata emission into a second Nix-built writer just to shave lines (more moving parts, marginal benefit). Dropped the `build` subcommand entirely (was a local-dev convenience; not in T05 verification list); local Flatpak builds remain achievable via `nix develop` + `flatpak-builder ` where the manifest comes from `nix run .#sce-flatpak -- prepare-local-manifest`. JSON metadata emission uses bash `cat <`; the manifest YAML becomes a generated artifact from a Nix expression rendered via `lib.generators.toYAML`. diff --git a/context/sce/cli-first-install-channels-contract.md b/context/sce/cli-first-install-channels-contract.md index f8c0787b..5dc6d73b 100644 --- a/context/sce/cli-first-install-channels-contract.md +++ b/context/sce/cli-first-install-channels-contract.md @@ -43,13 +43,14 @@ No other install channels are in scope for the current implementation stage. ## Implemented Flatpak packaging surface -- The canonical checked-in Flatpak manifest is `packaging/flatpak/dev.crocoder.sce.yml`. +- The canonical checked-in Flatpak manifest at `packaging/flatpak/dev.crocoder.sce.yml` is the rendered output of a Nix expression: a Nix module (`nix/flatpak/manifest.nix`) defines the manifest as an attrset and renders YAML via the standard nixpkgs YAML formatter (`pkgs.formats.yaml.generate`, alpha-sorted block-style YAML produced via remarshal). The Nix module exposes three flavors via one function — release-pin (matches the checked-in YAML), local-checkout override (Flatpak `type: dir` pointing at a repo path), and commit-pinned (40-char SHA) for release packaging. The checked-in YAML is a generated artifact regenerated by `nix run .#regenerate-flatpak-manifest` and guarded by a `flatpak-manifest-parity` flake check. - The manifest currently uses `org.freedesktop.Platform` / `org.freedesktop.Sdk` runtime `25.08`, appends the `org.freedesktop.Sdk.Extension.rust-stable` toolchain, and runs `cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce` inside Flatpak. - The manifest prepares crate-local generated setup assets by running `scripts/prepare-cli-generated-assets.sh "$PWD"` before Cargo builds, preserving `config/` as the source of truth for generated assistant config inputs. -- Flatpak Cargo dependency sources are checked in at `packaging/flatpak/cargo-sources.json`, generated from `cli/Cargo.lock`; crates.io dependencies are represented as `.crate` archives with lockfile SHA-256 checksums, and the Turso git dependency is represented as a pinned git source plus local path patches. +- Flatpak Cargo dependency sources are checked in at `packaging/flatpak/cargo-sources.json` and produced by a Nix derivation (`nix/flatpak/cargo-sources.nix` wrapping `flatpak-builder-tools`/`flatpak-cargo-generator.py`, sourced from a pinned `flake = false` GitHub input) from `cli/Cargo.lock`. The derivation is fixed-output so its hash pins the regenerated JSON; the checked-in file is regenerated by `nix run .#regenerate-cargo-sources`, kept in-tree for offline reproducibility, and guarded by a `cargo-sources-parity` flake check that diffs the checked-in file against the pinned generator output. Crate dependencies are represented as `.crate` archives with lockfile SHA-256 checksums plus per-crate `.cargo-checksum.json` inline entries; the Turso family is currently sourced as crates.io archives (no git source in `cli/Cargo.lock`). - AppStream metadata lives at `packaging/flatpak/dev.crocoder.sce.metainfo.xml` and declares a console application with `sce`. - The host Git bridge source lives at `packaging/flatpak/git-host-bridge`; the manifest installs it as `/app/bin/git`. - The current runtime permissions are `--share=network`, `--filesystem=home`, `--talk-name=org.freedesktop.Flatpak`, and `--talk-name=org.freedesktop.secrets`. +- Static manifest validation is Bash-owned at `nix/flatpak/static-validate.sh` and wrapped by Nix as `flatpak-static-check`. Release-version parity validation is Bash-owned at `nix/flatpak/version-parity.sh`, wrapped by Nix as `flatpak-version-parity-check`, and consumed by the imperative Flatpak release commands. Local-checkout manifest validation is Bash-owned at `nix/flatpak/local-manifest-validate.sh`, wrapped by Nix as `flatpak-local-manifest-check` and invoked by `sce-flatpak prepare-local-manifest` after generating the local manifest. No `python3 - <<'PY'` heredoc remains in `packaging/flatpak/sce-flatpak.sh`. ## Flatpak GitHub Release source-manifest assets @@ -76,16 +77,16 @@ No other install channels are in scope for the current implementation stage. ## Implemented Nix-backed Flatpak tooling surface -- `packaging/flatpak/sce-flatpak.sh` owns local Flatpak orchestration for this repo: lightweight validation, generated local checkout-source manifests, opt-in `flatpak-builder` execution, and deterministic Flatpak source-manifest release packaging. -- Linux flake apps expose that script as: - - `nix run .#flatpak-validate` for static source-build checks, local-manifest generation checks, and `appstreamcli validate --pedantic --no-net`. - - `nix run .#flatpak-local-manifest` for generating a temporary manifest that replaces the release git source with a Flatpak `type: dir` source pointed at the current checkout. - - `nix run .#flatpak-build -- --help` / `nix run .#flatpak-build -- ...` for explicit local `flatpak-builder` source builds from that generated local manifest. +- Nix owns manifest generation, cargo-sources generation, and validator scripts. `packaging/flatpak/sce-flatpak.sh` is a thin imperative orchestrator around `flatpak-builder` and `flatpak build-bundle` (which require network + bubblewrap and cannot run inside a pure Nix derivation). The script contains no manifest text rewriting, no embedded Python validation, and no version-parity logic; all such inputs are produced by Nix and passed in. +- The Linux flake exposes exactly these Flatpak-related app outputs: + - `nix run .#sce-flatpak -- ` — umbrella entrypoint that delegates to `sce-flatpak.sh` for static validation (`validate`), local manifest preparation (`prepare-local-manifest --repo-root `), and other thin orchestration subcommands previously split across separate wrapper apps. - `nix run .#release-flatpak-package -- --version --out-dir ` for deterministic GitHub Release source-manifest tarball/checksum/JSON assets. - `nix run .#release-flatpak-bundle -- --version --arch --out-dir ` for deterministic GitHub Release source-built `.flatpak` bundle/checksum/JSON assets. -- `checks..flatpak-static-validation` runs the lightweight validation path during default `nix flake check`; it does not run a full Flatpak build or require network access. -- The Linux dev shell includes `appstreamcli`, `flatpak`, and `flatpak-builder`, and its banner lists the Flatpak local/release flake apps alongside existing repo app entrypoints. -- The checked-in release manifest remains Flathub-style and source-built; Nix orchestration only supplies tools and generated local manifests and must not provide a prebuilt `sce` binary to the Flatpak package. + - Plus helper apps for the generated artifacts: `regenerate-flatpak-manifest`, `regenerate-cargo-sources`. +- The previously separate `flatpak-validate`, `flatpak-local-manifest`, and `flatpak-build` wrapper apps are removed; their behaviors live under `sce-flatpak `. +- `checks..flatpak-static-validation` runs the Nix-built static validator script during default `nix flake check`; `checks..flatpak-manifest-parity` and `checks..cargo-sources-parity` enforce the generated-artifact parity contracts. Default `nix flake check` remains lightweight: no network-heavy `flatpak-builder` runs. +- The Linux dev shell includes `appstreamcli`, `flatpak`, and `flatpak-builder`, and its banner lists the reduced Flatpak app surface alongside existing repo app entrypoints. +- The checked-in release manifest remains Flathub-style and source-built; Nix orchestration only supplies generated manifests/cargo-sources/validators and must not provide a prebuilt `sce` binary to the Flatpak package. ## Explicitly out of scope in this phase diff --git a/context/sce/cli-release-artifact-contract.md b/context/sce/cli-release-artifact-contract.md index 9f31bfff..e5a0bced 100644 --- a/context/sce/cli-release-artifact-contract.md +++ b/context/sce/cli-release-artifact-contract.md @@ -37,7 +37,7 @@ This file captures the current shared release artifact foundation plus the appro - `sce-v-aarch64.flatpak` + `.sha256` + `.json` - The JSON metadata describes `asset_type: flatpak-bundle`, architecture field (`x86_64` / `aarch64`), app ID `dev.crocoder.sce`, version, and SHA-256 checksum. - Bundle assets are separate from native binary release archives, the signed native release manifest consumed by npm, and the existing Flatpak source-manifest packaging assets. -- The release-bundle command (`packaging/flatpak/sce-flatpak.sh release-bundle`) is exposed as `nix run .#release-flatpak-bundle -- --version --arch --out-dir `. GitHub workflow upload for these assets is implemented (`.github/workflows/release-sce-linux.yml` / `release-sce-linux-arm.yml` build and upload; `release-sce.yml` assembles and publishes with the GitHub Release). +- The release-bundle command (`packaging/flatpak/sce-flatpak.sh release-bundle`) is exposed as `nix run .#release-flatpak-bundle -- --version --arch --out-dir `. GitHub workflow upload for these assets is implemented (`.github/workflows/release-sce-linux.yml` / `release-sce-linux-arm.yml` build and upload; `release-sce.yml` assembles and publishes with the GitHub Release). The bash script is a thin imperative orchestrator around `flatpak-builder` + `flatpak build-bundle`; the manifest YAML it consumes is rendered from a Nix expression (`nix/flatpak/manifest.nix`) and the `cargo-sources.json` it consumes is generated from `cli/Cargo.lock` by a Nix derivation (`nix/flatpak/cargo-sources.nix`), both guarded by parity checks (`flatpak-manifest-parity`, `cargo-sources-parity`) in `nix flake check`. Static manifest validation and release-version parity validation run from dedicated Nix-built Python scripts rather than from inline bash heredocs, while local-checkout manifest validation is a Nix-wrapped Bash script used by `sce-flatpak prepare-local-manifest`. ## Archive contents diff --git a/context/sce/optional-install-channel-integration-test-entrypoint.md b/context/sce/optional-install-channel-integration-test-entrypoint.md index ccb54e9e..0fc9cf2f 100644 --- a/context/sce/optional-install-channel-integration-test-entrypoint.md +++ b/context/sce/optional-install-channel-integration-test-entrypoint.md @@ -24,6 +24,6 @@ The repository exposes an explicit opt-in flake app for install-channel integrat - The Rust runner already has dedicated default-flake checks: `integrations-install-fmt`, `integrations-install-clippy`, and `integrations-install-tests`. - The opt-in app remains outside default `nix flake check`. -- Real npm, Bun, and Cargo install orchestration now run through the Rust runner behind the unchanged selector contract for the existing binary install channels. Flatpak validation/build orchestration is implemented as a separate Nix-owned source-build path (`flatpak-validate`, `flatpak-local-manifest`, `flatpak-build`, and `flatpak-static-validation`) rather than part of this app's current selector contract. +- Real npm, Bun, and Cargo install orchestration now run through the Rust runner behind the unchanged selector contract for the existing binary install channels. Flatpak validation/build orchestration is implemented as a separate Nix-owned source-build path (umbrella `sce-flatpak ` for `validate` / `prepare-local-manifest`, `release-flatpak-package`, `release-flatpak-bundle`, and the `flatpak-static-validation` flake check) rather than part of this app's current selector contract. See also: [../overview.md](../overview.md), [../architecture.md](../architecture.md), [../patterns.md](../patterns.md) diff --git a/flake.lock b/flake.lock index 27ab8baf..c17bd232 100644 --- a/flake.lock +++ b/flake.lock @@ -33,6 +33,22 @@ "type": "github" } }, + "flatpak-builder-tools": { + "flake": false, + "locked": { + "lastModified": 1780983860, + "narHash": "sha256-fiSSy1858b3V6VFEViQq6nkU57XInnDsB5LByxfpgf4=", + "owner": "flatpak", + "repo": "flatpak-builder-tools", + "rev": "737c0085912f9f7dabf9341d4608e2a77a51a73a", + "type": "github" + }, + "original": { + "owner": "flatpak", + "repo": "flatpak-builder-tools", + "type": "github" + } + }, "nixpkgs": { "locked": { "lastModified": 1778869304, @@ -88,6 +104,7 @@ "inputs": { "crane": "crane", "flake-utils": "flake-utils", + "flatpak-builder-tools": "flatpak-builder-tools", "nixpkgs": "nixpkgs", "opencode": "opencode", "opencode-nixpkgs": [ diff --git a/flake.nix b/flake.nix index 3bcb0a7f..f713e08e 100644 --- a/flake.nix +++ b/flake.nix @@ -14,6 +14,8 @@ turso.inputs.flake-utils.follows = "flake-utils"; turso.inputs.crane.follows = "crane"; turso.inputs.rust-overlay.follows = "rust-overlay"; + flatpak-builder-tools.url = "github:flatpak/flatpak-builder-tools"; + flatpak-builder-tools.flake = false; }; outputs = @@ -26,6 +28,7 @@ opencode, opencode-nixpkgs, turso, + flatpak-builder-tools, }: flake-utils.lib.eachDefaultSystem ( system: @@ -808,35 +811,52 @@ pkgs.flatpak-builder ]; - flatpakToolApp = pkgs.writeShellApplication { - name = "sce-flatpak"; - runtimeInputs = flatpakToolRuntimeInputs; - text = '' - exec ${pkgs.bash}/bin/bash ${./packaging/flatpak/sce-flatpak.sh} "$@" - ''; + flatpakManifest = import ./nix/flatpak/manifest.nix { + inherit pkgs; + checkedInYaml = ./packaging/flatpak/dev.crocoder.sce.yml; }; - flatpakValidateApp = pkgs.writeShellApplication { - name = "flatpak-validate"; - runtimeInputs = [ flatpakToolApp ]; - text = '' - exec sce-flatpak validate "$@" - ''; + flatpakStaticCheckApp = pkgs.writeShellApplication { + name = "flatpak-static-check"; + runtimeInputs = [ + pkgs.coreutils + pkgs.gawk + pkgs.jq + pkgs.libxml2 + ]; + text = builtins.readFile ./nix/flatpak/static-validate.sh; }; - flatpakLocalManifestApp = pkgs.writeShellApplication { - name = "flatpak-local-manifest"; - runtimeInputs = [ flatpakToolApp ]; - text = '' - exec sce-flatpak prepare-local-manifest "$@" - ''; + flatpakVersionParityCheckApp = pkgs.writeShellApplication { + name = "flatpak-version-parity-check"; + runtimeInputs = [ + pkgs.coreutils + pkgs.gnused + pkgs.jq + pkgs.libxml2 + ]; + text = builtins.readFile ./nix/flatpak/version-parity.sh; }; - flatpakBuildApp = pkgs.writeShellApplication { - name = "flatpak-build"; - runtimeInputs = [ flatpakToolApp ]; + flatpakLocalManifestCheckApp = pkgs.writeShellApplication { + name = "flatpak-local-manifest-check"; + runtimeInputs = [ pkgs.coreutils ]; + text = builtins.readFile ./nix/flatpak/local-manifest-validate.sh; + }; + + flatpakToolApp = pkgs.writeShellApplication { + name = "sce-flatpak"; + runtimeInputs = flatpakToolRuntimeInputs; text = '' - exec sce-flatpak build "$@" + export SCE_FLATPAK_RELEASE_MANIFEST="${flatpakManifest.releaseManifest}" + export SCE_FLATPAK_LOCAL_MANIFEST_TEMPLATE="${flatpakManifest.localManifestTemplate}" + export SCE_FLATPAK_COMMIT_MANIFEST_TEMPLATE="${flatpakManifest.commitManifestTemplate}" + export SCE_FLATPAK_LOCAL_PATH_PLACEHOLDER="${flatpakManifest.localPathPlaceholder}" + export SCE_FLATPAK_COMMIT_PLACEHOLDER="${flatpakManifest.commitPlaceholder}" + export SCE_FLATPAK_STATIC_CHECK="${flatpakStaticCheckApp}/bin/flatpak-static-check" + export SCE_FLATPAK_VERSION_PARITY_CHECK="${flatpakVersionParityCheckApp}/bin/flatpak-version-parity-check" + export SCE_FLATPAK_LOCAL_MANIFEST_CHECK="${flatpakLocalManifestCheckApp}/bin/flatpak-local-manifest-check" + exec ${pkgs.bash}/bin/bash ${./packaging/flatpak/sce-flatpak.sh} "$@" ''; }; @@ -856,9 +876,16 @@ ''; }; + flatpakCargoSources = import ./nix/flatpak/cargo-sources.nix { + inherit pkgs; + flatpakBuilderToolsSrc = flatpak-builder-tools; + cargoLock = ./cli/Cargo.lock; + checkedInJson = ./packaging/flatpak/cargo-sources.json; + }; + flatpakStaticValidationCheck = pkgs.runCommand "flatpak-static-validation" { - nativeBuildInputs = [ flatpakToolApp ]; + nativeBuildInputs = [ flatpakStaticCheckApp ]; } '' set -euo pipefail @@ -867,7 +894,7 @@ chmod -R u+w ./repo cd ./repo - sce-flatpak validate --repo-root "$PWD" --skip-optional-lint + flatpak-static-check --repo-root "$PWD" mkdir -p "$out" ''; @@ -1146,6 +1173,8 @@ } // pkgs.lib.optionalAttrs pkgs.stdenv.isLinux { flatpak-static-validation = flatpakStaticValidationCheck; + cargo-sources-parity = flatpakCargoSources.parityCheck; + flatpak-manifest-parity = flatpakManifest.parityCheck; }; apps = @@ -1202,43 +1231,67 @@ }; } // pkgs.lib.optionalAttrs pkgs.stdenv.isLinux { - flatpak-validate = { + sce-flatpak = { type = "app"; - program = "${flatpakValidateApp}/bin/flatpak-validate"; + program = "${flatpakToolApp}/bin/sce-flatpak"; meta = { - description = "Validate Flatpak packaging metadata and local-source manifest generation"; + description = "Flatpak packaging umbrella (validate, prepare-local-manifest, release-package, release-bundle)"; }; }; - flatpak-local-manifest = { + release-flatpak-package = { type = "app"; - program = "${flatpakLocalManifestApp}/bin/flatpak-local-manifest"; + program = "${releaseFlatpakPackageApp}/bin/release-flatpak-package"; meta = { - description = "Generate a Flatpak manifest that builds from the current checkout"; + description = "Build Flatpak source-manifest GitHub Release assets"; }; }; - flatpak-build = { + release-flatpak-bundle = { type = "app"; - program = "${flatpakBuildApp}/bin/flatpak-build"; + program = "${releaseFlatpakBundleApp}/bin/release-flatpak-bundle"; meta = { - description = "Build the sce Flatpak from the current checkout with flatpak-builder"; + description = "Build Flatpak bundle GitHub Release assets"; }; }; - release-flatpak-package = { + flatpak-static-check = { type = "app"; - program = "${releaseFlatpakPackageApp}/bin/release-flatpak-package"; + program = "${flatpakStaticCheckApp}/bin/flatpak-static-check"; meta = { - description = "Build Flatpak source-manifest GitHub Release assets"; + description = "Static Flatpak packaging validation (manifest, banned snippets, cargo-sources, metainfo)"; }; }; - release-flatpak-bundle = { + flatpak-version-parity-check = { type = "app"; - program = "${releaseFlatpakBundleApp}/bin/release-flatpak-bundle"; + program = "${flatpakVersionParityCheckApp}/bin/flatpak-version-parity-check"; meta = { - description = "Build Flatpak bundle GitHub Release assets"; + description = "Validate sce release version parity across .version, Cargo.toml, npm package.json, and Flatpak metainfo"; + }; + }; + + flatpak-local-manifest-check = { + type = "app"; + program = "${flatpakLocalManifestCheckApp}/bin/flatpak-local-manifest-check"; + meta = { + description = "Validate a generated local-checkout Flatpak manifest"; + }; + }; + + regenerate-cargo-sources = { + type = "app"; + program = "${flatpakCargoSources.regenerateApp}/bin/regenerate-cargo-sources"; + meta = { + description = "Regenerate packaging/flatpak/cargo-sources.json from cli/Cargo.lock"; + }; + }; + + regenerate-flatpak-manifest = { + type = "app"; + program = "${flatpakManifest.regenerateApp}/bin/regenerate-flatpak-manifest"; + meta = { + description = "Regenerate packaging/flatpak/dev.crocoder.sce.yml from nix/flatpak/manifest.nix"; }; }; }; @@ -1296,9 +1349,7 @@ echo "- flatpak: $(version_of flatpak)" echo "- flatpak-builder: $(version_of flatpak-builder)" echo "- appstreamcli: $(version_of appstreamcli)" - echo "- flatpak-validate: nix run .#flatpak-validate" - echo "- flatpak-local-manifest: nix run .#flatpak-local-manifest" - echo "- flatpak-build: nix run .#flatpak-build -- --help" + echo "- sce-flatpak: nix run .#sce-flatpak -- --help" echo "- release-flatpak-package: nix run .#release-flatpak-package -- --help" echo "- release-flatpak-bundle: nix run .#release-flatpak-bundle -- --help" ''} diff --git a/nix/flatpak/cargo-sources.nix b/nix/flatpak/cargo-sources.nix new file mode 100644 index 00000000..fcaa7e07 --- /dev/null +++ b/nix/flatpak/cargo-sources.nix @@ -0,0 +1,74 @@ +{ + pkgs, + flatpakBuilderToolsSrc, + cargoLock, + checkedInJson, +}: + +let + pythonEnv = pkgs.python3.withPackages (ps: [ ps.aiohttp ps.tomlkit ]); + + generatorScript = "${flatpakBuilderToolsSrc}/cargo/flatpak-cargo-generator.py"; + + cargoSourcesJson = pkgs.stdenvNoCC.mkDerivation { + pname = "sce-flatpak-cargo-sources"; + version = "1"; + + dontUnpack = true; + + nativeBuildInputs = [ pythonEnv ]; + + buildPhase = '' + runHook preBuild + cp ${cargoLock} ./Cargo.lock + python3 ${generatorScript} ./Cargo.lock -o cargo-sources.json + runHook postBuild + ''; + + installPhase = '' + runHook preInstall + cp cargo-sources.json $out + runHook postInstall + ''; + + outputHashMode = "flat"; + outputHashAlgo = "sha256"; + outputHash = "sha256-3xwpQ3TQ3/QSduJ5rKmpiU55heIEDOs2P89RFOZoqeo="; + }; + + regenerateApp = pkgs.writeShellApplication { + name = "regenerate-cargo-sources"; + runtimeInputs = [ pkgs.coreutils ]; + text = '' + set -euo pipefail + if [ -z "''${SCE_REPO_ROOT:-}" ]; then + SCE_REPO_ROOT="$(pwd)" + fi + target="$SCE_REPO_ROOT/packaging/flatpak/cargo-sources.json" + if [ ! -e "$target" ]; then + echo "regenerate-cargo-sources: expected $target to exist" >&2 + exit 1 + fi + install -m 0644 ${cargoSourcesJson} "$target" + echo "regenerate-cargo-sources: wrote $target" >&2 + ''; + }; + + parityCheck = pkgs.runCommand "cargo-sources-parity" + { + nativeBuildInputs = [ pkgs.diffutils ]; + } + '' + set -euo pipefail + if ! diff -u ${checkedInJson} ${cargoSourcesJson}; then + echo "" >&2 + echo "cargo-sources-parity: packaging/flatpak/cargo-sources.json is out of sync with cli/Cargo.lock." >&2 + echo " Regenerate with: nix run .#regenerate-cargo-sources" >&2 + exit 1 + fi + mkdir -p "$out" + ''; + +in { + inherit cargoSourcesJson regenerateApp parityCheck; +} diff --git a/nix/flatpak/local-manifest-validate.sh b/nix/flatpak/local-manifest-validate.sh new file mode 100644 index 00000000..2126ef3a --- /dev/null +++ b/nix/flatpak/local-manifest-validate.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + printf 'usage: local-manifest-validate.sh --repo-root --manifest-path \n' >&2 +} + +fail_usage() { + usage + exit 2 +} + +repo_root="" +manifest_path="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --repo-root) + [[ $# -ge 2 ]] || fail_usage + repo_root="$2" + shift 2 + ;; + --manifest-path) + [[ $# -ge 2 ]] || fail_usage + manifest_path="$2" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + fail_usage + ;; + esac +done + +[[ -n "$repo_root" ]] || fail_usage +[[ -n "$manifest_path" ]] || fail_usage + +repo_root="$(realpath "$repo_root")" +manifest="$(<"$manifest_path")" +expected_path="path: $repo_root" + +errors=() + +if [[ "$manifest" != *"type: dir"* ]]; then + errors+=("local manifest does not contain a Flatpak type: dir source") +fi + +if [[ "$manifest" != *"$expected_path"* ]]; then + errors+=("local manifest does not point at the requested checkout path") +fi + +if [[ "$manifest" == *"nix build .#sce"* || "$manifest" == *"nix build .#default"* ]]; then + errors+=("local manifest references a Nix-built sce binary") +fi + +if [[ "$manifest" != *"cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce"* ]]; then + errors+=("local manifest no longer runs the Flatpak Cargo source build") +fi + +if [[ ${#errors[@]} -gt 0 ]]; then + for error in "${errors[@]}"; do + printf 'Flatpak local-manifest validation failed: %s\n' "$error" >&2 + done + exit 1 +fi diff --git a/nix/flatpak/manifest.nix b/nix/flatpak/manifest.nix new file mode 100644 index 00000000..431cf8ee --- /dev/null +++ b/nix/flatpak/manifest.nix @@ -0,0 +1,150 @@ +{ + pkgs, + checkedInYaml, + releaseCommit ? "b7f0fa002fca5f5320791ff5e4abfaadfcddf187", +}: + +let + inherit (pkgs) lib; + + yamlFormat = pkgs.formats.yaml { }; + + appId = "dev.crocoder.sce"; + manifestFileName = "${appId}.yml"; + + localPathPlaceholder = "__SCE_LOCAL_REPO_PATH__"; + commitPlaceholder = "__SCE_RELEASE_COMMIT__"; + + releaseGitSource = commit: { + type = "git"; + url = "https://github.com/crocoder-dev/shared-context-engineering.git"; + inherit commit; + }; + + localDirSource = repoPath: { + type = "dir"; + path = repoPath; + }; + + baseManifest = sceSource: { + id = appId; + runtime = "org.freedesktop.Platform"; + runtime-version = "25.08"; + sdk = "org.freedesktop.Sdk"; + sdk-extensions = [ "org.freedesktop.Sdk.Extension.rust-stable" ]; + command = "sce"; + finish-args = [ + "--share=network" + "--filesystem=home" + "--talk-name=org.freedesktop.Flatpak" + "--talk-name=org.freedesktop.secrets" + ]; + modules = [ + { + name = "sce"; + buildsystem = "simple"; + build-options = { + append-path = "/usr/lib/sdk/rust-stable/bin"; + env = { + CARGO_HOME = "/run/build/sce/cargo"; + }; + }; + build-commands = [ + ''bash ./scripts/prepare-cli-generated-assets.sh "$PWD"'' + "cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce" + "install -Dm755 cli/target/release/sce /app/bin/sce" + "install -Dm755 packaging/flatpak/git-host-bridge /app/bin/git" + "install -Dm644 packaging/flatpak/dev.crocoder.sce.metainfo.xml /app/share/metainfo/dev.crocoder.sce.metainfo.xml" + ]; + sources = [ + sceSource + { + type = "file"; + path = "dev.crocoder.sce.metainfo.xml"; + dest = "packaging/flatpak"; + } + { + type = "file"; + path = "git-host-bridge"; + dest = "packaging/flatpak"; + } + "cargo-sources.json" + ]; + } + ]; + }; + + manifestFor = + { + sourceKind, + commit ? null, + repoPath ? null, + }: + let + sceSource = + if sourceKind == "release" then + releaseGitSource releaseCommit + else if sourceKind == "commit" then + releaseGitSource ( + if commit == null then commitPlaceholder else commit + ) + else if sourceKind == "local" then + localDirSource (if repoPath == null then localPathPlaceholder else repoPath) + else + throw "manifest.nix: unknown sourceKind '${sourceKind}'"; + in + baseManifest sceSource; + + releaseManifest = yamlFormat.generate manifestFileName (manifestFor { sourceKind = "release"; }); + localManifestTemplate = yamlFormat.generate "${appId}.local-template.yml" ( + manifestFor { sourceKind = "local"; } + ); + commitManifestTemplate = yamlFormat.generate "${appId}.commit-template.yml" ( + manifestFor { sourceKind = "commit"; } + ); + + regenerateApp = pkgs.writeShellApplication { + name = "regenerate-flatpak-manifest"; + runtimeInputs = [ pkgs.coreutils ]; + text = '' + set -euo pipefail + if [ -z "''${SCE_REPO_ROOT:-}" ]; then + SCE_REPO_ROOT="$(pwd)" + fi + target="$SCE_REPO_ROOT/packaging/flatpak/${manifestFileName}" + if [ ! -e "$target" ]; then + echo "regenerate-flatpak-manifest: expected $target to exist" >&2 + exit 1 + fi + install -m 0644 ${releaseManifest} "$target" + echo "regenerate-flatpak-manifest: wrote $target" >&2 + ''; + }; + + parityCheck = + pkgs.runCommand "flatpak-manifest-parity" + { + nativeBuildInputs = [ pkgs.diffutils ]; + } + '' + set -euo pipefail + if ! diff -u ${checkedInYaml} ${releaseManifest}; then + echo "" >&2 + echo "flatpak-manifest-parity: packaging/flatpak/${manifestFileName} is out of sync with nix/flatpak/manifest.nix." >&2 + echo " Regenerate with: nix run .#regenerate-flatpak-manifest" >&2 + exit 1 + fi + mkdir -p "$out" + ''; +in +{ + inherit + releaseManifest + localManifestTemplate + commitManifestTemplate + regenerateApp + parityCheck + localPathPlaceholder + commitPlaceholder + ; +} diff --git a/nix/flatpak/static-validate.sh b/nix/flatpak/static-validate.sh new file mode 100644 index 00000000..12cf7897 --- /dev/null +++ b/nix/flatpak/static-validate.sh @@ -0,0 +1,135 @@ +#!/usr/bin/env bash +set -euo pipefail + +APP_ID="dev.crocoder.sce" + +usage() { + printf 'usage: static-validate.sh --repo-root \n' >&2 +} + +fail_usage() { + usage + exit 2 +} + +repo_root="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --repo-root) + [[ $# -ge 2 ]] || fail_usage + repo_root="$2" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + fail_usage + ;; + esac +done + +[[ -n "$repo_root" ]] || fail_usage + +flatpak_dir="$repo_root/packaging/flatpak" +manifest_path="$flatpak_dir/$APP_ID.yml" +metainfo_path="$flatpak_dir/$APP_ID.metainfo.xml" +cargo_sources_path="$flatpak_dir/cargo-sources.json" + +if [[ ! -r "$manifest_path" ]]; then + printf 'could not read %s\n' "$manifest_path" >&2 + exit 1 +fi + +manifest="$(<"$manifest_path")" +errors=() + +require_contains() { + local needle="$1" + local message="$2" + + if [[ "$manifest" != *"$needle"* ]]; then + errors+=("$message") + fi +} + +require_contains "id: dev.crocoder.sce" "manifest app ID is not dev.crocoder.sce" +require_contains "command: sce" "manifest command is not sce" +require_contains "org.freedesktop.Sdk.Extension.rust-stable" "Rust SDK extension is missing" +require_contains "bash ./scripts/prepare-cli-generated-assets.sh \"\$PWD\"" "generated-asset preparation command is missing" +require_contains "cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce" "offline Cargo source-build command is missing" +require_contains "install -Dm755 cli/target/release/sce /app/bin/sce" "sce install command is missing" +require_contains "install -Dm755 packaging/flatpak/git-host-bridge /app/bin/git" "host git bridge install command is missing" +require_contains "--talk-name=org.freedesktop.Flatpak" "host Flatpak permission is missing" +require_contains "cargo-sources.json" "Cargo source descriptor is missing from manifest" + +if ! awk ' + /^[[:space:]]*-[[:space:]]+commit:[[:space:]]+[0-9a-f]{40}[[:space:]]*$/ { state = 1; next } + state == 1 && /^[[:space:]]+type:[[:space:]]+git[[:space:]]*$/ { state = 2; next } + state == 2 && /^[[:space:]]+url:[[:space:]]+https:\/\/github\.com\/crocoder-dev\/shared-context-engineering\.git[[:space:]]*$/ { found = 1 } + { state = 0 } + END { exit found ? 0 : 1 } +' "$manifest_path"; then + errors+=("release manifest does not use a pinned git source") +fi + +banned_snippets=( + "nix build .#sce" + "nix build .#default" + "github.com/crocoder-dev/shared-context-engineering/releases" + "release-artifacts" + "@crocoder-dev/sce" +) + +for path in "$flatpak_dir"/*; do + [[ -f "$path" ]] || continue + [[ "$(basename "$path")" != "sce-flatpak.sh" ]] || continue + + if [[ ! -r "$path" ]]; then + printf 'could not read %s\n' "$path" >&2 + exit 1 + fi + + text="$(<"$path")" + relative_path="${path#"$repo_root/"}" + + for snippet in "${banned_snippets[@]}"; do + if [[ "$text" == *"$snippet"* ]]; then + errors+=("$relative_path references disallowed artifact source: $snippet") + fi + done +done + +if ! jq -e 'type == "array" and length > 0' "$cargo_sources_path" >/dev/null 2>&1; then + errors+=("cargo-sources.json is empty or not a list") +fi + +if ! jq -e 'any(.[]; type == "object" and ((.url // "") | contains("turso")))' "$cargo_sources_path" >/dev/null 2>&1; then + errors+=("Turso cargo-sources entry is missing") +fi + +if ! jq -e 'any(.[]; type == "object" and .type == "archive" and ((.url // "") | contains("static.crates.io")))' "$cargo_sources_path" >/dev/null 2>&1; then + errors+=("crates.io archive source entries are missing") +fi + +if ! metainfo_id="$(xmllint --xpath 'string(/component/id)' "$metainfo_path" 2>/dev/null)"; then + printf 'could not parse %s\n' "$metainfo_path" >&2 + exit 1 +fi + +if [[ "$metainfo_id" != "$APP_ID" ]]; then + errors+=("metainfo ID is not dev.crocoder.sce") +fi + +if ! xmllint --xpath '/component/provides/binary[text()="sce"]' "$metainfo_path" >/dev/null 2>&1; then + errors+=("metainfo does not provide binary sce") +fi + +if [[ ${#errors[@]} -gt 0 ]]; then + for error in "${errors[@]}"; do + printf 'Flatpak static validation failed: %s\n' "$error" >&2 + done + exit 1 +fi diff --git a/nix/flatpak/version-parity.sh b/nix/flatpak/version-parity.sh new file mode 100755 index 00000000..284bfc9c --- /dev/null +++ b/nix/flatpak/version-parity.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +set -euo pipefail + +APP_ID="dev.crocoder.sce" + +usage() { + printf 'usage: version-parity.sh --repo-root --version \n' >&2 +} + +fail_usage() { + usage + exit 2 +} + +trim_whitespace() { + sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' +} + +read_trimmed_file() { + local path="$1" + + if [[ ! -r "$path" ]]; then + printf 'could not read %s\n' "$path" >&2 + exit 1 + fi + + trim_whitespace < "$path" +} + +repo_root="" +version="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --repo-root) + [[ $# -ge 2 ]] || fail_usage + repo_root="$2" + shift 2 + ;; + --version) + [[ $# -ge 2 ]] || fail_usage + version="$2" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + fail_usage + ;; + esac +done + +[[ -n "$repo_root" ]] || fail_usage +[[ -n "$version" ]] || fail_usage + +version_path="$repo_root/.version" +cargo_toml_path="$repo_root/cli/Cargo.toml" +npm_package_path="$repo_root/npm/package.json" +metainfo_path="$repo_root/packaging/flatpak/$APP_ID.metainfo.xml" + +checked_in_version="$(read_trimmed_file "$version_path")" + +if [[ ! -r "$cargo_toml_path" ]]; then + printf 'could not read %s\n' "$cargo_toml_path" >&2 + exit 1 +fi + +cargo_version="$(sed -n 's/^version = "\([^"]*\)"$/\1/p' "$cargo_toml_path" | sed -n '1p')" + +if [[ ! -r "$npm_package_path" ]]; then + printf 'could not parse %s\n' "$npm_package_path" >&2 + exit 1 +fi + +if ! npm_version="$(jq -r '.version // ""' "$npm_package_path")"; then + printf 'could not parse %s\n' "$npm_package_path" >&2 + exit 1 +fi + +if [[ ! -r "$metainfo_path" ]]; then + printf 'could not parse %s\n' "$metainfo_path" >&2 + exit 1 +fi + +if ! flatpak_version="$(xmllint --xpath 'string((/component/releases/release/@version)[1])' "$metainfo_path" 2>/dev/null)"; then + printf 'could not parse %s\n' "$metainfo_path" >&2 + exit 1 +fi + +errors=() + +if [[ -z "$cargo_version" ]]; then + errors+=("cli/Cargo.toml package version is missing") +fi + +if [[ -z "$flatpak_version" ]]; then + errors+=("Flatpak metainfo release metadata is missing") +fi + +if [[ "$version" != "$checked_in_version" ]]; then + errors+=("requested release version $version does not match .version $checked_in_version") +fi + +if [[ "$version" != "$cargo_version" ]]; then + errors+=("cli/Cargo.toml version $cargo_version does not match release version $version") +fi + +if [[ "$version" != "$npm_version" ]]; then + errors+=("npm/package.json version $npm_version does not match release version $version") +fi + +if [[ "$version" != "$flatpak_version" ]]; then + errors+=("Flatpak metainfo release version $flatpak_version does not match release version $version") +fi + +if [[ ${#errors[@]} -gt 0 ]]; then + for error in "${errors[@]}"; do + printf 'Flatpak release version validation failed: %s\n' "$error" >&2 + done + exit 1 +fi diff --git a/packaging/flatpak/cargo-sources.json b/packaging/flatpak/cargo-sources.json index f5b2fc60..242c90fe 100644 --- a/packaging/flatpak/cargo-sources.json +++ b/packaging/flatpak/cargo-sources.json @@ -1,10 +1,4 @@ [ - { - "type": "git", - "url": "https://github.com/tursodatabase/turso", - "commit": "fed0b50b8aa697f56ab015aa9e905e0f56371092", - "dest": "flatpak-cargo/git/turso-fed0b50" - }, { "type": "archive", "archive-type": "tar-gzip", @@ -1123,19 +1117,6 @@ "dest": "cargo/vendor/crossbeam-epoch-0.9.18", "dest-filename": ".cargo-checksum.json" }, - { - "type": "archive", - "archive-type": "tar-gzip", - "url": "https://static.crates.io/crates/crossbeam-skiplist/crossbeam-skiplist-0.1.3.crate", - "sha256": "df29de440c58ca2cc6e587ec3d22347551a32435fbde9d2bff64e78a9ffa151b", - "dest": "cargo/vendor/crossbeam-skiplist-0.1.3" - }, - { - "type": "inline", - "contents": "{\"package\": \"df29de440c58ca2cc6e587ec3d22347551a32435fbde9d2bff64e78a9ffa151b\", \"files\": {}}", - "dest": "cargo/vendor/crossbeam-skiplist-0.1.3", - "dest-filename": ".cargo-checksum.json" - }, { "type": "archive", "archive-type": "tar-gzip", @@ -5387,6 +5368,123 @@ "dest": "cargo/vendor/try-lock-0.2.5", "dest-filename": ".cargo-checksum.json" }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso/turso-0.7.0-pre.10.crate", + "sha256": "1d01c4d88407d9f140d7c20700820a3ed848a0f38f1502557b731cb7432e7025", + "dest": "cargo/vendor/turso-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"1d01c4d88407d9f140d7c20700820a3ed848a0f38f1502557b731cb7432e7025\", \"files\": {}}", + "dest": "cargo/vendor/turso-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_core/turso_core-0.7.0-pre.10.crate", + "sha256": "41524b79d4649f57b242dc8624a583c1070277da3f35845b668822e549c84a6d", + "dest": "cargo/vendor/turso_core-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"41524b79d4649f57b242dc8624a583c1070277da3f35845b668822e549c84a6d\", \"files\": {}}", + "dest": "cargo/vendor/turso_core-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_ext/turso_ext-0.7.0-pre.10.crate", + "sha256": "054da6e06172506a9c71488777a57abc378bf00bb612e7a175d226019678f74e", + "dest": "cargo/vendor/turso_ext-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"054da6e06172506a9c71488777a57abc378bf00bb612e7a175d226019678f74e\", \"files\": {}}", + "dest": "cargo/vendor/turso_ext-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_macros/turso_macros-0.7.0-pre.10.crate", + "sha256": "ef5f24b2872ee08384c0b52a576f2ad27e56596832676efad437b219925232f4", + "dest": "cargo/vendor/turso_macros-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"ef5f24b2872ee08384c0b52a576f2ad27e56596832676efad437b219925232f4\", \"files\": {}}", + "dest": "cargo/vendor/turso_macros-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_parser/turso_parser-0.7.0-pre.10.crate", + "sha256": "8ae264e3470c5849d2b05d6759908b543cb805cf188145d7ea07be2b3e765144", + "dest": "cargo/vendor/turso_parser-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"8ae264e3470c5849d2b05d6759908b543cb805cf188145d7ea07be2b3e765144\", \"files\": {}}", + "dest": "cargo/vendor/turso_parser-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_sdk_kit/turso_sdk_kit-0.7.0-pre.10.crate", + "sha256": "941d1ee5e6a27e83b239410db6af385acc3bfc54e81e344e48dd7e42428d23ee", + "dest": "cargo/vendor/turso_sdk_kit-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"941d1ee5e6a27e83b239410db6af385acc3bfc54e81e344e48dd7e42428d23ee\", \"files\": {}}", + "dest": "cargo/vendor/turso_sdk_kit-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_sdk_kit_macros/turso_sdk_kit_macros-0.7.0-pre.10.crate", + "sha256": "59374eb3d3c7ffe4f9fccd3352f1c80fb1265f265fc0131bc62ab7573bee1355", + "dest": "cargo/vendor/turso_sdk_kit_macros-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"59374eb3d3c7ffe4f9fccd3352f1c80fb1265f265fc0131bc62ab7573bee1355\", \"files\": {}}", + "dest": "cargo/vendor/turso_sdk_kit_macros-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_sync_engine/turso_sync_engine-0.7.0-pre.10.crate", + "sha256": "e88f961e7a0b7679ca09122ead618faf14094fb7b88f51691e8263486fc7fe3f", + "dest": "cargo/vendor/turso_sync_engine-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"e88f961e7a0b7679ca09122ead618faf14094fb7b88f51691e8263486fc7fe3f\", \"files\": {}}", + "dest": "cargo/vendor/turso_sync_engine-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, + { + "type": "archive", + "archive-type": "tar-gzip", + "url": "https://static.crates.io/crates/turso_sync_sdk_kit/turso_sync_sdk_kit-0.7.0-pre.10.crate", + "sha256": "122e7c09a1470e0400f8db13ee7589e610c4ad1f2cc2034309e7ca9cfcd7c978", + "dest": "cargo/vendor/turso_sync_sdk_kit-0.7.0-pre.10" + }, + { + "type": "inline", + "contents": "{\"package\": \"122e7c09a1470e0400f8db13ee7589e610c4ad1f2cc2034309e7ca9cfcd7c978\", \"files\": {}}", + "dest": "cargo/vendor/turso_sync_sdk_kit-0.7.0-pre.10", + "dest-filename": ".cargo-checksum.json" + }, { "type": "archive", "archive-type": "tar-gzip", @@ -6780,8 +6878,8 @@ }, { "type": "inline", - "contents": "[source.crates-io]\nreplace-with = \"vendored-sources\"\n\n[source.vendored-sources]\ndirectory = \"cargo/vendor\"\n\n[patch.\"https://github.com/tursodatabase/turso\"]\nturso = { path = \"flatpak-cargo/git/turso-fed0b50/bindings/rust\" }\nturso_core = { path = \"flatpak-cargo/git/turso-fed0b50/core\" }\nturso_ext = { path = \"flatpak-cargo/git/turso-fed0b50/extensions/core\" }\nturso_macros = { path = \"flatpak-cargo/git/turso-fed0b50/macros\" }\nturso_parser = { path = \"flatpak-cargo/git/turso-fed0b50/parser\" }\nturso_sdk_kit = { path = \"flatpak-cargo/git/turso-fed0b50/sdk-kit\" }\nturso_sdk_kit_macros = { path = \"flatpak-cargo/git/turso-fed0b50/sdk-kit-macros\" }\nturso_sync_engine = { path = \"flatpak-cargo/git/turso-fed0b50/sync/engine\" }\nturso_sync_sdk_kit = { path = \"flatpak-cargo/git/turso-fed0b50/sync/sdk-kit\" }\n", + "contents": "[source.vendored-sources]\ndirectory = \"cargo/vendor\"\n\n[source.crates-io]\nreplace-with = \"vendored-sources\"\n", "dest": "cargo", "dest-filename": "config" } -] +] \ No newline at end of file diff --git a/packaging/flatpak/dev.crocoder.sce.yml b/packaging/flatpak/dev.crocoder.sce.yml index 9f8fe3ad..197cc5e7 100644 --- a/packaging/flatpak/dev.crocoder.sce.yml +++ b/packaging/flatpak/dev.crocoder.sce.yml @@ -1,38 +1,36 @@ +command: sce +finish-args: +- --share=network +- --filesystem=home +- --talk-name=org.freedesktop.Flatpak +- --talk-name=org.freedesktop.secrets id: dev.crocoder.sce +modules: +- build-commands: + - bash ./scripts/prepare-cli-generated-assets.sh "$PWD" + - cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce + - install -Dm755 cli/target/release/sce /app/bin/sce + - install -Dm755 packaging/flatpak/git-host-bridge /app/bin/git + - install -Dm644 packaging/flatpak/dev.crocoder.sce.metainfo.xml /app/share/metainfo/dev.crocoder.sce.metainfo.xml + build-options: + append-path: /usr/lib/sdk/rust-stable/bin + env: + CARGO_HOME: /run/build/sce/cargo + buildsystem: simple + name: sce + sources: + - commit: b7f0fa002fca5f5320791ff5e4abfaadfcddf187 + type: git + url: https://github.com/crocoder-dev/shared-context-engineering.git + - dest: packaging/flatpak + path: dev.crocoder.sce.metainfo.xml + type: file + - dest: packaging/flatpak + path: git-host-bridge + type: file + - cargo-sources.json runtime: org.freedesktop.Platform -runtime-version: "25.08" +runtime-version: '25.08' sdk: org.freedesktop.Sdk sdk-extensions: - - org.freedesktop.Sdk.Extension.rust-stable -command: sce - -finish-args: - - --share=network - - --filesystem=home - - --talk-name=org.freedesktop.Flatpak - - --talk-name=org.freedesktop.secrets - -modules: - - name: sce - buildsystem: simple - build-options: - append-path: /usr/lib/sdk/rust-stable/bin - env: - CARGO_HOME: /run/build/sce/cargo - build-commands: - - bash ./scripts/prepare-cli-generated-assets.sh "$PWD" - - cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce - - install -Dm755 cli/target/release/sce /app/bin/sce - - install -Dm755 packaging/flatpak/git-host-bridge /app/bin/git - - install -Dm644 packaging/flatpak/dev.crocoder.sce.metainfo.xml /app/share/metainfo/dev.crocoder.sce.metainfo.xml - sources: - - type: git - url: https://github.com/crocoder-dev/shared-context-engineering.git - commit: b7f0fa002fca5f5320791ff5e4abfaadfcddf187 - - type: file - path: dev.crocoder.sce.metainfo.xml - dest: packaging/flatpak - - type: file - path: git-host-bridge - dest: packaging/flatpak - - cargo-sources.json +- org.freedesktop.Sdk.Extension.rust-stable diff --git a/packaging/flatpak/sce-flatpak.sh b/packaging/flatpak/sce-flatpak.sh index de148289..23f7a425 100755 --- a/packaging/flatpak/sce-flatpak.sh +++ b/packaging/flatpak/sce-flatpak.sh @@ -14,7 +14,6 @@ Usage: sce-flatpak [options] Commands: validate Run lightweight Flatpak packaging validation prepare-local-manifest Generate a local-checkout Flatpak manifest - build Build the Flatpak from the local checkout release-package Package Flatpak source-manifest release assets release-bundle Build and bundle Flatpak release assets @@ -26,17 +25,6 @@ prepare-local-manifest options: --repo-root Repository checkout used as the Flatpak source --out-dir Directory for generated manifest/support files -build options: - --repo-root Repository checkout used as the Flatpak source - --build-dir flatpak-builder build directory - (default: ${TMPDIR:-/tmp}/sce-flatpak-build/dev.crocoder.sce) - --manifest-out Directory for generated local manifest/support files - --install Forward --install to flatpak-builder - --user Forward --user to flatpak-builder - --install-deps-from Forward --install-deps-from= to flatpak-builder - --no-force-clean Do not pass --force-clean to flatpak-builder - -- Extra arguments forwarded to flatpak-builder - release-package options: --version Release version to package; must match checked-in metadata --out-dir Directory for release tarball, checksum, and JSON metadata @@ -47,59 +35,28 @@ release-bundle options: --arch Target architecture (default: host arch via uname -m) --out-dir Directory for bundle, checksum, and JSON metadata --repo-root Repository checkout to build from (default: git root or cwd) - -The generated local manifest replaces the checked-in release git source with a -Flatpak type: dir source pointed at the checkout. It still runs the manifest's -Cargo source build inside Flatpak and does not consume a Nix-built sce binary. EOF } -die() { - printf 'sce-flatpak: %s\n' "$1" >&2 - exit 1 -} +die() { printf 'sce-flatpak: %s\n' "$1" >&2; exit 1; } resolve_repo_root() { local override="${1:-}" - if [ -n "${override}" ]; then [ -d "${override}" ] || die "--repo-root does not point to a directory: ${override}" - (cd "${override}" && pwd -P) - return + (cd "${override}" && pwd -P); return fi - local git_root git_root="$(git rev-parse --show-toplevel 2>/dev/null || true)" - if [ -n "${git_root}" ]; then - (cd "${git_root}" && pwd -P) - return - fi - - if [ -f "flake.nix" ] && [ -d "packaging/flatpak" ]; then - pwd -P - return - fi - + if [ -n "${git_root}" ]; then (cd "${git_root}" && pwd -P); return; fi + if [ -f "flake.nix" ] && [ -d "packaging/flatpak" ]; then pwd -P; return; fi die "could not resolve repository root; run from the repo or pass --repo-root" } -flatpak_dir_for() { - local repo_root="$1" - printf '%s/packaging/flatpak\n' "${repo_root}" -} - -require_file() { - local path="$1" - [ -f "${path}" ] || die "missing required file: ${path}" -} - +flatpak_dir_for() { printf '%s/packaging/flatpak\n' "$1"; } +require_file() { [ -f "$1" ] || die "missing required file: $1"; } require_command() { - local name="$1" - local guidance="$2" - - if ! command -v "${name}" >/dev/null 2>&1; then - die "${name} is required. ${guidance}" - fi + command -v "$1" >/dev/null 2>&1 || die "$1 is required. $2" } ensure_flatpak_user_remote() { @@ -107,327 +64,145 @@ ensure_flatpak_user_remote() { flatpak --user remote-add --if-not-exists --from "${FLATHUB_REMOTE_NAME}" "${FLATHUB_REMOTE_URL}" } -generate_local_manifest() { - local repo_root="$1" - local out_dir="$2" - local flatpak_dir - flatpak_dir="$(flatpak_dir_for "${repo_root}")" +substitute_placeholder() { + # Streams a template, replacing every occurrence of $1 with literal $2. + awk -v ph="$1" -v val="$2" \ + 'BEGIN { len = length(ph) } + { + out = ""; line = $0 + while ((i = index(line, ph)) > 0) { + out = out substr(line, 1, i - 1) val + line = substr(line, i + len) + } + print out line + }' "$3" +} - require_file "${flatpak_dir}/${MANIFEST_NAME}" +generate_local_manifest() { + local repo_root="$1" out_dir="$2" + local flatpak_dir; flatpak_dir="$(flatpak_dir_for "${repo_root}")" require_file "${flatpak_dir}/${METAINFO_NAME}" require_file "${flatpak_dir}/git-host-bridge" require_file "${flatpak_dir}/cargo-sources.json" - + [ -n "${SCE_FLATPAK_LOCAL_MANIFEST_TEMPLATE:-}" ] && [ -n "${SCE_FLATPAK_LOCAL_PATH_PLACEHOLDER:-}" ] \ + || die "generate_local_manifest: Nix-emitted local manifest template not available; run via 'nix run .#sce-flatpak'" mkdir -p "${out_dir}" cp "${flatpak_dir}/${METAINFO_NAME}" "${out_dir}/${METAINFO_NAME}" cp "${flatpak_dir}/git-host-bridge" "${out_dir}/git-host-bridge" cp "${flatpak_dir}/cargo-sources.json" "${out_dir}/cargo-sources.json" - - python3 - "${repo_root}" "${flatpak_dir}/${MANIFEST_NAME}" "${out_dir}/${MANIFEST_NAME}" <<'PY' -import json -import pathlib -import re -import sys - -repo_root = pathlib.Path(sys.argv[1]).resolve() -source_manifest = pathlib.Path(sys.argv[2]) -target_manifest = pathlib.Path(sys.argv[3]) - -text = source_manifest.read_text(encoding="utf-8") -release_source = re.compile( - r"(?m)^ - type: git\n" - r" url: https://github\.com/crocoder-dev/shared-context-engineering\.git\n" - r" commit: [0-9a-f]{40}\n" -) -local_source = f" - type: dir\n path: {json.dumps(str(repo_root))}\n" -text, count = release_source.subn(local_source, text, count=1) -if count != 1: - raise SystemExit("could not replace release git source with local dir source") - -target_manifest.write_text(text, encoding="utf-8") -PY - + local abs_repo_root; abs_repo_root="$(cd "${repo_root}" && pwd -P)" + substitute_placeholder "${SCE_FLATPAK_LOCAL_PATH_PLACEHOLDER}" "${abs_repo_root}" \ + "${SCE_FLATPAK_LOCAL_MANIFEST_TEMPLATE}" > "${out_dir}/${MANIFEST_NAME}" printf '%s/%s\n' "${out_dir}" "${MANIFEST_NAME}" } run_static_checks() { - local repo_root="$1" - local flatpak_dir - flatpak_dir="$(flatpak_dir_for "${repo_root}")" - - python3 - "${repo_root}" <<'PY' -import json -import pathlib -import re -import sys -import xml.etree.ElementTree as ET - -APP_ID = "dev.crocoder.sce" -repo_root = pathlib.Path(sys.argv[1]) -flatpak_dir = repo_root / "packaging" / "flatpak" -manifest_path = flatpak_dir / f"{APP_ID}.yml" -metainfo_path = flatpak_dir / f"{APP_ID}.metainfo.xml" -cargo_sources_path = flatpak_dir / "cargo-sources.json" - -errors = [] - -def require(condition, message): - if not condition: - errors.append(message) - -manifest = manifest_path.read_text(encoding="utf-8") -require("id: dev.crocoder.sce" in manifest, "manifest app ID is not dev.crocoder.sce") -require("command: sce" in manifest, "manifest command is not sce") -require("org.freedesktop.Sdk.Extension.rust-stable" in manifest, "Rust SDK extension is missing") -require("bash ./scripts/prepare-cli-generated-assets.sh \"$PWD\"" in manifest, "generated-asset preparation command is missing") -require("cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce" in manifest, "offline Cargo source-build command is missing") -require("install -Dm755 cli/target/release/sce /app/bin/sce" in manifest, "sce install command is missing") -require("install -Dm755 packaging/flatpak/git-host-bridge /app/bin/git" in manifest, "host git bridge install command is missing") -require("--talk-name=org.freedesktop.Flatpak" in manifest, "host Flatpak permission is missing") -require("cargo-sources.json" in manifest, "Cargo source descriptor is missing from manifest") - -release_source = re.compile( - r"(?m)^ - type: git\n" - r" url: https://github\.com/crocoder-dev/shared-context-engineering\.git\n" - r" commit: [0-9a-f]{40}\n" -) -require(release_source.search(manifest) is not None, "release manifest does not use a pinned git source") - -banned_snippets = [ - "nix build .#sce", - "nix build .#default", - "github.com/crocoder-dev/shared-context-engineering/releases", - "release-artifacts", - "@crocoder-dev/sce", -] -for path in sorted(flatpak_dir.iterdir()): - if not path.is_file(): - continue - if path.name == "sce-flatpak.sh": - continue - text = path.read_text(encoding="utf-8") - for snippet in banned_snippets: - require(snippet not in text, f"{path.relative_to(repo_root)} references disallowed artifact source: {snippet}") - -cargo_sources = json.loads(cargo_sources_path.read_text(encoding="utf-8")) -require(isinstance(cargo_sources, list) and cargo_sources, "cargo-sources.json is empty or not a list") -require(any(entry.get("type") == "git" and entry.get("url") == "https://github.com/tursodatabase/turso" for entry in cargo_sources if isinstance(entry, dict)), "Turso git source entry is missing") -require(any(entry.get("type") == "archive" and "static.crates.io" in entry.get("url", "") for entry in cargo_sources if isinstance(entry, dict)), "crates.io archive source entries are missing") - -root = ET.parse(metainfo_path).getroot() -require(root.findtext("id") == APP_ID, "metainfo ID is not dev.crocoder.sce") -provides = root.find("provides") -require(provides is not None and any(child.tag == "binary" and child.text == "sce" for child in list(provides)), "metainfo does not provide binary sce") - -if errors: - for error in errors: - print(f"Flatpak static validation failed: {error}", file=sys.stderr) - raise SystemExit(1) -PY + [ -n "${SCE_FLATPAK_STATIC_CHECK:-}" ] \ + || die "run_static_checks: Nix-built static validator not available; run via 'nix run .#sce-flatpak'" + "${SCE_FLATPAK_STATIC_CHECK}" --repo-root "$1" } validate_generated_local_manifest() { - local repo_root="$1" - local local_manifest="$2" - - python3 - "${repo_root}" "${local_manifest}" <<'PY' -import json -import pathlib -import sys - -repo_root = pathlib.Path(sys.argv[1]).resolve() -manifest_path = pathlib.Path(sys.argv[2]) -manifest = manifest_path.read_text(encoding="utf-8") -expected_path = f" path: {json.dumps(str(repo_root))}" - -errors = [] -if " - type: dir\n" not in manifest: - errors.append("local manifest does not contain a Flatpak type: dir source") -if expected_path not in manifest: - errors.append("local manifest does not point at the requested checkout path") -if "nix build .#sce" in manifest or "nix build .#default" in manifest: - errors.append("local manifest references a Nix-built sce binary") -if "cargo --offline build --release --manifest-path cli/Cargo.toml --bin sce" not in manifest: - errors.append("local manifest no longer runs the Flatpak Cargo source build") - -if errors: - for error in errors: - print(f"Flatpak local-manifest validation failed: {error}", file=sys.stderr) - raise SystemExit(1) -PY + [ -n "${SCE_FLATPAK_LOCAL_MANIFEST_CHECK:-}" ] \ + || die "validate_generated_local_manifest: Nix-built local-manifest validator not available; run via 'nix run .#sce-flatpak'" + "${SCE_FLATPAK_LOCAL_MANIFEST_CHECK}" --repo-root "$1" --manifest-path "$2" } resolve_release_commit() { - local repo_root="$1" - local release_commit - - if ! release_commit="$(git -C "${repo_root}" rev-parse --verify "HEAD^{commit}" 2>/dev/null)"; then - die "could not resolve release commit from repository checkout: ${repo_root}" - fi - - if [[ ! "${release_commit}" =~ ^[0-9a-f]{40}$ ]]; then - die "resolved release commit is not a full 40-character git SHA: ${release_commit}" - fi - + local repo_root="$1" release_commit + release_commit="$(git -C "${repo_root}" rev-parse --verify "HEAD^{commit}" 2>/dev/null)" \ + || die "could not resolve release commit from repository checkout: ${repo_root}" + [[ "${release_commit}" =~ ^[0-9a-f]{40}$ ]] \ + || die "resolved release commit is not a full 40-character git SHA: ${release_commit}" printf '%s\n' "${release_commit}" } validate_release_version_parity() { - local repo_root="$1" - local version="$2" - - python3 - "${repo_root}" "${version}" <<'PY' -import json -import pathlib -import re -import sys -import xml.etree.ElementTree as ET - -repo_root = pathlib.Path(sys.argv[1]) -version = sys.argv[2] -app_id = "dev.crocoder.sce" -errors = [] - -def require(condition, message): - if not condition: - errors.append(message) - -version_path = repo_root / ".version" -cargo_toml_path = repo_root / "cli" / "Cargo.toml" -npm_package_path = repo_root / "npm" / "package.json" -metainfo_path = repo_root / "packaging" / "flatpak" / f"{app_id}.metainfo.xml" - -try: - checked_in_version = version_path.read_text(encoding="utf-8").strip() -except OSError as error: - raise SystemExit(f"could not read {version_path}: {error}") from error - -try: - cargo_toml = cargo_toml_path.read_text(encoding="utf-8") -except OSError as error: - raise SystemExit(f"could not read {cargo_toml_path}: {error}") from error - -cargo_match = re.search(r'(?m)^version = "([^"]+)"$', cargo_toml) -require(cargo_match is not None, "cli/Cargo.toml package version is missing") -cargo_version = cargo_match.group(1) if cargo_match else "" - -try: - npm_version = json.loads(npm_package_path.read_text(encoding="utf-8")).get("version", "") -except (OSError, json.JSONDecodeError) as error: - raise SystemExit(f"could not parse {npm_package_path}: {error}") from error - -try: - metainfo_root = ET.parse(metainfo_path).getroot() -except (OSError, ET.ParseError) as error: - raise SystemExit(f"could not parse {metainfo_path}: {error}") from error - -release_versions = [ - release.attrib.get("version", "") - for release in metainfo_root.findall("./releases/release") -] -require(bool(release_versions), "Flatpak metainfo release metadata is missing") -flatpak_version = release_versions[0] if release_versions else "" - -require(version == checked_in_version, f"requested release version {version} does not match .version {checked_in_version}") -require(version == cargo_version, f"cli/Cargo.toml version {cargo_version} does not match release version {version}") -require(version == npm_version, f"npm/package.json version {npm_version} does not match release version {version}") -require(version == flatpak_version, f"Flatpak metainfo release version {flatpak_version} does not match release version {version}") - -if errors: - for error in errors: - print(f"Flatpak release version validation failed: {error}", file=sys.stderr) - raise SystemExit(1) -PY + [ -n "${SCE_FLATPAK_VERSION_PARITY_CHECK:-}" ] \ + || die "validate_release_version_parity: Nix-built version-parity validator not available; run via 'nix run .#sce-flatpak'" + "${SCE_FLATPAK_VERSION_PARITY_CHECK}" --repo-root "$1" --version "$2" } generate_release_manifest() { - local repo_root="$1" - local release_commit="$2" - local out_dir="$3" - local flatpak_dir - flatpak_dir="$(flatpak_dir_for "${repo_root}")" - - require_file "${flatpak_dir}/${MANIFEST_NAME}" - + local release_commit="$1" out_dir="$2" + [[ "${release_commit}" =~ ^[0-9a-f]{40}$ ]] \ + || die "release commit must be a full 40-character lowercase git SHA" + [ -n "${SCE_FLATPAK_COMMIT_MANIFEST_TEMPLATE:-}" ] && [ -n "${SCE_FLATPAK_COMMIT_PLACEHOLDER:-}" ] \ + || die "generate_release_manifest: Nix-emitted commit manifest template not available; run via 'nix run .#sce-flatpak'" mkdir -p "${out_dir}" + substitute_placeholder "${SCE_FLATPAK_COMMIT_PLACEHOLDER}" "${release_commit}" \ + "${SCE_FLATPAK_COMMIT_MANIFEST_TEMPLATE}" > "${out_dir}/${MANIFEST_NAME}" + printf '%s/%s\n' "${out_dir}" "${MANIFEST_NAME}" +} - python3 - "${release_commit}" "${flatpak_dir}/${MANIFEST_NAME}" "${out_dir}/${MANIFEST_NAME}" <<'PY' -import pathlib -import re -import sys - -release_commit = sys.argv[1] -source_manifest = pathlib.Path(sys.argv[2]) -target_manifest = pathlib.Path(sys.argv[3]) - -if not re.fullmatch(r"[0-9a-f]{40}", release_commit): - raise SystemExit("release commit must be a full 40-character lowercase git SHA") - -text = source_manifest.read_text(encoding="utf-8") -release_source = re.compile( - r"(?m)^( - type: git\n" - r" url: https://github\.com/crocoder-dev/shared-context-engineering\.git\n" - r" commit: )[0-9a-f]{40}(\n)" -) -text, count = release_source.subn(rf"\g<1>{release_commit}\2", text, count=1) -if count != 1: - raise SystemExit("could not rewrite release git source commit in staged manifest") - -target_manifest.write_text(text, encoding="utf-8") -PY +# Constrained inputs (semver, hex SHA, fixed filenames, x86_64/aarch64) — no JSON escaping needed. +emit_source_manifest_metadata() { + local path="$1" version="$2" release_commit="$3" package_name="$4" checksum_name="$5" checksum="$6" + cat > "${path}" < "${path}" <&2; exit 1; } - if [ -z "${version}" ] || [ -z "${out_dir}" ]; then - usage >&2 - exit 1 - fi - - local repo_root - repo_root="$(resolve_repo_root "${repo_root_override}")" - local flatpak_dir - flatpak_dir="$(flatpak_dir_for "${repo_root}")" - + local repo_root; repo_root="$(resolve_repo_root "${repo_root_override}")" + local flatpak_dir; flatpak_dir="$(flatpak_dir_for "${repo_root}")" require_file "${flatpak_dir}/${MANIFEST_NAME}" require_file "${flatpak_dir}/${METAINFO_NAME}" require_file "${flatpak_dir}/git-host-bridge" @@ -628,19 +260,11 @@ cmd_release_package() { validate_release_version_parity "${repo_root}" "${version}" run_static_checks "${repo_root}" - local release_commit - release_commit="$(resolve_release_commit "${repo_root}")" + local release_commit; release_commit="$(resolve_release_commit "${repo_root}")" mkdir -p "${out_dir}" - - local tmp_dir - tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/sce-flatpak-release.XXXXXX")" - cleanup() { - if [ -n "${tmp_dir:-}" ]; then - rm -rf "${tmp_dir}" - fi - } - trap cleanup EXIT + local tmp_dir; tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/sce-flatpak-release.XXXXXX")" + trap '[ -n "${tmp_dir:-}" ] && rm -rf "${tmp_dir}"' EXIT local package_root="sce-v${version}-flatpak-manifest" local package_name="${package_root}.tar.gz" @@ -652,124 +276,44 @@ cmd_release_package() { local metadata_path="${out_dir}/${metadata_name}" mkdir -p "${stage_dir}" - generate_release_manifest "${repo_root}" "${release_commit}" "${stage_dir}" >/dev/null + generate_release_manifest "${release_commit}" "${stage_dir}" >/dev/null cp "${flatpak_dir}/${METAINFO_NAME}" "${stage_dir}/${METAINFO_NAME}" cp "${flatpak_dir}/cargo-sources.json" "${stage_dir}/cargo-sources.json" cp "${flatpak_dir}/git-host-bridge" "${stage_dir}/git-host-bridge" - chmod 0644 "${stage_dir}/${MANIFEST_NAME}" "${stage_dir}/${METAINFO_NAME}" "${stage_dir}/cargo-sources.json" chmod 0755 "${stage_dir}/git-host-bridge" - tar \ - --sort=name \ - --mtime='UTC 1970-01-01' \ - --owner=0 \ - --group=0 \ - --numeric-owner \ - -C "${tmp_dir}" \ - -cf - "${package_root}" | gzip -n > "${package_path}" - - local checksum - checksum="$(sha256sum "${package_path}" | cut -d ' ' -f 1)" - printf '%s %s\n' "${checksum}" "${package_name}" > "${checksum_path}" + tar --sort=name --mtime='UTC 1970-01-01' --owner=0 --group=0 --numeric-owner \ + -C "${tmp_dir}" -cf - "${package_root}" | gzip -n > "${package_path}" - python3 - \ - "${metadata_path}" \ - "${version}" \ - "${release_commit}" \ - "${package_name}" \ - "${checksum_name}" \ - "${checksum}" \ - <<'PY' -import json -import pathlib -import sys - -metadata_path = pathlib.Path(sys.argv[1]) -version = sys.argv[2] -release_commit = sys.argv[3] -package_name = sys.argv[4] -checksum_name = sys.argv[5] -checksum = sys.argv[6] - -manifest_name = "dev.crocoder.sce.yml" -support_files = [ - "dev.crocoder.sce.metainfo.xml", - "cargo-sources.json", - "git-host-bridge", -] -metadata = { - "asset_type": "flatpak-source-manifest", - "app_id": "dev.crocoder.sce", - "version": version, - "release_commit": release_commit, - "manifest_name": manifest_name, - "package_file": package_name, - "checksum_file": checksum_name, - "checksum_sha256": checksum, - "packaged_support_files": support_files, - "packaged_files": [manifest_name, *support_files], -} -metadata_path.write_text(json.dumps(metadata, indent=2) + "\n", encoding="utf-8") -PY + local checksum; checksum="$(sha256sum "${package_path}" | cut -d ' ' -f 1)" + printf '%s %s\n' "${checksum}" "${package_name}" > "${checksum_path}" - rm -rf "${tmp_dir}" - trap - EXIT + emit_source_manifest_metadata \ + "${metadata_path}" "${version}" "${release_commit}" \ + "${package_name}" "${checksum_name}" "${checksum}" + rm -rf "${tmp_dir}"; trap - EXIT printf 'Built Flatpak source-manifest release assets:\n' - printf ' %s\n' "${package_path}" - printf ' %s\n' "${checksum_path}" - printf ' %s\n' "${metadata_path}" + printf ' %s\n' "${package_path}" "${checksum_path}" "${metadata_path}" } cmd_release_bundle() { - local repo_root_override="" - local version="" - local arch="" - local out_dir="" - + local repo_root_override="" version="" arch="" out_dir="" while [ $# -gt 0 ]; do case "$1" in - --repo-root) - repo_root_override="${2:-}" - [ -n "${repo_root_override}" ] || die "--repo-root requires a path" - shift 2 - ;; - --version) - version="${2:-}" - [ -n "${version}" ] || die "--version requires a semver value" - shift 2 - ;; - --arch) - arch="${2:-}" - [ -n "${arch}" ] || die "--arch requires an architecture value" - shift 2 - ;; - --out-dir) - out_dir="${2:-}" - [ -n "${out_dir}" ] || die "--out-dir requires a path" - shift 2 - ;; - --help|-h) - usage - exit 0 - ;; - *) - die "unknown release-bundle argument: $1" - ;; + --repo-root) repo_root_override="${2:-}"; [ -n "${repo_root_override}" ] || die "--repo-root requires a path"; shift 2 ;; + --version) version="${2:-}"; [ -n "${version}" ] || die "--version requires a semver value"; shift 2 ;; + --arch) arch="${2:-}"; [ -n "${arch}" ] || die "--arch requires an architecture value"; shift 2 ;; + --out-dir) out_dir="${2:-}"; [ -n "${out_dir}" ] || die "--out-dir requires a path"; shift 2 ;; + --help|-h) usage; exit 0 ;; + *) die "unknown release-bundle argument: $1" ;; esac done + [ -n "${version}" ] && [ -n "${out_dir}" ] || { usage >&2; exit 1; } - if [ -z "${version}" ] || [ -z "${out_dir}" ]; then - usage >&2 - exit 1 - fi - - local repo_root - repo_root="$(resolve_repo_root "${repo_root_override}")" - local flatpak_dir - flatpak_dir="$(flatpak_dir_for "${repo_root}")" - + local repo_root; repo_root="$(resolve_repo_root "${repo_root_override}")" + local flatpak_dir; flatpak_dir="$(flatpak_dir_for "${repo_root}")" require_file "${flatpak_dir}/${MANIFEST_NAME}" require_file "${flatpak_dir}/${METAINFO_NAME}" require_file "${flatpak_dir}/git-host-bridge" @@ -778,38 +322,21 @@ cmd_release_bundle() { validate_release_version_parity "${repo_root}" "${version}" run_static_checks "${repo_root}" - # Resolve architecture (default to host arch) - if [ -z "${arch}" ]; then - arch="$(uname -m)" - fi - - # Validate arch is supported + [ -n "${arch}" ] || arch="$(uname -m)" case "${arch}" in - x86_64|aarch64) - ;; - *) - die "unsupported architecture: ${arch} (supported: x86_64, aarch64)" - ;; + x86_64|aarch64) ;; + *) die "unsupported architecture: ${arch} (supported: x86_64, aarch64)" ;; esac - require_command "flatpak-builder" "Use 'nix run .#flatpak-build' or enter 'nix develop'." - require_command "flatpak" "Install flatpak or enter 'nix develop'." + require_command "flatpak-builder" "Enter 'nix develop' or install flatpak-builder." + require_command "flatpak" "Enter 'nix develop' or install flatpak." ensure_flatpak_user_remote mkdir -p "${out_dir}" + local tmp_dir; tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/sce-flatpak-release-bundle.XXXXXX")" + trap '[ -n "${tmp_dir:-}" ] && rm -rf "${tmp_dir}"' EXIT - local tmp_dir - tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/sce-flatpak-release-bundle.XXXXXX")" - cleanup() { - if [ -n "${tmp_dir:-}" ]; then - rm -rf "${tmp_dir}" - fi - } - trap cleanup EXIT - - # Generate local manifest into temp staging - local local_manifest - local_manifest="$(generate_local_manifest "${repo_root}" "${tmp_dir}/manifest")" + local local_manifest; local_manifest="$(generate_local_manifest "${repo_root}" "${tmp_dir}/manifest")" validate_generated_local_manifest "${repo_root}" "${local_manifest}" local build_dir="${tmp_dir}/build" @@ -821,97 +348,38 @@ cmd_release_bundle() { local metadata_name="sce-v${version}-${arch}.json" local metadata_path="${out_dir}/${metadata_name}" - # Build Flatpak from source (no --install) printf 'Building %s for %s from local checkout source: %s\n' "${APP_ID}" "${arch}" "${repo_root}" - flatpak-builder \ - --force-clean \ - --user \ + flatpak-builder --force-clean --user \ --install-deps-from="${FLATHUB_REMOTE_NAME}" \ - --repo="${export_repo}" \ - --arch="${arch}" \ - "${build_dir}" \ - "${local_manifest}" + --repo="${export_repo}" --arch="${arch}" \ + "${build_dir}" "${local_manifest}" - # Create bundle from the build repository printf 'Creating Flatpak bundle: %s\n' "${bundle_path}" flatpak build-bundle --arch="${arch}" "${export_repo}" "${bundle_path}" "${APP_ID}" - # Compute SHA-256 checksum - local checksum - checksum="$(sha256sum "${bundle_path}" | cut -d ' ' -f 1)" + local checksum; checksum="$(sha256sum "${bundle_path}" | cut -d ' ' -f 1)" printf '%s %s\n' "${checksum}" "${bundle_name}" > "${checksum_path}" - # Generate JSON metadata - python3 - \ - "${metadata_path}" \ - "${version}" \ - "${arch}" \ - "${bundle_name}" \ - "${checksum_name}" \ - "${checksum}" \ - <<'PY' -import json -import pathlib -import sys - -metadata_path = pathlib.Path(sys.argv[1]) -version = sys.argv[2] -arch = sys.argv[3] -bundle_name = sys.argv[4] -checksum_name = sys.argv[5] -checksum = sys.argv[6] - -metadata = { - "asset_type": "flatpak-bundle", - "app_id": "dev.crocoder.sce", - "version": version, - "architecture": arch, - "bundle_file": bundle_name, - "checksum_file": checksum_name, - "checksum_sha256": checksum, -} -metadata_path.write_text(json.dumps(metadata, indent=2) + "\n", encoding="utf-8") -PY - - rm -rf "${tmp_dir}" - trap - EXIT + emit_bundle_metadata \ + "${metadata_path}" "${version}" "${arch}" \ + "${bundle_name}" "${checksum_name}" "${checksum}" + rm -rf "${tmp_dir}"; trap - EXIT printf 'Built Flatpak bundle release assets:\n' - printf ' %s\n' "${bundle_path}" - printf ' %s\n' "${checksum_path}" - printf ' %s\n' "${metadata_path}" + printf ' %s\n' "${bundle_path}" "${checksum_path}" "${metadata_path}" } main() { local command="${1:-}" - if [ -z "${command}" ]; then - usage - exit 1 - fi + [ -n "${command}" ] || { usage; exit 1; } shift - case "${command}" in - validate) - cmd_validate "$@" - ;; - prepare-local-manifest) - cmd_prepare_local_manifest "$@" - ;; - build) - cmd_build "$@" - ;; - release-package) - cmd_release_package "$@" - ;; - release-bundle) - cmd_release_bundle "$@" - ;; - --help|-h|help) - usage - ;; - *) - die "unknown command: ${command}" - ;; + validate) cmd_validate "$@" ;; + prepare-local-manifest) cmd_prepare_local_manifest "$@" ;; + release-package) cmd_release_package "$@" ;; + release-bundle) cmd_release_bundle "$@" ;; + --help|-h|help) usage ;; + *) die "unknown command: ${command}" ;; esac } From 183c5e6de8ac384b4e8229a7defb07cdd5e78369 Mon Sep 17 00:00:00 2001 From: David Abram Date: Wed, 24 Jun 2026 00:55:05 +0200 Subject: [PATCH 2/4] release: Add manual GitHub prerelease option Add a workflow_dispatch boolean input that passes through to the GitHub Release prerelease flag while preserving tag-triggered release behavior and existing version/artifact semantics. Co-authored-by: SCE --- .github/workflows/release-sce.yml | 6 ++ context/architecture.md | 2 +- context/context-map.md | 4 +- .../plans/github-release-prerelease-option.md | 60 +++++++++++++++++++ .../cli-first-install-channels-contract.md | 2 +- context/sce/cli-release-artifact-contract.md | 2 +- 6 files changed, 71 insertions(+), 5 deletions(-) create mode 100644 context/plans/github-release-prerelease-option.md diff --git a/.github/workflows/release-sce.yml b/.github/workflows/release-sce.yml index 6e319e8a..097cd764 100644 --- a/.github/workflows/release-sce.yml +++ b/.github/workflows/release-sce.yml @@ -10,6 +10,11 @@ on: description: "Branch or commit SHA to tag" required: false default: main + prerelease: + description: "Mark the GitHub Release as a pre-release" + required: false + type: boolean + default: false permissions: contents: write @@ -222,6 +227,7 @@ jobs: with: tag_name: ${{ needs.resolve-release.outputs.tag }} name: sce ${{ needs.resolve-release.outputs.tag }} + prerelease: ${{ github.event_name == 'workflow_dispatch' && inputs.prerelease || false }} generate_release_notes: true body: | ## CLI release assets diff --git a/context/architecture.md b/context/architecture.md index e819b8b1..c0d32a0f 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -90,7 +90,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - Cargo/crates.io and npm registry publication belong to separate downstream publish stages that consume already-versioned checked-in package metadata rather than inventing release versions during workflow execution. - npm is a downstream consumer of the shared release artifact contract rather than a separate build owner. - `flake.nix` now exposes dedicated release-packaging apps for this rollout: `apps.release-artifacts` packages the current-platform `sce` archive/checksum/manifest-fragment set from `packages.default` and fails if `.version`, `cli/Cargo.toml`, `npm/package.json`, or the built CLI version disagree; `apps.release-manifest` merges per-platform metadata fragments into release-level manifest/checksum outputs and signs the merged manifest with a non-repo private key provided via env/file input; `apps.release-npm-package` packs the checked-in npm launcher package into a release-ready `sce-v-npm.tgz` tarball plus npm metadata JSON while refusing mismatched checked-in version metadata; and Linux-only `apps.release-flatpak-package` emits the Flatpak source-manifest tarball/checksum/JSON metadata while refusing `.version`/Cargo/npm/AppStream version drift and non-git checkouts that cannot resolve a release commit; Linux-only `apps.release-flatpak-bundle` builds a source-built `.flatpak` bundle from the checkout and emits per-architecture bundle/checksum/JSON metadata. -- CLI release automation: `.github/workflows/release-sce.yml` orchestrates `sce` GitHub release asset assembly, validates `.version`/Cargo/npm parity before tagging or releasing, calls three reusable per-platform workflow files (`release-sce-linux.yml`, `release-sce-linux-arm.yml`, `release-sce-macos-arm.yml`), assembles the signed native release manifest, builds npm and Flatpak source-manifest release packages, and uploads the CLI/npm/Flatpak asset set to the GitHub Release. +- CLI release automation: `.github/workflows/release-sce.yml` orchestrates `sce` GitHub release asset assembly, validates `.version`/Cargo/npm parity before tagging or releasing, calls three reusable per-platform workflow files (`release-sce-linux.yml`, `release-sce-linux-arm.yml`, `release-sce-macos-arm.yml`), assembles the signed native release manifest, builds npm and Flatpak source-manifest release packages, and uploads the CLI/npm/Flatpak asset set to the GitHub Release. Manual `workflow_dispatch` runs can set the GitHub Release-level `prerelease` flag through an explicit checkbox; tag-triggered releases keep the flag false and no tag-name prerelease inference is performed. - Downstream registry publication is now implemented in dedicated workflows: `.github/workflows/publish-crates.yml` publishes the checked-in crate version from a published release or manual dispatch after `.version`/tag/Cargo parity checks, and `.github/workflows/publish-npm.yml` publishes the checked-in npm package version from a published release or manual dispatch after `.version`/tag/npm parity checks plus verification of the canonical `sce-v-npm.tgz` GitHub release asset. - The npm distribution implementation lives under `npm/`: `package.json` defines the `sce` package surface, `bin/sce.js` launches the package-local native binary, `lib/install.js` resolves the current package version against the release manifest, verifies `sce-v-release-manifest.json.sig` with the bundled public key before trusting manifest contents, and then installs the checksum-verified native archive for supported macOS/Linux targets, while `test/platform.test.js` and `test/install.test.js` cover platform selection plus signed-manifest installer behavior. diff --git a/context/context-map.md b/context/context-map.md index 5e0d65d4..93ab0815 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -55,9 +55,9 @@ Feature/domain context: - `context/sce/bash-tool-policy-enforcement-contract.md` (approved bash-tool blocking contract plus current Rust evaluator seam and OpenCode/Claude delegation references, including config schema, argv-prefix matching, shell/nix unwrapping, fixed preset catalog/messages, and precedence rules) - `context/sce/generated-opencode-plugin-registration.md` (current generated OpenCode plugin-registration contract, canonical Pkl ownership, generated manifest/plugin paths including `sce-bash-policy` + `sce-agent-trace`, TypeScript source ownership, and Claude generated settings boundary including Agent Trace hooks plus `PreToolUse` Bash policy hook registration) - `context/sce/opencode-agent-trace-plugin-runtime.md` (current OpenCode agent-trace plugin runtime behavior, including captured `message.updated` handoff with `summary.diffs` branching: when diffs exist sends `-patch` variant `message.updated` + per-diff `message.part.updated` with `part_type: "patch"` concurrently via `Promise.all`, when no diffs sends original `message.updated` payload; in-memory dedup `Set` keyed by `"${sessionID}:${messageID}"`; captured `message.part.updated` handoff to `sce hooks conversation-trace` as `{ type: "message.part.updated", payloads: [{ session_id, message_id, part_type, text, generated_at_unix_ms }] }` with `text`/`reasoning` only; existing user-message diff extraction for `{ sessionID, diff, time, model_id }`; session-scoped OpenCode client version capture from `session.created`/`session.updated`; and CLI handoff to `sce hooks diff-trace` over STDIN JSON with required `tool_name="opencode"` plus required nullable `tool_version`; Rust hook parsing and AgentTraceDb insertion persist required payload fields including `model_id`) -- `context/sce/cli-first-install-channels-contract.md` (current `sce` install/distribution contract covering supported Nix/Cargo/npm plus source-built Flatpak channel scope, implemented GitHub Release Flatpak source-manifest and source-built `.flatpak` bundle asset packaging, canonical `.version` release authority, Nix-owned build/release policy, Nix-owned Flatpak manifest generation via `pkgs.formats.yaml.generate` plus Nix-owned cargo-sources generation from `cli/Cargo.lock` plus dedicated Nix-built Bash validator scripts for static, version-parity, and local-manifest validation, thin imperative `packaging/flatpak/sce-flatpak.sh` orchestration around `flatpak-builder` / `flatpak build-bundle`, the reduced Linux flake app surface (`sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`, plus `regenerate-flatpak-manifest` / `regenerate-cargo-sources` helpers) with `flatpak-static-validation` / `flatpak-manifest-parity` / `cargo-sources-parity` checks, the implemented `packaging/flatpak/dev.crocoder.sce.yml` + AppStream/Cargo-source packaging surface as generated artifacts, and the `dev.crocoder.sce` host-git bridge decision) +- `context/sce/cli-first-install-channels-contract.md` (current `sce` install/distribution contract covering supported Nix/Cargo/npm plus source-built Flatpak channel scope, implemented GitHub Release Flatpak source-manifest and source-built `.flatpak` bundle asset packaging, canonical `.version` release authority, manual GitHub Release `prerelease` checkbox behavior, Nix-owned build/release policy, Nix-owned Flatpak manifest generation via `pkgs.formats.yaml.generate` plus Nix-owned cargo-sources generation from `cli/Cargo.lock` plus dedicated Nix-built Bash validator scripts for static, version-parity, and local-manifest validation, thin imperative `packaging/flatpak/sce-flatpak.sh` orchestration around `flatpak-builder` / `flatpak build-bundle`, the reduced Linux flake app surface (`sce-flatpak`, `release-flatpak-package`, `release-flatpak-bundle`, plus `regenerate-flatpak-manifest` / `regenerate-cargo-sources` helpers) with `flatpak-static-validation` / `flatpak-manifest-parity` / `cargo-sources-parity` checks, the implemented `packaging/flatpak/dev.crocoder.sce.yml` + AppStream/Cargo-source packaging surface as generated artifacts, and the `dev.crocoder.sce` host-git bridge decision) - `context/sce/optional-install-channel-integration-test-entrypoint.md` (current opt-in flake app contract for existing binary install-channel integration coverage, including thin flake delegation to the Rust runner, shared harness ownership, real npm+Bun+Cargo install flows, channel selector semantics, the explicit non-default execution boundary, and the separate Flatpak source-build validation boundary) -- `context/sce/cli-release-artifact-contract.md` (shared `sce` binary release artifact naming, checksum/manifest outputs, GitHub Releases as the canonical artifact publication surface, the current three-target Linux/macOS release workflow topology, implemented Flatpak source-manifest and source-built `.flatpak` bundle package assets uploaded by `.github/workflows/release-sce.yml`, and Flatpak's explicit source-built non-binary exception) +- `context/sce/cli-release-artifact-contract.md` (shared `sce` binary release artifact naming, checksum/manifest outputs, GitHub Releases as the canonical artifact publication surface, manual dispatch `prerelease` flag behavior, the current three-target Linux/macOS release workflow topology, implemented Flatpak source-manifest and source-built `.flatpak` bundle package assets uploaded by `.github/workflows/release-sce.yml`, and Flatpak's explicit source-built non-binary exception) - `context/sce/cli-npm-distribution-contract.md` (implemented `sce` npm launcher package, release-manifest/checksum-verified native binary install flow, the supported darwin/arm64 plus linux x64+arm64 npm platform matrix, and dedicated `.github/workflows/publish-npm.yml` downstream npm publish-stage contract) - `context/sce/cli-cargo-distribution-contract.md` (implemented `sce` Cargo publication posture plus supported crates.io, git, and local checkout install guidance, dedicated crates.io publish workflow, and ephemeral crate-local generated-asset mirror requirement for published builds) diff --git a/context/plans/github-release-prerelease-option.md b/context/plans/github-release-prerelease-option.md new file mode 100644 index 00000000..8840f4c0 --- /dev/null +++ b/context/plans/github-release-prerelease-option.md @@ -0,0 +1,60 @@ +# GitHub Release Pre-release Option + +## Change summary + +Add a manual GitHub Actions `workflow_dispatch` checkbox/input to the `sce` release orchestrator so maintainers can mark a manually-created GitHub Release as a GitHub pre-release. The release command remains thin orchestration: the flag is passed to the GitHub Release creation step as the release-level `prerelease` boolean only. + +The requested behavior does not change tag naming, release version validation, generated release notes, title, body copy, artifact names, downstream Cargo/npm publication workflows, or tag-triggered release behavior. + +## Success criteria + +- Manual runs of `.github/workflows/release-sce.yml` expose a boolean pre-release checkbox/input. +- The GitHub Release creation step passes that manual input to `softprops/action-gh-release` as the release-level `prerelease` value. +- Push/tag-triggered releases preserve current behavior and are not automatically marked pre-release based on tag names such as `-pre`, `-alpha`, `-beta`, or `-rc`. +- Release title/body/notes remain unchanged except for the GitHub Release `prerelease` flag. +- The workflow remains valid under the repository's existing workflow validation surface. +- Current release context documents are updated only where they describe the release orchestrator contract. + +## Constraints and non-goals + +- In scope: `.github/workflows/release-sce.yml` manual dispatch input and `softprops/action-gh-release` `prerelease` wiring. +- In scope: targeted context sync for release-orchestrator behavior. +- Out of scope: a new `sce-release` CLI command, tag naming changes, semver prerelease parsing, automatic prerelease inference, release-note/body/title changes, GitHub CLI usage, GitHub API code, and registry publish-stage changes. +- Out of scope: changing native, npm, or Flatpak release asset names or artifact assembly logic. +- Constraint: preserve deterministic `.version`/tag/Cargo/npm validation behavior. +- Constraint: keep task slices suitable for one atomic commit each. + +## Task stack + +- [x] T01: `Add manual GitHub Release prerelease workflow input` (status:done) + - Task ID: T01 + - Goal: Add a boolean `workflow_dispatch` input to `.github/workflows/release-sce.yml` and wire it into the `Create GitHub release` action as the `prerelease` value. + - Boundaries (in/out of scope): In - release orchestrator workflow input definition, expression plumbing for manual runs, default false behavior for tag pushes, `softprops/action-gh-release` `prerelease` input. Out - release title/body/notes edits, tag/version parsing changes, reusable platform workflow changes, publish workflows, CLI code. + - Done when: Manual dispatch includes a pre-release checkbox/input; manual runs with the checkbox selected create/update the GitHub Release with `prerelease: true`; manual runs without it and tag-triggered runs keep `prerelease: false`; release artifact assembly and validation gates remain unchanged. + - Verification notes (commands or checks): Inspect `.github/workflows/release-sce.yml` for valid `workflow_dispatch.inputs..type: boolean`, default false, and `softprops/action-gh-release` `prerelease: ${{ ... }}` wiring; run `nix flake check` if implementation scope allows full workflow/actionlint validation. + - Completed: 2026-06-24 + - Files changed: `.github/workflows/release-sce.yml` + - Evidence: `nix run nixpkgs#actionlint -- .github/workflows/release-sce.yml` passed; `nix build .#checks.x86_64-linux.workflow-actionlint` passed; `nix flake check` passed. + - Notes: Added manual-only boolean `prerelease` input with default `false` and wired it to `softprops/action-gh-release`; non-dispatch tag releases resolve the release flag to `false`. + +- [x] T02: `Sync release context for prerelease option` (status:done) + - Task ID: T02 + - Goal: Update current-state release context to document that manual release dispatch can mark the GitHub Release as a pre-release flag without changing tag/version semantics. + - Boundaries (in/out of scope): In - focused updates to release-related context such as `context/sce/cli-release-artifact-contract.md`, `context/sce/cli-first-install-channels-contract.md`, and context-map/glossary entries only if needed. Out - broad rewrite of release docs, completed-plan summaries, unrelated install-channel changes. + - Done when: Context accurately states the manual pre-release flag behavior, preserves the distinction that GitHub pre-release is a release flag rather than a tag property, and does not imply auto-inference from tag names. + - Verification notes (commands or checks): Read changed context files for current-state wording; check that context references align with `.github/workflows/release-sce.yml`; run `nix run .#pkl-check-generated` if generated/context-adjacent changes are touched by the implementation session. + - Completed: 2026-06-24 + - Files changed: `context/architecture.md`, `context/sce/cli-release-artifact-contract.md`, `context/sce/cli-first-install-channels-contract.md`, `context/context-map.md` + - Evidence: Reviewed `.github/workflows/release-sce.yml` lines 13-17 and 225-230 against release context; `nix run .#pkl-check-generated` passed. + - Notes: Durable context states the manual checkbox controls only the GitHub Release-level `prerelease` flag; tag/version semantics, generated notes/title/body, and artifact naming remain unchanged with no tag-name inference. + +- [ ] T03: `Validate release workflow and cleanup` (status:todo) + - Task ID: T03 + - Goal: Run final validation for the completed plan and remove any temporary planning or implementation artifacts. + - Boundaries (in/out of scope): In - repository validation, workflow/actionlint coverage through the existing flake checks, generated-output parity, plan checkbox/status updates, cleanup of temporary files. Out - new feature work beyond the prerelease flag and context sync. + - Done when: Full required validation passes or any failures are captured with actionable notes; temporary files are removed; the plan reflects completed tasks; context sync has been verified. + - Verification notes (commands or checks): `nix run .#pkl-check-generated`; `nix flake check`; review `git diff` to confirm only intended workflow/context/plan changes remain. + +## Open questions + +None. The clarification gate resolved that this is a new manual checkbox only, and that the implementation should only set the GitHub Release `prerelease` true/false flag. diff --git a/context/sce/cli-first-install-channels-contract.md b/context/sce/cli-first-install-channels-contract.md index 5dc6d73b..002fcde5 100644 --- a/context/sce/cli-first-install-channels-contract.md +++ b/context/sce/cli-first-install-channels-contract.md @@ -25,7 +25,7 @@ No other install channels are in scope for the current implementation stage. - Nix-managed build/release entrypoints remain the required build source for existing binary release automation. - Nix may also orchestrate Flatpak tooling, local source overrides, validation, and local builds, but the Flatpak package itself must build `sce` from source inside Flatpak. - Repo-root `.version` is the canonical checked-in release version authority for the Nix, Cargo, npm, and release-artifact surfaces. -- GitHub Releases are the canonical publication surface for release archives, native release manifest/checksum assets, npm package assets, and approved Flatpak source-manifest package assets produced for the version declared in `.version`. +- GitHub Releases are the canonical publication surface for release archives, native release manifest/checksum assets, npm package assets, and approved Flatpak source-manifest package assets produced for the version declared in `.version`. Manual release dispatch may set the GitHub Release-level `prerelease` flag explicitly, but tag/version semantics and asset names still come from `.version`. - `npm` consumes release artifacts produced by Nix-managed build/release flows. - `Cargo` is a first-class supported install path and its publish metadata should stay aligned to `.version` without workflow-side version bumping. - npm registry publication should also consume the checked-in package version aligned to `.version` without workflow-side version bumping. diff --git a/context/sce/cli-release-artifact-contract.md b/context/sce/cli-release-artifact-contract.md index e5a0bced..fe94cc29 100644 --- a/context/sce/cli-release-artifact-contract.md +++ b/context/sce/cli-release-artifact-contract.md @@ -65,7 +65,7 @@ This file captures the current shared release artifact foundation plus the appro - `.github/workflows/release-sce.yml` remains the CLI release orchestrator that assembles GitHub release assets from the reusable platform build workflows. - The release orchestrator injects the non-repo manifest-signing private key through the `SCE_RELEASE_MANIFEST_SIGNING_KEY` secret when assembling release-level metadata. - The release orchestrator also runs `nix run .#release-flatpak-package -- --version --out-dir dist/flatpak` and uploads `dist/flatpak/*.tar.gz`, `dist/flatpak/*.sha256`, and `dist/flatpak/*.json` to the GitHub Release. -- Manual GitHub release dispatch resolves the tag from checked-in `.version` and refuses to create the tag when `.version`, `cli/Cargo.toml`, and `npm/package.json` are not already aligned. +- Manual GitHub release dispatch resolves the tag from checked-in `.version` and refuses to create the tag when `.version`, `cli/Cargo.toml`, and `npm/package.json` are not already aligned. Manual dispatch also exposes a boolean pre-release checkbox that is passed only to `softprops/action-gh-release` as the GitHub Release-level `prerelease` flag; it does not change tag naming, release title/body/notes, generated release notes, artifact names, or version validation. - Tag-triggered release execution also refuses to proceed when the pushed tag does not equal `v<.version>` or when checked-in Cargo/npm package metadata drift from `.version`. - `nix run .#release-artifacts` fails fast when the requested `--version` disagrees with `.version`, `cli/Cargo.toml`, `npm/package.json`, or the built CLI `sce version` output. - `nix run .#release-flatpak-package` fails fast when the requested `--version` disagrees with `.version`, `cli/Cargo.toml`, `npm/package.json`, or Flatpak AppStream release metadata, and also fails when it cannot resolve a release commit from a git checkout. From 8b8d381645e534acd77d4690cd512bb7abb70431 Mon Sep 17 00:00:00 2001 From: David Abram Date: Wed, 24 Jun 2026 10:31:58 +0200 Subject: [PATCH 3/4] flatpak: Allow network access during source builds Allow Flatpak builder to use network access for the Rust source build module so Cargo dependency resolution can complete in the sandboxed build environment. --- nix/flatpak/manifest.nix | 1 + packaging/flatpak/dev.crocoder.sce.yml | 2 ++ 2 files changed, 3 insertions(+) diff --git a/nix/flatpak/manifest.nix b/nix/flatpak/manifest.nix index 431cf8ee..fcd9e6eb 100644 --- a/nix/flatpak/manifest.nix +++ b/nix/flatpak/manifest.nix @@ -45,6 +45,7 @@ let buildsystem = "simple"; build-options = { append-path = "/usr/lib/sdk/rust-stable/bin"; + build-args = [ "--share=network" ]; env = { CARGO_HOME = "/run/build/sce/cargo"; }; diff --git a/packaging/flatpak/dev.crocoder.sce.yml b/packaging/flatpak/dev.crocoder.sce.yml index 197cc5e7..cf120e4b 100644 --- a/packaging/flatpak/dev.crocoder.sce.yml +++ b/packaging/flatpak/dev.crocoder.sce.yml @@ -14,6 +14,8 @@ modules: - install -Dm644 packaging/flatpak/dev.crocoder.sce.metainfo.xml /app/share/metainfo/dev.crocoder.sce.metainfo.xml build-options: append-path: /usr/lib/sdk/rust-stable/bin + build-args: + - --share=network env: CARGO_HOME: /run/build/sce/cargo buildsystem: simple From 5f505ca8df75063a065a74e74d78490c54bfd8b9 Mon Sep 17 00:00:00 2001 From: David Abram Date: Wed, 24 Jun 2026 10:53:51 +0200 Subject: [PATCH 4/4] ci: Allow Flatpak bundle sandbox on Linux runners Enable unprivileged user namespaces before Flatpak bundle builds so bubblewrap can set up the build sandbox on GitHub-hosted Linux runners. Co-authored-by: SCE --- .github/workflows/release-sce-linux-arm.yml | 7 +++++++ .github/workflows/release-sce-linux.yml | 7 +++++++ 2 files changed, 14 insertions(+) diff --git a/.github/workflows/release-sce-linux-arm.yml b/.github/workflows/release-sce-linux-arm.yml index f94229e8..9247f7a3 100644 --- a/.github/workflows/release-sce-linux-arm.yml +++ b/.github/workflows/release-sce-linux-arm.yml @@ -36,6 +36,13 @@ jobs: --version "${{ inputs.release_version }}" \ --out-dir dist + - name: Allow Flatpak build sandbox + run: | + sudo sysctl -w kernel.unprivileged_userns_clone=1 + if [ -e /proc/sys/kernel/apparmor_restrict_unprivileged_userns ]; then + sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 + fi + - name: Build Flatpak bundle release assets (aarch64) run: | nix run .#release-flatpak-bundle -- \ diff --git a/.github/workflows/release-sce-linux.yml b/.github/workflows/release-sce-linux.yml index 2eef993f..01fd0063 100644 --- a/.github/workflows/release-sce-linux.yml +++ b/.github/workflows/release-sce-linux.yml @@ -36,6 +36,13 @@ jobs: --version "${{ inputs.release_version }}" \ --out-dir dist + - name: Allow Flatpak build sandbox + run: | + sudo sysctl -w kernel.unprivileged_userns_clone=1 + if [ -e /proc/sys/kernel/apparmor_restrict_unprivileged_userns ]; then + sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 + fi + - name: Build Flatpak bundle release assets (x86_64) run: | nix run .#release-flatpak-bundle -- \