feat(cli): replace xgenext2fs and cartesi-machine spawns with native bindings - #509
feat(cli): replace xgenext2fs and cartesi-machine spawns with native bindings#509tuler wants to merge 2 commits into
Conversation
…bindings Build ext2 drives with @deroll/genext2fs and configure, boot, store and hash the Cartesi machine with @deroll/cm, instead of spawning xgenext2fs, cartesi-machine and cartesi-machine-stored-hash (falling back to running them inside the SDK docker image). machine.ts now translates a cartesi.toml Config into an emulator MachineConfig directly, mirroring what the cartesi-machine CLI does with its command line: the boot args it appends to, the init script (splash, flash drive mounts and chowns, environment exports, WORKDIR and USER), the flash drives with root first so it lands on pmem0, and the virtio console setup for an interactive shell. buildMachineConfig is a pure function, covered by unit tests. With no SDK image to take the kernel from, images.ts downloads the pinned cartesi/image-kernel release on first use, verifies its checksum and caches it under XDG_CACHE_HOME. CARTESI_IMAGES_PATH and machine.ram_image still win. The emulator moves from 0.20 to 0.21, so machine hashes change. The standalone binaries are gone: a bun single file executable has no node_modules, and both addons resolve their platform .node at runtime, so they cannot be embedded. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UTEd5g3mF849BATTstssR3
cartesi/image-kernel is the repository's old name; use the canonical one. Same release, same artifact, same checksum — GitHub was serving it through a redirect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UTEd5g3mF849BATTstssR3
🦋 Changeset detectedLatest commit: 99bb3fa The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Coverage Report
📁 File Coverage (20 files)
|
Note on the
|
| image: TEST_SDK, | ||
| cwd: machineDir, | ||
| }); | ||
| it("should return the hash of a stored machine", async () => { |
There was a problem hiding this comment.
I opened a PR regarding that
3 participants
Contributes to #72.
That issue asks for the external programs the CLI drives to stop being spawned binaries, and notes that "instead of spawning binaries another possibility is to build NodeJS bindings to native code components". This PR takes that route for two of the four programs listed there.
xgenext2fsexeca, falling back todocker runin the SDK image@deroll/genext2fscartesi-machineexeca, falling back todocker runin the SDK image@deroll/cmcartesi-machine-stored-hashexeca, falling back todocker runin the SDK image@deroll/cmmksquashfscartesi-rollups-cli/cartesi-rollups-nodeDocker is still needed to build the root drive from a Dockerfile (riscv64 via QEMU), for squashfs drives, and for the services
runbrings up — which is the state issue #72 describes as done. The remaining work there is the Node Unit's binaries.What changed
src/exec/genext2fs.ts—empty,fromDirectoryandfromTarcallcreateImage/tarToExt2with the same block size, faketime and readjustment settings as the old command lines.version()is now a synchronous lookup of the vendored xgenext2fs version rather than a--versionsubprocess.src/machine.ts— the substantial part. The CLI used to handcartesi-machinea command line and let it assemble the machine configuration. It now does that translation itself, transcribed fromcartesi-machine.luav0.21.0:dtb.bootargsfrom the emulator, withmachine.boot_argsappendeddtb.init: the splash, thendev=$(flashdrive <label>)+ mount + chown per non-root drive, thenexport K="V",WORKDIR,USER— in the order the old argv producedrootfirst, so it lands on/dev/pmem0regardless of the order drives appear incartesi.tomlconsole=hvc1andiunrep=1forcartesi shellbuildMachineConfigis a pure function so this is unit-testable without running a machine;src/exec/cartesi-machine.tsholds the run loop (automatic yields acknowledged, console I/O resumed, guest exit code read fromhtif_tohost_data).src/images.ts(new) — with no SDK image to takelinux.binfrom, the defaultram_imageis fetched from the pinnedcartesi/machine-linux-imagev0.21.0 release on first use, verified against its SHA-256 and cached under$XDG_CACHE_HOME/cartesi/images. ACARTESI_IMAGES_PATHdirectory holding the image is used when set, andmachine.ram_imagestill wins over both.Breaking
@deroll/cmlinks against emulator 0.21, the SDK image pins 0.20. Applications need redeploying.--append-bootargs="<arg>"with no shell in between, so the quotes ended up in the kernel command line. Building the configuration directly, reproducing that would have meant reproducing a bug.node_modules, and both addons resolve their platform.nodeat runtime — a host-native--compilefails too, so cross-compiling four targets was never going to work. The compile step inbuild.tsand the upload step inrelease.yamlare removed. The homebrew formula incartesi/homebrew-tapneeds to install from npm rather than the release tarball. Happy to revert this bit and solve distribution differently if you'd rather keep the binaries.Testing
tests/unit/machine.test.tsis new and covers the configuration translation (mount points, drive ordering, environment precedence, workdir/user, interactive mode,parseMemorySize).genext2fsandcartesi-machinemoved fromtests/integration/totests/unit/, since neither needs Docker anymore, and the genext2fs one now actually builds images and asserts they are reproducible.cartesi-machine-stored-hash.test.tsasserted a literal hash that the emulator bump invalidates. It now asserts shape and stability — worth pinning the real value once CI produces it.Not verified locally: a successful rollup application boot, which needs Docker with riscv64. The integration tests cover that path.
Generated by Claude Code