binfmt/fdpic: Add an FDPIC ELF loader for execute-in-place modules - #19584
Open
casaroli wants to merge 9 commits into
Open
binfmt/fdpic: Add an FDPIC ELF loader for execute-in-place modules#19584casaroli wants to merge 9 commits into
casaroli wants to merge 9 commits into
Conversation
DT_INIT_ARRAY, DT_FINI_ARRAY and their size tags are in the base ELF specification (Figure 5-10) but were missing from the header, which stopped at DT_BINDNOW. A loader that wants to run an object's constructors has nothing to compare d_tag against. Assisted-by: Claude Code:claude-opus-4-8 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Lets a NOMMU target execute downloadable modules in place from memory-mapped NOR flash, so a module's text and rodata never consume RAM. It is the consumer of xipfs: the loader maps a module's read-only segment with MAP_XIP_STRICT, which resolves to a direct flash pointer or fails with -ENXIO rather than falling back to a RAM copy, and pins the extent for as long as the module is loaded so the defragmenter cannot relocate code that is executing. Only the writable segment is copied to RAM, once per running instance. The loader follows DT_NEEDED so a module can use shared libraries, each object getting its own GOT and its own data. FDPIC is what makes this possible: text is position-independent and each LOAD segment is placed independently, with the text-to-data offset communicated at load time through function descriptors and the GOT. A descriptor is a pair -- entry pointer plus data base -- so a module function handed back to the firmware carries the data base it needs. r9 holds that base at runtime, per the ARM FDPIC ABI. Reserving r9 across the base firmware is what allows a firmware routine to call back into module code and still arrive with the module's data base intact. Toolchain.defs puts --fixed-r9 in ARCHCPUFLAGS rather than CFLAGS, because almost every board Make.defs assigns CFLAGS with ':=' after including it, which would discard the flag; ARCHCPUFLAGS is re-expanded by that same assignment and so survives it. The CONFIG_PIC --fixed-r10 case is skipped under FDPIC, since reserving both registers would cost one for nothing. The DT_NEEDED walk is depth capped. fdpic_loaddepends() recursed once per link of a dependency chain with a path buffer on each frame and nothing to stop it, so a malformed module set overflowed the stack of whichever task called the loader instead of being rejected. A dependency *cycle* was never the hazard -- an object joins the load's list before its own dependencies are walked, so a library naming something already loaded finds it there and stops -- what was unbounded is a chain of distinct names, which the list cannot bound, hence an explicit cap rather than cycle detection. A module's .rofixup section is skipped, and the file header records why. .rofixup is the FDPIC self-relocation list a static executable's crt0 walks to derive its own GOT when no loader is present. A module links -shared -nostartfiles, so no crt0 runs, and the built objects hold exactly one entry there -- the address of the GOT itself, which this loader computes and installs at every entry into module code anyway. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Two gaps that both fail quietly. DT_INIT_ARRAY and DT_FINI_ARRAY were ignored entirely. A C++ module with any global object therefore loaded, resolved every symbol and ran, with all of its globals left as .bss reading back zero -- no fault, no log, just wrong answers. Both arrays are now walked per object and in dependency order: an object joins the load's list before the DT_NEEDED walk appends its own dependencies, so walking that list backwards constructs a library before the module that needs it, and destruction mirrors it. Constructors run in whichever task called the loader, before the module's own task exists, so the FDPIC register does not already hold the object's data base the way it does once the module is running. fdpic_callfn() installs it around each call. That is safe only because the firmware reserves the register; being preempted mid-constructor is harmless, since the register is part of the saved context. DT_JMPREL was not parsed at all. Which table an imported function's descriptor lands in is a linker decision -- -z now puts it in DT_REL, and without it the same entry goes to DT_JMPREL -- so a module linked the second way loaded cleanly and then branched to an unrelocated address on its first call into the firmware. The symptom is an INVSTATE UsageFault escalated to a HardFault: no console, no crash dump. There is no lazy resolver here, so an unwalked table is not deferred work; both are now bound eagerly, and nothing is lost by that because a module carries a handful of relocations. The two tables are not walked identically, which is the part worth remembering. In DT_REL the word being overwritten is the addend, and dropping it breaks a static function reached through its section symbol. In DT_JMPREL that same word is the lazy-binding bootstrap -- the address of the entry's own PLT resolution stub, with a GOT half of -1 -- and adding it to the resolved symbol value produces an arbitrary address that faults exactly like the bug this change fixes. An eager binder overwrites the descriptor outright. The descriptor pool is sized from relsize + pltrelsz rather than relsize alone, so an R_ARM_FUNCDESC in the PLT table cannot run off the end of the allocation. GNU ld does not appear to emit that combination -- an address-taken function is not a call and so never becomes a PLT relocation -- but the failure it would cause is heap corruption, and the guard is two additions. An object declaring RELA PLT relocations is refused rather than misread: nothing here reads RELA, whose entries are twelve bytes rather than eight. Assisted-by: Claude Code:claude-opus-4-8 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A module's function pointer is the address of a two-word descriptor in its writable segment, not a code address. Firmware that accepts one, stores it, and later branches to it therefore jumps into the module's RAM data. There is no diagnostic: the board takes a HardFault with a dead console. qsort and bsearch have resolved the descriptor since FDPIC support landed. Nothing else did, and nothing noticed, because qsort's comparison function was the only callback anything exercised. That left every other entry point looking perfectly usable -- a module could call one, it would return success, and the fault arrived later from somewhere else. Resolution is opt-in per call site via fdpic_callback(), and this covers the rest of them: pthread_create, signal, task_create, pthread_once, task_spawn, scandir and sigaction. Resolution happens once, in the innermost routine the paths share, for the reason qsort already documents: resolving twice would treat a code address as a descriptor. task_create_with_stack resolves and task_create forwards to it; nxsig_action() resolves for both signal() and a direct sigaction(), on the local copy it already makes, so the caller's const struct is untouched and the SA_SIGINFO form is covered through the union. Two entry points need care beyond that pattern: signal() has to exclude the dispositions by hand. SIG_IGN, SIG_DFL, SIG_HOLD and SIG_ERR are the integers 0, 1, 2 and -1 rather than addresses, and fdpic_callback() declines to dereference NULL and nothing else -- handing it SIG_ERR would read through (void *)-1. scandir takes two pointers. Its filter is called by scandir itself and is resolved here; its comparison function is handed to qsort(), whose own entry point resolves it, so resolving it here as well would resolve twice. The test passes both at once, which is what exercises that distinction. Resolving the code address is sufficient at all of these. A thread or task created from a module inherits its D-Space -- nxtask_dup_dspace() runs before up_initial_state() installs it in the FDPIC register -- and a signal handler or pthread_once init routine runs in a context that already holds the module's data base. Verified on an RP2350 with a module that hands its own function to each entry point. Each assertion was also checked against a deliberate breakage: removing the scandir filter resolution, and adding a second resolution of the comparison function, each take the board down with a HardFault rather than printing a FAIL line -- which is how a broken callback resolution manifests on Cortex-M, and confirms the assertions test what they name. mq_notify and timer_create with SIGEV_THREAD remain, and are harder: their callback runs on a work-queue worker that carries no module data base at all. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A SIGEV_THREAD notification -- from mq_notify() or timer_create() -- runs its callback on a shared signal-notification work queue, not in the registering task. That worker carries no FDPIC data base, so a module's callback reaches it with the wrong base and cannot touch its own globals. This is unlike every other callback entry point, where the callback runs in a task that inherited the module's data space and resolving the code address is enough. The base is knowable exactly once, at registration, when the call is still in the module's own context: capture it there with fdpic_base() into the persisted work structure (mq's ntwork, the timer's pt_work). At send or expiry the descriptor is resolved to its code address -- a plain memory read that needs no base -- and stored in work->func. The worker, seeing a non-zero base, installs it in the FDPIC register around the call and restores it after; a zero base, which is every non-module callback, takes the direct path unchanged. fdpic_base() exposes the test fdpic_callback() already makes internally -- whether the caller is a module -- for a site that has to decide before it stores a pointer somewhere the register will no longer be correct. fdpic_invoke() is the install-call-restore, in the same ARM-thumb inline asm as the rest of fdpic.h. It saves the register on the stack and keeps the push 8-byte aligned, and pins the argument in r0, so it asks the allocator for only two free registers -- enough on builds that also reserve a frame pointer. It is safe against preemption: the FDPIC register is REG_PIC in the saved context, preserved across a context switch, and base firmware reserves it so no interrupt handler disturbs it. A context switch or interrupt while the callback runs therefore keeps the module's base. Verified with CONFIG_SIG_EVTHREAD on two ARM cores: an RP2350 (Cortex-M33, armv8-m) on hardware and mps2-an500 (Cortex-M7, armv7e-m) under QEMU. On each a module's mq_notify and timer_create SIGEV_THREAD callbacks run on the worker and write a distinctive value into a module global, proving the base was installed. Removing just the register install makes the same callback HardFault the board, confirming it is load-bearing. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Documentation/components/fdpic.rst covers what someone building or loading an FDPIC module needs: how the format works and what the loader does with it, how it differs from NXFLAT and from the ELF loader's own PIC/XIP path, what the target and toolchain have to provide, how to build a module, a shared library and a C++ module, and what a firmware entry point must do to accept a module callback. The comparison is the part worth stating plainly, because two of the three properties are shared. All three formats run position-independent code from flash with no MMU and give several instances of one module a shared .text with private .data. NXFLAT needs its own tools and cannot export symbols, so it has no shared libraries. ELF PIC needs no extra tools at all, but one base register per task means a shared object is loaded as a single allocation -- its text cannot stay in flash -- and there is no DT_NEEDED walk. FDPIC costs an arm-uclinuxfdpiceabi linker and buys a pointer that carries its own data base, which is what makes shared libraries and callbacks on a thread the module never created possible. The reference material at the end is the part that is expensive to rediscover: the FDPIC marker being in EI_OSABI rather than e_flags, the three relocation types that survive a static link, why both relocation tables are bound eagerly, why .rofixup is skipped, and the two binfmt contract details that fail as a panic or a bare -EINVAL. Also adds the fdpic and reject sections to the xipfs test suite document, which apps/testing/fs/xipfs grows in the matching apps change. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Modules are built out of tree, and the makefile fragment and scripts that do it lived in a separate repository until now, which left the loader documented in this tree but not buildable from it. ``nuttx-fdpic.mk`` reduces a module to three lines of makefile. The split it encodes is the whole trick: the stock arm-none-eabi compiler emits correct FDPIC objects for both C and C++, and only the *link* needs arm-uclinuxfdpiceabi binutils, so the from-source dependency is binutils alone. fdpic-verify checks that a built module's imports resolve against the firmware's export table, because a module importing a symbol the firmware does not export links cleanly and fails only once it is on the target, as a bare -ENOENT that names nothing. nuttx-exports produces that table from libs/libc/exec_symtab.c, which has to be run through the preprocessor rather than read textually. fdpic-embed turns a built module into a C header, for an app that has to load a module before there is any way to put files on the target. build-binutils.sh builds that one dependency, about a minute. build-toolchain.sh builds a full FDPIC GCC for anyone who wants one; nothing here needs it. BINDNOW is added while moving: the makefile passes -z now by default, and emptying it leaves imported descriptors in the lazy binding table, which is the case one of the test fixtures exists to cover. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
The xipfs configuration alongside it stops at the filesystem. This one turns on CONFIG_FDPIC as well, so the board runs the module loader and the FDPIC half of the test suite, and it carries the four options the loader needs but which are not obviously part of it: CONFIG_LIBC_EXECFUNCS with a system symbol table, because a module resolves its imports against one. POSIX timers stay enabled -- the callback test module imports timer_create, and with them disabled the loader cannot resolve that symbol, so the whole module fails to load and the FDPIC section reports twelve failures that name the callback rather than the missing timer. CONFIG_SCHED_HPWORK and CONFIG_SIG_EVTHREAD carry the SIGEV_THREAD delivery itself. CONFIG_INIT_STACKSIZE is raised to 16 KB: the test suite runs the power-loss sweeps from the init task. CONFIG_NXFLAT is deliberately left out. The nxflatxip demo's build runs mknxflat, which the CI container does not have, and that is why xipfs-nxflat is excluded from the CI build list. The FDPIC modules are prebuilt blobs instead, so this configuration builds with a plain toolchain and CI covers the loader. Verified on a Pimoroni Pico Plus 2. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Describes what each of the four subcommands shows and why it is worth showing -- each is one loader property that fails quietly if it is wrong -- and records that the modules are prebuilt blobs because building them needs a toolchain the tree does not require. Also enables the example in the board's xipfs-fdpic configuration. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
casaroli
requested review from
Donny9,
GUIDINGLI,
Ouss4,
davids5,
fdcavalcanti,
gustavonihei,
hartmannathan,
jerpelea,
masayuki2009,
pkarashchenko,
pussuw,
raiden00pl,
xiaoxiang781216,
yamt and
yf13
as code owners
July 30, 2026 12:42
github-actions
Bot
added
Arch: arm
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an FDPIC ELF loader so a downloadable module can execute in place out of memory-mapped flash: the read-only segment is mapped where it already sits on the media and never copied to RAM, and only the writable segment is copied, once per running instance.
The in-tree ELF loader already does shared
.textwith private.dataon a no-MMU target (CONFIG_PIC,mps3-an547:picostest), so that part is not new. What FDPIC adds is a function pointer that carries its own data base, as a two-word descriptor{entry, GOT}rather than a bare code address. Two things follow that a single base register per task cannot express: a module can be called back on a thread it never created, such as the work-queue worker that runs aSIGEV_THREADnotification; and two objects can hold distinct data bases at once, which is what makesDT_NEEDEDshared libraries work with per-instance library data.Compared with NXFLAT: FDPIC is standard ELF, needs no
mknxflat/ldnxflatand no linker script, puts.rodatain the RX segment on its own, supports shared libraries, and needs no workaround for a pointer to astaticfunction. The cost is anarm-uclinuxfdpiceabilinker; the stockarm-none-eabicompiler emits correct FDPIC objects for both C and C++, so only the link needs it.The loader is
binfmt/fdpic.cunderCONFIG_FDPIC(depends onCONFIG_ARCH_ARM, selectsBINFMT_LOADABLEandPIC). It requires a filesystem that answersBIOC_XIPBASE, such as XIPFS or ROMFS.arch/arm/src/common/Toolchain.defsadds--fixed-r9so the base firmware reserves the FDPIC register; without it a firmware callback into module code arrives with the wrong data base.Nine commits: the
DT_*_ARRAYtags ininclude/elf.h; the loader; constructors and PLT relocation binding; descriptor resolution at the libc/sched entry points that accept a module callback;SIGEV_THREADnotifications; the developer guide inDocumentation/components/fdpic.rst; the out-of-tree module build tooling intools/fdpic; and a board configuration plus the demo's documentation.Paired with apache/nuttx-apps#3682, which adds the demo and the test suite sections. This PR stands alone and is the one to merge first: nothing in the apps PR can be enabled until
CONFIG_FDPICexists here. Merging this first leavespimoroni-pico-2-plus:xipfs-fdpicbuilding the loader and the filesystem tests but not the demo, becauseCONFIG_EXAMPLES_FDPICXIPdoes not resolve yet; that corrects itself when the apps PR lands, with no follow-up change needed.Impact
New feature, off by default.
CONFIG_FDPICdefaults ton, so a tree that does not enable it is unaffected.Existing code touched in two places, both no-ops without
CONFIG_FDPIC. First, ten libc/sched entry points that can accept a callback from a module now resolve an FDPIC descriptor before storing or branching to it:qsort,bsearch,pthread_create,signal,sigaction,task_create/task_create_with_stack,task_spawn,pthread_once,scandir, andmq_notify/timer_createwithSIGEV_THREAD. Each is guarded by#ifdef CONFIG_FDPIC. Second,Toolchain.defsadds--fixed-r9toARCHCPUFLAGS, again only underCONFIG_FDPIC, which costs the base firmware one register in that configuration.Adds one board configuration,
pimoroni-pico-2-plus:xipfs-fdpic. It builds with a plainarm-none-eabitoolchain because the test modules are committed as prebuilt blobs, so CI covers it; it deliberately does not enableCONFIG_NXFLAT, which would needmknxflatand require excluding the configuration from the build list.Documentation: one new page,
Documentation/components/fdpic.rst, added to the components toctree, plus thefdpicandrejectsections in the xipfs test suite page.No change to any existing API, ABI, or default configuration. FDPIC is ARM Thumb-2 only; RISC-V has no FDPIC ABI, so a RISC-V target cannot use this loader.
Testing
Host: macOS 26.5.1 on arm64. Arm GNU Toolchain 15.2.Rel1 (
arm-none-eabi-gcc15.2.1),arm-uclinuxfdpiceabi-ldfrom GNU Binutils 2.43, QEMU 11.0.3.QEMU,
mps2-an500(Cortex-M7, ARMv7E-M) — the loader on a different core generation from the hardware below, with xipfs on arammtddevice that answersBIOC_XIPBASE. Built frommps2-an500:xipfsplus the FDPIC options. Fullxipfs_testsuite:The two FDPIC sections on their own:
The
fdpicxipdemo, all four subcommands.solib, showing two instances sharing one copy of a library's text in flash while each gets its own copy of its data:cxxadditionally confirms each object's global constructors ran in dependency order beforemain, andjmprelthat a module whose imports are all inDT_JMPRELbinds and calls out.Hardware, Pimoroni Pico Plus 2 (RP2350, Cortex-M33, ARMv8-M) against real QSPI flash,
pimoroni-pico-2-plus:xipfs-fdpic: full suite 130/130,fdpic33/33,reject7/7, and all four demo subcommands. The same module blobs run on both cores; they are built forcortex-m3, so one set serves v7-M and v8-M.Simulator,
sim:xipfson the host, which exercises the filesystem half only since the loader is ARM-only:Loader assertions verified against deliberate breakage. Each check in the
fdpicsection was confirmed to fail when the thing it names is broken — removing thescandirfilter resolution, adding a second resolution of the comparison function, and removing the FDPIC register install around aSIGEV_THREADcallback each take the board down with a HardFault rather than printing a FAIL line, which is how a broken callback resolution manifests on Cortex-M.--fixed-r9reaches the compiler, checked per the guide, and prints that flag and no other reserved-register flag:tools/checkpatch.sh -fpasses on every changed C and header file.🤖 Generated with Claude Code
https://claude.ai/code/session_01X77FVXXW4bvvBtfFmz1JPz