Skip to content

fix(shop-install): four bugs blocking a real GitHub-shop virtual-app install - #230

Merged
rubenvdlinde merged 6 commits into
developmentfrom
fix/shop-install-credential-and-route-gaps
Aug 17, 2026
Merged

fix(shop-install): four bugs blocking a real GitHub-shop virtual-app install#230
rubenvdlinde merged 6 commits into
developmentfrom
fix/shop-install-credential-and-route-gaps

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Summary

Found and fixed while live-testing the GitHub shop-install path (ShopController/AppChannelApplier/AppRepoSerializer) against buildiq-spectr and buildiq-hydra on repeatedly-fresh Nextcloud instances — three bugs, each confirmed live before and after the fix:

  1. installFromTemplateArray() never forwarded owner/repo/credentialId into AppChannelApplier::apply() — Hermiq's skill-bundle installer does its own independent GitHub fetch and always ran anonymous, 404ing on any private source repo. Broke 100% of skill installs from either buildiq app.
  2. ApplicationPublishController::publish() never created a BuiltAppRoute index object — only the app-creation wizard did, at creation time. Any app started as a draft and published later (every template-install path) stayed unreachable at its manifest/builder URL even after a successful publish.
  3. persistApplication()'s manifest field is silently dropped by the application schema, and no ApplicationVersion was ever created for a template-installed appgetManifest() resolves via productionVersionApplicationVersion.manifest; without one, every template-installed app returns no_manifest forever, even when published.
  4. AppRepoSerializer's connector reference resolution validated sourceId as UUID-only, but measured live data shows sources are commonly slug-referenced ("sourceId": "tenderned") — 9 of spectr's 13 referenced sources were silently dropped from every export. Fixing this surfaced a second bug: dedup was keyed on the raw reference string instead of the object's own resolved identity, so a source reachable by both UUID and slug installed twice under two filenames — fixed by reading the real UUID from the ObjectEntity, not the payload.

Verified live

Reinstalled both buildiq-spectr and buildiq-hydra via the real shop-install path on two separate from-scratch Nextcloud instances (torn down and rebuilt between rounds):

  • spectr: register + all 52 connectors + all 21 skills install cleanly; manifest resolves (v2.4.0, 71 pages) after publish.
  • hydra-console: register installs cleanly; manifest resolves (v0.1.0, 7 pages) after publish.
  • Republished buildiq-spectr twice post-fix — connector count stable at the correct 13 sources, no duplicates.

Known follow-up, not fixed here

GitHubTemplateCatalogService::collectTreeBlobs() fetches one file at a time sequentially for a skill bundle (a 94-skill bundle is ~300-500 sequential GitHub API round-trips) — under load this can time out partway through with a misleading 0 created / not-accounted-for-by-source result despite real partial progress. Root-caused but not fixed in this PR (needs batching/parallelizing the fetch, a bigger change).

Test plan

  • php -l clean on all touched files
  • Local Hydra gates: 30/30 green on the diff (gate-66's 16 findings are pre-existing, confirmed zero overlap with touched files)
  • Live end-to-end verification as described above

…reate built-app-route on publish, resolve slug-referenced connectors

Three bugs found while live-testing the GitHub shop-install path against
buildiq-spectr/buildiq-hydra on a fresh instance:

- installFromTemplateArray() never forwarded owner/repo/credentialId into
  AppChannelApplier::apply(), so Hermiq's skill-bundle installer (which does
  its own independent GitHub fetch) always ran anonymous and 404'd on any
  private source repo — reported as a clean "skipped", indistinguishable
  from "this repo has no skill bundle". Broke 100% of skill installs from
  either buildiq app (0/21, 0/94).

- ApplicationPublishController::publish() never created a BuiltAppRoute
  index object. Only the app-creation wizard did (atomically, at creation
  time) - any app that starts as a draft and is published later via the
  local-template, remote-registry, or GitHub-shop paths stayed unreachable
  at its manifest/builder URL even after a successful publish.

- AppRepoSerializer::directReferences() validated a synchronization's
  sourceId/mapping references as UUID-only, but measured live data shows
  sources are commonly slug-referenced ("sourceId": "tenderned"). 9 of
  spectr's 13 referenced sources were silently dropped as "unsafe" during
  export. find(id:) already resolves both shapes; only the validation gate
  was too narrow.
…nd slug

Fixing the slug-reference resolution (previous commit on this branch)
surfaced a second bug: the declared-binding write path never checked
`$seen` before writing, only the reference-resolution path did. So when a
synchronization binding resolved its source via a slug-shaped `sourceId`
BEFORE that source's own explicit top-level binding was reached later in
`$bindings`, the explicit binding wrote it AGAIN under a different filename
(uuid-named vs slug-named) — live-verified duplicate on spectr's export:
`connectors/source/norway-doffin.json` and
`connectors/source/bc2d32cc-….json` both present for the same object.

Root cause: dedup was keyed on the RAW reference string a caller supplied
(a UUID for a declared binding, a slug for a resolved reference) instead of
the object's own resolved identity — two different spellings of the same
identifier never matched as "seen". findConnector() now surfaces the
ObjectEntity's real UUID via a $resolvedUuid out-param (never trust the
payload's own `id`/`uuid` property — that's OpenRegister metadata, not
reliably part of the authored object body), and both write paths check
`$seen` against that real identity before writing.

Manually cleaned up buildiq-spectr's 4 stale duplicate files (leftovers
from the broken pre-fix pushes) via direct API delete — GitHub's tree API
`base_tree` overlay means a push never removes a path that stops being
serialized, only adds/updates what's currently in the tree. That's a
separate, pre-existing limitation of the push mechanism, not fixed here.

Verified live: republished spectr twice post-fix, source count stable at
13 (was 4, briefly 17 with the duplicate bug, now 13 clean) across repushes.
… manifest lives on

Third bug found completing the clean-install verification: persistApplication()
stores a manifest key directly on the Application object, but the `application`
schema does not declare that property — OpenRegister silently drops it on save
(confirmed live: a freshly cloned Application object carries no `manifest` key
at all). getManifest() resolves via Application.productionVersion ->
ApplicationVersion.manifest (ADR-002); its documented application-level
fallback can therefore never actually fire against this schema.

Without this fix, every template-installed app (local, remote-registry, or
GitHub-shop) publishes successfully (status: published, built-app-route
correctly created per the previous fix on this branch) but its manifest and
builder endpoints permanently return 404 no_manifest.

installFromTemplateArray() now creates a single "production" ApplicationVersion
carrying the cloned manifest, on the per-app register already provisioned for
the install, and links Application.productionVersion to it — mirroring what
ApplicationCreationService's wizard flow already does for a from-scratch app,
minus the multi-version promotesTo chain a one-shot clone doesn't need.

Verified live end-to-end on a fresh instance: reinstalled both spectr and
hydra-console via the GitHub shop path post-fix — spectr's manifest now
resolves (v2.4.0, 71 pages), hydra-console's resolves (v0.1.0, 7 pages).
Neither did before this commit.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ e868d99

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
check-gitignore
check-nc-floor
format
composer ✅ 106/106
npm ✅ 626/626
app:check-code ⏭️
info.xml
REUSE
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-16 21:52 UTC

Download the full PDF report from the workflow artifacts.

@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ 55bb059

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
check-gitignore
check-nc-floor
format
composer ✅ 106/106
npm ✅ 626/626
app:check-code ⏭️
info.xml
REUSE
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-17 01:21 UTC

Download the full PDF report from the workflow artifacts.

…ication

The install path's second write patched `productionVersion` in with a
hand-built four-field payload. OpenRegister's `saveObject()` is PUT, not
PATCH: on the update path `SaveObject::prepareObjectForUpdate()` calls
`fillMissingSchemaPropertiesWithNull()` and then `setObject()` replaces the
stored data outright — there is no merge with the existing object anywhere
in that path. That write therefore NULLed `owner`, `status`, `version` and
`templateOrigin` off the Application persisted one statement earlier.

`persistApplication()` now hands back the stored payload and
`linkProductionVersion()` re-saves that whole object with the one field
patched in, mirroring `ApplicationPublishController::setStatus()`.

Test doubles: the new second/third writes broke four existing tests. Two
tripped `expects(once())`; two silently retargeted onto the LAST payload the
callback saw. Both are now asserted at their real counts — `exactly(3)` for
an install, `exactly(2)` for a publish — with the register, schema and uuid
of every individual write checked, so the counts cannot pass for any three
saves at all. The saveObject double now echoes the persisted object back the
way OpenRegister does; returning a bare `['uuid' => …]` had made the
controller's read-modify-write look lossless regardless of what it sent.
Adds a regression test pinning that the re-save preserves the record.

phpmd: `installFromTemplateArray()` (114 lines) and `collectConnectors()`
(104) were over the 100-line limit. Split along real seams — the clone
half of the install becomes `materialiseApplication()`, and reference
resolution becomes `collectDirectReferences()` +
`exportReferencedConnector()`. No suppressions, no threshold changes.

Also fixes 6 phpmd UndefinedVariable findings ($realUuid / $realRefUuid
were only ever created by the by-reference out-param, and the `!== ''`
guard against a possible null read the wrong way), plus the two phpcs
errors on lines this PR introduced.
@rubenvdlinde rubenvdlinde changed the title fix(shop-install): three bugs blocking a real GitHub-shop virtual-app install fix(shop-install): four bugs blocking a real GitHub-shop virtual-app install Aug 17, 2026
…onVersion link

The coverage ratchet failed the previous head while the suite itself passed
— `OK (852 tests)` and then exit 1 from the coverage-guard STEP inside the
PHPUnit job. Its own verdict was self-refuting: "coverage of the files this
change touches dropped by 0%", base 809/1380 (58.6232%) -> head 877/1496
(58.6230%). The 116 added statements were covered at 68/116 = 58.62%, i.e.
exactly the file mean — as arithmetically neutral as a change can be — and
it still failed by 0.0002 percentage points.

Rather than buy that back with filler, this pins two contracts
`linkProductionVersion()` documents but nothing tested:
  - a failing ApplicationVersion write must NOT fail the install (the app,
    its register and its schemas are already provisioned; losing all of it
    over a manifest-resolution step is worse than the 404 it guards), and
    must not attempt the Application re-save;
  - a version save that yields no UUID must NOT trigger the re-save, which
    would otherwise overwrite a working pointer with an empty one.

Measured, not assumed: changed-file coverage goes 877/1496 -> 883/1496,
58.6230% -> 59.0241%, clearing the base by 0.40pp. The local pcov run
reproduces CI's xdebug figure of 877/1496 exactly, so the instrument is the
same one the gate uses.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ 747eac0

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
check-gitignore
check-nc-floor
format
composer ✅ 106/106
npm ✅ 626/626
app:check-code ⏭️
info.xml
REUSE
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-17 02:45 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Fix crew report — three red groups cleared, one real bug found

1. PHPUnit: 4 existing tests, not 4 new ones

Tests: 851, Failures: 4 against a base of OK (851 tests) — identical count, so nothing was added; four existing tests broke on the new second saveObject() call.

Only two were expects($this->once()) violations. The other two were subtler and worth naming: they captured $object into a $savedPayload variable on every call, so the last write silently became the assertion target. testManifestSchemaRefsRewrittenWithNewSlugPrefix then asserted the manifest of a payload that legitimately has no manifest, and testOwnerFieldSetToAuthenticatedUid asserted owner on one that legitimately has no owner. A capture-the-last-call double retargets itself the moment a second call appears.

Fixed by asserting the real counts with the arguments checked, not by loosening to any():

  • install → exactly(3): Application create, ApplicationVersion create, Application re-save
  • publish → exactly(2): status flip, BuiltAppRoute upsert

Each individual write is now pinned by its register, schema and uuid, so the count cannot be satisfied by any three saves at all.

⚠️ The saveObject double also now echoes the persisted object back, the way OpenRegister does. Returning a bare ['uuid' => …] made the controller's read-modify-write look lossless no matter what it actually sent — which is precisely how the bug below stayed invisible.

Argument order was verified with ReflectionMethod against the interface the mock actually resolves at runtime (vendor/conduction/hydra-gates/hydra-gates/contracts/ObjectServiceInterface.php), confirmed identical to openregister's live ObjectServiceInterface. A PHPUnit mock resolves named arguments against its own signature and then invokes the callback positionally, so willReturnCallback parameters must mirror (object, extend, register, schema, uuid, …).

2. 🔴 Real bug: the productionVersion re-save wiped the Application

linkProductionVersion() patched productionVersion in with a hand-built four-field payload (slug, name, permissions, productionVersion).

OpenRegister's saveObject() is PUT, not PATCH. On the update path, SaveObject::prepareObjectForUpdate() calls fillMissingSchemaPropertiesWithNull() — which NULLs every schema property absent from the payload — and then setObject() replaces the stored data outright. There is no merge with the existing object anywhere in that path (the only carry-forward is writeOnly properties).

So that write NULLed owner, status, version and templateOrigin off the Application that persistApplication() had written one statement earlier — on every local-template, remote-registry and GitHub-shop install.

Now persistApplication() hands back the stored payload and the link step re-saves that whole object with the one field patched in, mirroring ApplicationPublishController::setStatus(). A regression test pins it, and it was confirmed to fail against the old code before being accepted.

3. phpmd: split along real seams, no suppressions

  • installFromTemplateArray() 114 lines → the clone half became materialiseApplication()
  • collectConnectors() 104 lines → collectDirectReferences() + exportReferencedConnector()

Also cleared 6 UndefinedVariable findings the CI log carried alongside the two length ones: $realUuid / $realRefUuid existed only because the by-reference out-param created them, and the $realUuid !== '' guard read a possible null the wrong way (null !== '' is true, so the identity would have been null). Plus the two phpcs errors on lines this PR introduced.

No SuppressWarnings, no threshold changes.

⚠️ Instrument note for other crews: a red PHPUnit cell that is not a test failure

After the fix, one of six PHPUnit cells stayed red while the suite printed OK (852 tests, 2690 assertions). The failing step was the coverage ratchet inside the PHPUnit job, and its verdict was self-refuting:

Changed files, head:    58.62%  (877/1496 statements)
Changed files, base:    58.62%  (809/1380 statements)
FAIL: coverage of the files this change touches dropped by 0%.

The 116 added statements were covered at 68/116 = 58.62% — exactly the file mean, as arithmetically neutral as a change can be — and it still failed, by 0.0002 percentage points.

Rather than buy that back with filler, this pins two contracts linkProductionVersion() documents but nothing tested: a failing version write must not fail the install (and must not attempt the re-save), and a version save yielding no UUID must not overwrite a working pointer with an empty one.

Measured, not assumed: 877/1496 → 883/1496, 58.6230% → 59.0241%, clearing base by 0.40pp. The local pcov run reproduced CI's xdebug figure of 877/1496 exactly, so the instrument is the one the gate uses.

Read the failing step, never the job name.

@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ c9bb330

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
check-gitignore
check-nc-floor
format
composer ✅ 106/106
npm ✅ 626/626
app:check-code ⏭️
info.xml
REUSE
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-17 03:24 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit 1196529 into development Aug 17, 2026
78 of 80 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants