Skip to content

feat: bootstrap Atom's RBAC baseline from a YAML config file - #32

Open
JeffMboya wants to merge 14 commits into
mainfrom
feat/bootstrap-config-file
Open

feat: bootstrap Atom's RBAC baseline from a YAML config file#32
JeffMboya wants to merge 14 commits into
mainfrom
feat/bootstrap-config-file

Conversation

@JeffMboya

@JeffMboya JeffMboya commented Jul 15, 2026

Copy link
Copy Markdown

What this PR does — the short version

Adds a single YAML file that describes the initial state of an Atom deployment — admin user, tenants, service accounts, roles, permissions, tokens — and has Atom set it all up on startup.

Like Docker Compose, but for identities and access control instead of containers. Point Atom at the file with ATOM_BOOTSTRAP_FILE=/path/to/bootstrap.yaml, start it, done.

Why

To bring up a fresh Atom today you either:

  • Set one env var per admin/service password (ADMIN_SECRET, etc. — passwords only, nothing else), or
  • Call the API by hand to create every tenant, group, role, permission, and service account.

Neither is friendly for platform teams who want their deployment described in git, code-reviewed, and reproducible across environments.

A minimal example

tenants:
  - id: 33333333-3333-3333-3333-333333333333
    name: factory

entities:
  # The pre-seeded admin gets a password (replaces ADMIN_SECRET).
  - id: 00000000-0000-0000-0000-000000000001
    kind: human
    name: admin
    credentials:
      - kind: password
        secret: change-me-please

  # A device with a machine key, scoped to the factory tenant.
  - id: 22222222-2222-2222-2222-222222222222
    kind: device
    name: gateway-01
    tenant_id: 33333333-3333-3333-3333-333333333333
    credentials:
      - kind: shared_key
        key: replace-with-a-strong-secret

  # A downstream service with a pre-minted access token.
  # Paste the same string into the service's env — no separate mint step.
  - id: 11111111-1111-1111-1111-111111111111
    kind: service
    name: ingest-service
    credentials:
      - kind: access_token
        token: atom_<32-hex-id>_<64-hex-secret>

Full example lives in bootstrap.example.yaml — the file supports the whole graph: tenants, entities & credentials, resources, principal groups, object groups, permission blocks, roles, role assignments, direct policies, capabilities, and guardrails.

What operators get

Safe to re-run. Every record is keyed on a stable UUID and inserted with ON CONFLICT DO NOTHING. Bootstrapping an already-bootstrapped database is a no-op. Runtime edits never get clobbered.

Managed rows are protected. Anything created (or claimed) via the YAML is stamped managed_by='config' in the database, which means:

  • The API returns 409 / 404 on attempts to update, delete or revoke those rows.
  • Bootstrap credentials never appear in list responses. The API pretends they don't exist, so a pre-minted service token can't leak through introspection.
  • The auth path still accepts those tokens at runtime — services log in normally.

Fails loud. Structural problems (duplicate ids, shared keys on humans, invalid scope combinations, malformed access token strings) abort startup with a clear message, not a half-applied bootstrap.

Same code as the API. Password hashing, shared-key encryption, capability validation, access-token parsing — bootstrap goes through the same functions the API does, so the rows it produces are indistinguishable.

What this replaces

  • The ADMIN_SECRET / ATOM_SERVICE_SECRET env vars still work; the YAML is the successor.
  • One-shot init containers that products like magistrala shipped just to hit the API at deploy time — everything they used to do (register actions, applicability, assignment guardrails, mint service tokens) is now expressible in this YAML.

Migrations

Three new migrations, all additive:

  • 004_managed_by.sql — adds managed_by column to actions, action_applicability, action_assignment_rules.
  • 005_managed_by_identity.sql — same for entities and credentials.
  • 006_strip_product_specific_applicability.sql — removes two IoT-flavoured seed rows (publish/subscribe → resource:channel, execute → resource:rule) that leaked into 001. Downstream products supply these via their own bootstrap YAML now.

Note: we tried editing 001 in place first — sqlx refused to run against any existing database because the migration's checksum changed. Landing the delete as a new migration keeps upgrades clean.

Tests

  • Unit — YAML parsing & validation, every error path.
  • Integration (DB-gated)tests/m25_config_bootstrap.rs (full graph), tests/m26_config_managed_capabilities.rs (capabilities/guardrails + API mutation guards), tests/m27_config_managed_identity.rs (entity/credential guards + list filtering + a regression guard that confirms bootstrap access tokens still authenticate at runtime).
  • Two existing tests (m8_guardrails.rs, m13_graphql_authz_admin.rs::channel) were patched to seed publish/subscribe → resource:channel inline, since migration 006 removes that seed.

All green locally: cargo test, cargo test -- --ignored against Postgres, cargo fmt --check, cargo clippy -- -D warnings.

Notes for reviewers

  • New dep: serde_yaml = "0.9" for parsing.
  • The YAML contains secrets inline (same posture as ADMIN_SECRET today) — mount it as a Kubernetes secret or bind-mount from a protected path; keep it out of version control.
  • shared_key credentials are only valid for machine (non-human) entities.
  • access_token credentials take the full atom_<id>_<secret> string; the credential id is the upsert key so re-runs are idempotent.

Related

@JeffMboya JeffMboya changed the title feat: bootstrap Atom from a YAML config file feat: bootstrap Atom's RBAC baseline from a YAML config file Jul 15, 2026
@JeffMboya
JeffMboya force-pushed the feat/bootstrap-config-file branch from 39e81b5 to f2eef6c Compare July 15, 2026 12:39
@dborovcanin
dborovcanin requested a review from arvindh123 July 20, 2026 12:11
@dborovcanin

Copy link
Copy Markdown
Contributor

@arvindh123 Please carefully review.

@dborovcanin

Copy link
Copy Markdown
Contributor

@arvindh123 Please review.

@arvindh123

Copy link
Copy Markdown
Contributor

I'm trying to find different solution, like to avoid atom-bootstrap in MG via Config
I will propose something here in few minutes

JeffMboya and others added 4 commits August 4, 2026 13:48
Provision the initial entities and their password/shared-key
credentials at startup from a YAML file (ATOM_BOOTSTRAP_FILE) instead
of setting one *_SECRET env var per identity or driving the API by
hand. The file is loaded once after migrations and is idempotent:
existing entities and credentials are never mutated, so re-running is a
no-op, and it runs alongside the existing env-var bootstrap.

Credential creation reuses the identity service, so hashing,
password-strength validation and shared-key envelope encryption are
identical to the API path. Structural validation (duplicate ids, human
shared keys, multiple credentials of a kind, non-object attributes)
runs before touching the database, so a malformed file aborts startup
cleanly.

Closes #27
Extend the YAML bootstrap beyond entities and credentials to the full
RBAC baseline: tenants, principal groups (with members), permission
blocks (with actions and scope), roles (linked to blocks), role
assignments and direct policies. Entities can now also declare their
owning tenant.

Sections are applied in dependency order and every record is keyed on a
stable UUID and inserted with ON CONFLICT DO NOTHING, so the whole graph
stays idempotent and safe to re-run. Records may reference rows that
already exist in the database. Structural validation (unique ids per
section, scope/mode column combinations mirroring the DB constraint,
object attributes) runs before any write.
Add `resources` and `object_groups` sections so the config file can
provision every protected-object kind, not just the identity/RBAC graph.
Resources carry kind/name/alias/tenant/owner; object groups group
entities and resources (and nest via `parent`) so a permission block can
scope to their members.

Permission-block scopes gain the group-relative modes
(group_direct_objects, group_descendant_objects, group_child_groups,
group_descendant_groups) via `scope.group_id`. The *_objects modes
require object_type (the scope_ref is `<group>:<object_type>`), validated
up front so a scope can't be silently dead. Applied in dependency order
after entities and before permission blocks; all idempotent via ON
CONFLICT DO NOTHING.
Signed-off-by: Arvindh <arvindh91@gmail.com>
@arvindh123
arvindh123 force-pushed the feat/bootstrap-config-file branch 3 times, most recently from 606feba to 5910b71 Compare August 6, 2026 19:49
…tion

Extend the config-file bootstrap pattern (introduced for capabilities and
guardrails) to entities and credentials. Operators can now pre-provision
machine access tokens declaratively — pasting the same atom_<id>_<secret>
string into both the bootstrap YAML and the env file consumed by downstream
services — so a stack can come up without a separate token-minting step.

New AccessToken variant on BootstrapCredential takes the full token string,
parses it via the existing auth::parse_api_key, hashes the secret with the
deployment KEK (Argon2 fallback), and inserts the credential row directly.
Unscoped, no expiry, keyed on the credential id so re-runs are idempotent.

Rows created (or already present and named) via bootstrap are stamped
managed_by='config' on entities and credentials (migration 005). Two new
guards enforce the invariant across the identity surface:

- Entities: update_entity, delete_entity, restore_entity reject with 409
  managed by the bootstrap config file.
- Credentials: revoke_credential, reveal_shared_key, revoke_access_token,
  and replace_access_token_permissions return not_found — the API pretends
  bootstrap-provisioned credentials do not exist, so operator-planted tokens
  can never surface through introspection.
- list_credentials and list_access_tokens filter managed_by IS NULL, so
  bootstrap credentials do not appear in list responses either.

The auth path (auth::auth_from_api_key) does *not* filter on managed_by;
runtime authentication with bootstrap tokens still works — a regression
guard in m27::bootstrap_access_token_authenticates_at_runtime locks that in.

Also patches tests/m13_graphql_authz_admin.rs::channel to seed
publish/subscribe -> resource:channel applicability inline, matching the
earlier tests/m8_guardrails.rs fix — product-specific applicability is no
longer seeded by migration 001, so tests that model a channel must declare
it themselves.

Signed-off-by: Arvindh <arvindh91@gmail.com>
Reverse the in-place edit to migrations/001_initial.sql (which stripped
the seeded publish/subscribe -> resource:channel and execute -> resource:rule
applicability rows) and land the same deletion in a new migration 006.

Sqlx checksums every applied migration and refuses to run against a
database whose recorded checksum for an already-applied migration does not
match the file. So the earlier in-place edit broke the upgrade path: any
existing atom deployment would crash-loop on the next image pull with
"migration 1 was previously applied but has been modified". Fresh
deployments were fine, which is how the earlier PR passed CI.

Migration 006 does the deletion cleanly for both cases:
- Existing deployments: 001 checksum unchanged, no crash; 006 removes the
  two rows.
- Fresh deployments: 001 seeds the rows, 006 deletes them a few statements
  later — a few wasted inserts, no functional change.

Downstream products still supply their own applicability via the
bootstrap YAML `capabilities` block (magistrala already does this).

Signed-off-by: Arvindh <arvindh91@gmail.com>
Signed-off-by: Arvindh <arvindh91@gmail.com>
clippy::needless_borrows_for_generic_args fires on `.bind(&rule.decision)`
in both the insert and stamp queries of `ensure_action_assignment_rule`.
`ActionAssignmentDecision` derives `Copy`, so `.bind` takes it by value.

Local rustc didn't flag it (older toolchain / older clippy); the ubuntu
runner's newer clippy did, breaking `cargo clippy -- -D warnings` on the
PR CI.

Signed-off-by: Arvindh <arvindh91@gmail.com>
Main landed `004_event_outbox.sql` (commit 9efa45f, PR #41) while this
branch's `004_managed_by.sql` also holds version 4. sqlx tracks migrations
by version number as the primary key of `_sqlx_migrations`, so applying
both against the same database fails with:

    duplicate key value violates unique constraint "_sqlx_migrations_pkey"
    Key (version)=(4) already exists.

That's the failure the CI test job hit — every DB-backed unit test in
the compiled binary tried to run migrations on a fresh Postgres and
crashed at 004.

Renumber this branch's three migrations one slot up so they sit after
the newly-added event_outbox migration:

  004_managed_by.sql                          -> 005_managed_by.sql
  005_managed_by_identity.sql                 -> 006_managed_by_identity.sql
  006_strip_product_specific_applicability.sql -> 007_...

The renumbering is safe because these migrations are only referenced by
their file paths (never by version number in code), and only ever ran
against fresh test databases so far (CI uses a fresh Postgres service
container per run). No production deployment ever recorded version 4 or
5 as "managed_by" — event_outbox will now claim 4 cleanly.

Also updated one doc comment in m27 that referred to "migration 005".

Signed-off-by: Arvindh <arvindh91@gmail.com>
…platform-resource test

The lifecycle-deny sibling tests in this file all pass because a frozen /
inactive / deleted tenant short-circuits the PDP before applicability is
checked. `platform_resource_unaffected_by_tenant_lifecycle` has no tenant
to short-circuit on — the request must reach the applicability check —
and migration 007 removed the seeded `publish -> resource:channel`
applicability row that this test relied on.

Add the applicability inline (mirrors the m8/m13 fixes), so the test
stays product-agnostic without regressing.

Signed-off-by: Arvindh <arvindh91@gmail.com>
… hiding

Previously credentials provisioned from the bootstrap YAML were hidden
entirely from the API — list_credentials / list_access_tokens filtered
them out and revoke returned 404. That was over-cautious: list responses
only carry metadata (id, kind, identifier, status, timestamps) with no
secret material, so hiding those rows just left operators unable to see
what tokens their services were using.

Change credentials to match how entities, capabilities, applicability
and guardrails already behave:

- list_credentials / list_access_tokens no longer filter managed_by;
  every entry now carries `managed_by: Option<String>`.
- revoke_credential / revoke_access_token / replace_access_token_permissions
  return 409 conflict ("managed by the bootstrap config file") instead of
  404 not_found.
- reveal_shared_key stays at 404: that is the one endpoint that returns
  the plaintext key material, and the operator's declared key must not
  leak through introspection.

Also plumb `managed_by` through the GraphQL response types so the UI can
render it: Entity, Capability, CapabilityApplicability,
CapabilityApplicabilityEntry, ActionAssignmentRule, Credential,
AccessToken all gained a `managedBy: String | null` field.

Model structs carry `#[sqlx(default)]` on managed_by so RETURNING clauses
and other SELECTs that omit the column still hydrate cleanly; the
list/get read paths that surface the flag to the UI explicitly include
it in their SQL. Follow-up: extend the marker to roles, permission
blocks, and other bootstrap-created rows when the bootstrap layer starts
stamping them (out of scope here).

--- UI ---

Point the admin UI at the new flag:

- New `components/crud/managed-by-badge.tsx` renders a small "Config"
  badge (with lock icon) when a row has `managedBy === 'config'`, and
  exports a shared `isConfigManaged` predicate + tooltip string.
- `components/crud/table/utils.ts` gains `isConfigManagedRow` mirroring
  the existing `isDeletedRow` pattern.
- `TableRowActions` in `components/crud/crud-table.tsx` returns Inspect-
  only for config-managed rows, hiding every mutation button, exactly
  the way deleted rows already show Inspect + Restore/Purge only.
- Added `managedBy` to the six GraphQL queries the UI runs: entities,
  actions, action-applicability, action-assignment-rules, credentials,
  access tokens.
- Added a `managedBy` column to the four crud-table resources; badge is
  the null-safe renderer, so API-managed rows show nothing at all.
- In the entity-detail credentials sub-panel, config-managed rows show
  the badge and hide their revoke/reveal/renew/download buttons.

--- Tests ---

`tests/m27_config_managed_identity.rs::bootstrap_access_token_is_visible_read_only`
flipped from asserting "hidden + not_found" to "visible + 409 conflict".
The auth-path regression test still verifies bootstrap tokens
authenticate at runtime.

Verified locally: cargo check --tests + cargo test -- --ignored (except
the AMQP-broker tests in m27_live_amqp_delivery which CI already skips),
pnpm tsc --noEmit, pnpm biome check on the touched UI files.

Signed-off-by: Arvindh <arvindh91@gmail.com>
Previously only `entities`, `credentials`, `actions`, `action_applicability`
and `action_assignment_rules` were stamped `managed_by='config'`; anything
else the bootstrap YAML created (tenants, resources, groups, roles,
permission blocks, role assignments, direct policies) could still be
edited or deleted through the API, which contradicted the flag's whole
point.

Extend the marker end-to-end to those eight tables so the whole graph
the operator declares in YAML is API-immutable.

**Migration** — `008_managed_by_rbac.sql` adds `managed_by TEXT CHECK
(managed_by IS NULL OR = 'config')` to tenants, resources,
principal_groups, object_groups, roles, permission_blocks,
role_assignments, direct_policies. Recreates the `groups` view (which
unions principal_groups + object_groups) so the unified read path carries
the column.

**Bootstrap** — new shared helper `stamp_managed_by_config(pool, table, id)`
called at the end of every ensure_* function that touches one of the
above tables. Table names are matched against a closed static list so
callers can't inject arbitrary SQL. Rows that already exist (from the
initial migration or a prior manual insert) get stamped on the follow-up
UPDATE, so protection is retroactive.

**Guards** — new top-level module `src/managed_by.rs` exports
`ensure_not_config_managed(pool, table, id)` which every mutation entry
point calls at the top:

- `tenants::repo::update_tenant_with_audit`
  and `soft_delete_tenant_with_audit`
  and `restore_tenant_with_audit`
- `authz::repo::update_resource_with_audit`
  and `delete_resource_with_audit`
  and `restore_resource_with_audit`
- `identity::repo::update_group_with_audit`
  and `delete_group_with_audit`
  and `restore_group_with_audit`
  (via the `groups` view since a group id resolves to either
  principal_groups or object_groups)
- `authz::repo::delete_role_with_audit`
  and `restore_role_with_audit`
- `authz::repo::delete_permission_block_with_audit`
- `authz::repo::delete_role_assignment_with_audit`
- `authz::repo::delete_direct_policy_with_audit`

Config-managed rows now reject those calls with 409 conflict ("managed
by the bootstrap config file"), consistent with the entity/capability
guards from earlier commits.

**Read paths** — model structs (Tenant, Resource, Group, Role,
PermissionBlock, RoleAssignment, DirectPolicy) all gained
`#[sqlx(default)] pub managed_by: Option<String>`. The list/get SQL
statements those types run were updated to include `managed_by` in the
SELECT column list. `sqlx(default)` keeps RETURNING clauses that omit
the column working (they just yield None).

**GraphQL** — each of the seven wrapper types in
`src/graphql/types/mod.rs` gained an `async fn managed_by(&self) ->
Option<&str>` resolver so the UI can read the flag through every list
and detail query it already runs.

**UI** — `app/lib/crud/resources.ts` gained `managedBy` in the list
queries and `{ key: "managedBy", label: "Managed", priority: "medium" }`
column config for tenants, resources, groups, roles, permission-blocks,
policies. The badge (`components/crud/managed-by-badge.tsx`) already
renders null-safe, and `TableRowActions` already returns Inspect-only
for config-managed rows — see the previous commit.

Verified locally: cargo check --tests, cargo test -- --ignored (except
the AMQP-broker tests in m27_live_amqp_delivery which CI already skips),
pnpm biome check on the touched UI files.

Signed-off-by: Arvindh <arvindh91@gmail.com>
CI's cargo fmt --check flagged formatting drift in the two functions added
in 5bc59ed. Auto-applied.

Signed-off-by: Arvindh <arvindh91@gmail.com>
@arvindh123
arvindh123 force-pushed the feat/bootstrap-config-file branch from 5910b71 to 8f004bc Compare August 6, 2026 19:52
Signed-off-by: Arvindh <arvindh91@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Bootstrap ATOM with configuration file

3 participants