feat: kbc→kbagent CI/CD migration skill + sync CLI behavior tests by Matovidlo · Pull Request #402 · keboola/cli

Matovidlo · 2026-06-09T13:04:42Z

Summary

Adds the kbagent-cicd-migration skill (plugins/kbagent/skills/kbagent-cicd-migration/): a guided, evidence-based runbook plus a stdlib-only generator (scripts/migrate_cicd.py) that migrates a kbc (keboola-as-code) project-as-code GitHub CI/CD pipeline to the new kbagent sync engine.
- The generator discovers every project from .keboola/manifest.json, detects the legacy kbc workflows/actions it supersedes, and emits clean kbagent-native validate / pull / push GitHub workflows using uv tool install + kbagent sync with the KBAGENT_PROJECT_FROM_ENV=1 auth model (per-project token secrets, GitHub-Environment approval gating, no committed tokens).
- References cover the kbc↔kbagent command/flag/env mapping, GitHub secrets/environments setup, the one-time breaking-conversion runbook, and the single-branch vs git-branching decision guide.
Adds sync CLI behavior tests (tests/test_sync_cli_behavior.py): pin the project-selection guards that define kbagent's orchestrator model (--project/--all-projects required and mutually exclusive, --branch is per-project) and that push --dry-run propagates the dry-run flag without writing.

Why

Customers run kbc-based GitHub CI/CD (per-project pull/push PRs, multi-project promotion). The new CLI's sync is a manifest-compatible successor but not a drop-in: the on-disk format differs (config.json/meta.json → _config.yml) and sync is an orchestrator over registered project aliases, not cwd-per-folder. This skill gives a safe, repeatable cutover, and the tests lock the guard behaviors that prevent wrong-target or whole-tree operations.

Validated live against a real project: native pull round-trips to zero drift; the adopt-existing → diff "136 to delete" footgun and the orphaned-config.json cleanup are documented in the runbook.

Change type

Feature — migration tooling + tests. No source/CLI-command changes, no version bump.

Impact analysis

New files only: one skill tree under plugins/kbagent/skills/ and one test file under tests/.
No changes to src/, no new CLI commands, no public API or behavior change.
Fully backwards-compatible.

Test plan

tests/test_sync_cli_behavior.py — 6 tests, green.
Local gates pass: ruff lint + format, ty (no new diagnostics), command-sync-check (229 commands, none added), skill-check (main SKILL.md unchanged), changelog-check.
Generator exercised against the CLI-based-sync-demo repo (2 projects discovered, 3 legacy actions flagged, 3 valid workflows generated).

Deployment

Merge & automatic deploy. No migration.

Rollback plan

Revert of this PR.

…ry-run safety Pin the behaviors that make sync safe and that distinguish kbagent's orchestrator model from kbc's cwd-per-folder model: sync pull/diff/push require --project or --all-projects (and the two are mutually exclusive), --branch is per-project, and push --dry-run propagates the dry-run flag without writing. 6 tests via CliRunner with a mocked SyncService. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Adds the kbagent-cicd-migration skill: a guided, evidence-based runbook plus a stdlib-only generator (scripts/migrate_cicd.py) that discovers projects from .keboola/manifest.json, detects the legacy kbc CI it replaces, and emits clean kbagent-native validate/pull/push GitHub workflows using uv tool install + kbagent sync with the KBAGENT_PROJECT_FROM_ENV auth model. References cover the kbc<->kbagent command/flag/env mapping, GitHub secrets/environments setup, the one-time breaking-conversion runbook (verified against a live project: the adopt-existing 136-delete footgun and the orphaned config.json cleanup), and the single-branch vs git-branching decision guide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Matovidlo · 2026-06-09T13:04:53Z

@claude review

Copilot

Pull request overview

This PR adds a new kbagent CI/CD migration skill (docs + a stdlib-only workflow generator) intended to help customers migrate legacy kbc GitHub Actions pipelines to kbagent sync, and it adds behavioral CLI tests to lock down key sync safety guards (explicit project selection and dry-run propagation).

Changes:

Added kbagent-cicd-migration skill documentation + reference materials under plugins/kbagent/skills/.
Added scripts/migrate_cicd.py generator to detect legacy kbc workflows and emit kbagent-native validate/pull/push workflows.
Added tests/test_sync_cli_behavior.py to pin CLI guard behavior for project selection and push --dry-run.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
`tests/test_sync_cli_behavior.py`	Adds CLI behavior regression tests for project selection guards and dry-run propagation.
`plugins/kbagent/skills/kbagent-cicd-migration/SKILL.md`	Migration skill runbook explaining the breaking JSON→YAML conversion and CI/CD cutover steps.
`plugins/kbagent/skills/kbagent-cicd-migration/scripts/migrate_cicd.py`	Analyzer + generator that discovers projects and emits GitHub Actions workflows + secrets checklist.
`plugins/kbagent/skills/kbagent-cicd-migration/references/secrets-setup.md`	Documents GitHub secrets/environments setup for the generated workflows.
`plugins/kbagent/skills/kbagent-cicd-migration/references/migration-runbook.md`	Ordered PR sequence / operational runbook for a safe migration.
`plugins/kbagent/skills/kbagent-cicd-migration/references/command-mapping.md`	Reference mapping between `kbc` and `kbagent` commands/flags/env vars.
`plugins/kbagent/skills/kbagent-cicd-migration/references/branching-model.md`	Decision guide for single-branch vs git-branching workflow models.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        f"          KBC_TOKEN: ${{{{ secrets.{p.token_secret} }}}}\n"
+        f"          KBC_STORAGE_API_URL: {p.stack_url}\n"
+        "        run: |\n"
+        f"          kbagent sync init --adopt-existing --project __env__ --directory '{p.directory}' || true\n"


+
+
+def gen_validate(projects: list[Project], main_branch: str) -> str:
+    diff_steps = "".join(_project_step(p, "diff --json", f"Diff {p.directory}") for p in projects)


+        f"          KBC_STORAGE_API_URL: {p.stack_url}\n"
+        "        run: |\n"
+        f"          kbagent sync init --adopt-existing --project __env__ --directory '{p.directory}' || true\n"
+        f"          kbagent sync {command} --project __env__ --directory '{p.directory}'\n"


+    # expression maps it to the --allow-delete flag (opt-in deletion of remote
+    # configs that were removed locally).
+    delete_expr = "${{ github.event.inputs.allow_delete == 'true' && '--allow-delete' || '' }}"


+KBAGENT_PROJECT_FROM_ENV=1 KBC_TOKEN=$L0_TOKEN KBC_STORAGE_API_URL=https://connection.keboola.com \
+  kbagent sync pull --project __env__ --directory L0
+# Remove orphaned kbc JSON files that kbagent no longer reads:
+find L0 -name config.json -o -name meta.json | xargs git rm --cached --ignore-unmatch


+| `kbc persist -d DIR` | *(folded into `sync pull`)* | No separate persist step; pull writes manifest + new objects |
+| `kbc pull -d DIR --force` | `kbagent sync pull --directory DIR --force` | `--force` overrides local-vs-remote conflicts (3-way diff) |
+| `kbc push -d DIR` | `kbagent sync push --directory DIR` | Encrypts `#`-secrets fail-closed before write |
+| `kbc push -d DIR --force` | `kbagent sync push --directory DIR --allow-delete` | `--allow-delete` removes remote configs deleted locally |


+| `kbc push -d DIR` | `kbagent sync push --directory DIR` | Encrypts `#`-secrets fail-closed before write |
+| `kbc push -d DIR --force` | `kbagent sync push --directory DIR --allow-delete` | `--allow-delete` removes remote configs deleted locally |
+| `kbc push --dry-run` / push-dry action | `kbagent sync push --dry-run --directory DIR` | Shows planned changes without writing |
+| `kbc diff -d DIR` | `kbagent sync diff --directory DIR [--json]` | `--json` gives structured drift for CI gating |


+`kbagent` parses both (`sync/manifest.py:120`). Additionally, `sync pull` flags
+`--skip-storage` / `--skip-jobs` / `--with-table-samples` control how much
+*metadata* (beyond configs) is pulled — orthogonal to the config subset.


+    )
+
+
+def gen_push(projects: list[Project], main_branch: str, git_branching: bool) -> str:


+>   local deletion. **A `sync push --allow-delete` here would delete all 136 remote
+>   configs.** Adopt-existing adopts only the manifest, NOT the configs.


Matovidlo and others added 2 commits June 9, 2026 15:03

Matovidlo requested a review from Copilot June 9, 2026 13:04

Copilot started reviewing on behalf of Matovidlo June 9, 2026 13:05 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: kbc→kbagent CI/CD migration skill + sync CLI behavior tests#402

feat: kbc→kbagent CI/CD migration skill + sync CLI behavior tests#402
Matovidlo wants to merge 2 commits into
mainfrom
martinvasko-kbc-to-kbagent-cicd-migration

Matovidlo commented Jun 9, 2026

Uh oh!

Matovidlo commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants



		def gen_validate(projects: list[Project], main_branch: str) -> str:
		diff_steps = "".join(_project_step(p, "diff --json", f"Diff {p.directory}") for p in projects)

		)


		def gen_push(projects: list[Project], main_branch: str, git_branching: bool) -> str:

		> local deletion. **A `sync push --allow-delete` here would delete all 136 remote
		> configs.** Adopt-existing adopts only the manifest, NOT the configs.

Conversation

Matovidlo commented Jun 9, 2026

Summary

Why

Change type

Impact analysis

Test plan

Deployment

Rollback plan

Uh oh!

Matovidlo commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants