Features for working with synthetitc data in sandbox

.specify/feature.json

@@ -1,3 +1,3 @@
 {
-  "feature_directory": "specs/001-api-safety-modes"
-}
+  "feature_directory": "specs/002-daily-data-pipeline"
+}

agent-docs/current-work.md

@@ -2,6 +2,46 @@
 
 This file is the current milestone and handoff note for coding agents. It should stay short and should not duplicate the long-term roadmap in [../docs/roadmap.md](../docs/roadmap.md).
 
+## 001 Daily Data Pipeline (Implemented 2026-08-12)
+
+The sibling `ams-sandbox-data-synthesis` repository now contains the stateful,
+append-only daily pipeline described by `specs/002-daily-data-pipeline`.
+Applicable connector aims are B2, B4, B6-B8, C1-C3, C5-C8, E1, E4, E6-E9,
+F3, F5, F7, and F8; the highest risk remains **High** because a separate local
+interactive command can insert sandbox events.
+
+Implemented boundaries:
+
+- deterministic date transitions with immutable hash-linked checkpoints,
+  missing-date catch-up, stable IDs, weekly context, pending events, and
+  effective-dated roster cutovers;
+- one local CLI for WSV, KNLTB, or both, including read-only remote
+  classification, immutable daily/master upload plans, status, and reporting;
+- a protected-Volume/Delta Databricks adapter and a paused serverless
+  prepare-only job; scheduled publication is deliberately absent; and
+- interactive event-only publication with a fresh complete scope read, exact
+  target/plan/hash checks, a real-TTY typed phrase, single-use authorization,
+  no mutation retry, and exact read-only reconciliation.
+
+Verification evidence:
+
+- 47 daily-focused synthesis tests pass after final safety changes;
+- the complete connector suite passes 147 tests with 1 opt-in live test skipped;
+- the synthesis wheel and connector wheel build, the synthesis wheel contains
+  all required root modules/package files and no state or credential files, and
+  an external wheel-path import smoke test passes;
+- CLI help, form-map JSON, Databricks YAML structural safeguards, Python
+  compilation, task format/completion, and scoped `git diff --check` pass; and
+- the full sibling synthesis suite runs 188 tests; 186 pass and two pre-existing
+  refresh tests error because current ignored template/form-map fixtures do not
+  match their assumptions. Neither failing test touches the new daily modules.
+
+No live AMS request, mutation, Databricks bundle validation, deployment, or
+schedule activation was performed. The first recommended next action is the
+README's local WSV-yesterday `init`/`prepare --remote-check skip` smoke test,
+followed by the read-only `--remote-check require` step under an authorized
+operator account.
+
 ## Current Milestone
 
 The next major workflow is a sandbox-safe synthetic-data pipeline:
@@ -169,6 +209,18 @@ or on an independent machine.
 
 ## Recent Session Summary
 
+2026-08-06:
+
+- Replaced the workflow tests' dependency on ignored local
+  `use_case_examples/synthetic_data` files with a committed, deterministic
+  fixture pair under `tests/fixtures/workflow`.
+- The fixture contains fictional test-only values and is passed explicitly to
+  `load_example_event_workflow_input()`; the ignored operational example
+  directory remains excluded from version control.
+- The 13 previously known missing-fixture failures are resolved. All 21
+  workflow tests pass, and the full connector suite passes 144 tests with the
+  opt-in live test skipped. No live AMS request or mutation was performed.
+
 2026-08-05:
 
 - Corrected roster parsing for the official nested user-result batch shape.

agent-docs/proposed-skill-api-use.md

@@ -0,0 +1,280 @@
+skill idea ams-api
+---
+name: ams-connector-review
+description: Review and improve changes to the AMS Python Connector. Use this skill whenever working on the ams-python-connector repository, reviewing pull requests, implementing new API endpoints, fixing bugs, improving robustness, adding tests, refactoring, or planning changes. Prioritize API correctness, backwards compatibility, safe handling of Smartabase data, thorough testing, small reviewable commits, and collaborative Git workflows over generating large amounts of code.
+---
+
+# AMS Connector Review
+
+The objective of this skill is to help maintain a robust, maintainable Python client for Teamworks AMS / Smartabase while encouraging good collaborative development practices.
+
+Assume this repository is a shared project where code quality is more important than writing code quickly.
+
+## Overall priorities
+
+Always optimize for the following order:
+
+1. Correctness
+2. Safety
+3. Testability
+4. Simplicity
+5. Readability
+6. Performance
+
+Never sacrifice correctness for cleverness.
+
+---
+
+# Development workflow
+
+Before making changes:
+
+- Read the existing implementation.
+- Understand how the feature currently works.
+- Avoid unnecessary rewrites.
+- Preserve the public API unless a breaking change is explicitly requested.
+
+If requirements are unclear:
+
+Ask questions instead of making assumptions.
+
+---
+
+# Code review checklist
+
+Review every proposed change for:
+
+## Correctness
+
+- Does the implementation satisfy the requested behavior?
+- Are edge cases handled?
+- Are failure cases considered?
+
+## Robustness
+
+Look specifically for:
+
+- missing timeout handling
+- authentication failures
+- malformed responses
+- missing fields
+- null values
+- pagination
+- rate limiting
+- retries where appropriate
+- duplicate requests
+- idempotency
+
+Prefer graceful failure over unexpected exceptions.
+
+---
+
+## Security
+
+Never expose:
+
+- passwords
+- API keys
+- tokens
+- session cookies
+
+Ensure:
+
+- secrets come from environment variables
+- logs do not contain credentials
+- destructive operations require explicit confirmation
+- sandbox protections remain intact unless intentionally changed
+
+Never recommend disabling security safeguards merely for convenience.
+
+---
+
+## Backwards compatibility
+
+Public interfaces should remain stable whenever possible.
+
+Avoid changing:
+
+- function names
+- parameter names
+- return structures
+
+If a breaking change is unavoidable:
+
+- explain why
+- document migration steps
+- identify downstream impact
+
+---
+
+# Testing
+
+Every bug fix should include a regression test whenever practical.
+
+Prefer automated tests over manual verification.
+
+When reviewing tests, check that they include:
+
+- success cases
+- expected failures
+- invalid input
+- empty responses
+- unexpected API responses
+- authentication failures
+
+If a change cannot easily be tested automatically, explain why.
+
+---
+
+# API design
+
+Encourage:
+
+- descriptive exceptions
+- meaningful error messages
+- small focused methods
+- minimal side effects
+
+Avoid:
+
+- deeply nested logic
+- duplicated code
+- hidden state
+- silent failures
+
+---
+
+# Pull requests
+
+Prefer:
+
+- one feature per pull request
+- one logical change per commit
+- descriptive commit messages
+
+If a proposed change is too large:
+
+Suggest splitting it into multiple pull requests.
+
+---
+
+# Review comments
+
+When reviewing code:
+
+Explain:
+
+- what is wrong
+- why it matters
+- how it could be improved
+
+Do not simply rewrite code unless necessary.
+
+Teach as well as review.
+
+---
+
+# Collaboration
+
+Assume multiple developers are working simultaneously.
+
+Encourage:
+
+- small commits
+- frequent pushes
+- draft pull requests
+- discussion before large refactors
+
+Avoid recommending force pushes unless absolutely necessary.
+
+---
+
+# Refactoring
+
+Only recommend refactoring when it clearly improves:
+
+- readability
+- maintainability
+- robustness
+- testability
+
+Avoid cosmetic rewrites that increase review effort.
+
+---
+
+# Documentation
+
+Whenever behavior changes:
+
+Recommend updating:
+
+- README
+- examples
+- docstrings
+- changelog if applicable
+
+Documentation should describe *why* something exists, not merely repeat the code.
+
+---
+
+# Response format
+
+When reviewing code, structure the response as follows:
+
+## Summary
+
+One paragraph describing the overall quality.
+
+## Strengths
+
+Bullet list.
+
+## Issues
+
+Order by severity:
+
+- Critical
+- High
+- Medium
+- Low
+
+For each issue include:
+
+- problem
+- reason
+- suggested improvement
+
+## Testing
+
+Describe:
+
+- existing coverage
+- missing tests
+- recommended new tests
+
+## Merge readiness
+
+End with one of:
+
+- Ready to merge
+- Ready after minor fixes
+- Needs additional work
+- Do not merge yet
+
+Always explain the recommendation.
+
+---
+
+# Philosophy
+
+The goal is not simply to make the code work.
+
+The goal is to leave the repository in a better state after every change.
+
+Prefer maintainable code over clever code.
+
+Prefer explicit behavior over hidden magic.
+
+Prefer automated tests over manual confidence.
+
+Prefer small improvements made consistently over large disruptive rewrites.