Skip to content

fix: migrate create_table's upload path to the presigned upload flow - #41

Merged
rohan-hotdata merged 3 commits into
mainfrom
fix/presigned-upload-flow
Jul 22, 2026
Merged

fix: migrate create_table's upload path to the presigned upload flow#41
rohan-hotdata merged 3 commits into
mainfrom
fix/presigned-upload-flow

Conversation

@rohan-hotdata

Copy link
Copy Markdown
Contributor

Summary

  • runtimedb PR #952 removed the legacy POST /v1/files upload endpoint in favor of the presigned direct-to-storage flow (POST /v1/uploadsPUT → finalize). hotdata-ibis's create_table/upload_file was still calling the now-dead endpoint via the SDK's generated UploadsApi.upload_file, breaking every write against any runtimedb deployment past that commit — confirmed 404 on production (api.hotdata.dev).
  • Switches to hotdata.uploads.UploadsApi, the SDK's own hand-written orchestration of the presigned flow (session create, storage PUT, finalize, with retries/multipart already built in), rather than hand-rolling it here. No new dependency, no SDK version bump needed — hotdata.uploads is already present across the whole >=0.7,<0.9 pin range.
  • One call-site change: upload["id"]upload["upload_id"] (FinalizeUploadResponse's field name differs from the old UploadResponse).

Cross-repo check

hotdata-dlt-destination only depends on hotdata-ibis for the read-side dataset().ibis() bridge — its write path goes through hotdata_framework.ManagedDatabaseClient entirely independently, so this has no surface overlap with what that repo actually uses.

Verification

  • Full offline suite passes (101 tests), ruff clean. Test fixtures updated to mock the three-stage presigned flow (new shared mock_presigned_upload_flow helper in conftest.py) instead of the single POST /v1/files call.
  • Live-verified against production (where /v1/files is confirmed 404): examples 01, 03 (read-only, unaffected), 04, 05, and the vector-search example all pass end to end, including the full create_databasecreate_table → query → drop_database cycle.

Note

This branch and #40 (vector-search helpers) both touch CHANGELOG.md's [Unreleased] section at the same anchor point — expect a small, easy textual conflict when the second one merges; no code overlap otherwise.

Test plan

  • uv run pytest
  • uv run ruff check src tests examples
  • Live run against production, where the legacy endpoint is confirmed removed

🤖 Generated with Claude Code

runtimedb PR #952 removed the legacy POST /v1/files endpoint in favor of
the presigned direct-to-storage flow (POST /v1/uploads -> PUT -> finalize).
hotdata-ibis's create_table/upload_file called the now-dead endpoint via
the SDK's generated UploadsApi.upload_file, breaking every write against
any runtimedb deployment past that commit (confirmed on production).

Switches to hotdata.uploads.UploadsApi (the SDK's own hand-written
orchestration of the presigned flow: session create, storage PUT,
finalize, with retries/multipart handling) instead of hand-rolling the
flow here. No SDK version bump needed -- hotdata.uploads is already
present across the whole >=0.7,<0.9 pin range.

upload_file's return value now carries upload_id (FinalizeUploadResponse)
instead of id (the old UploadResponse). Test fixtures updated to mock the
three-stage presigned flow instead of the single POST /v1/files call.

Verified: full offline suite (101 tests) against the updated flow, and
live against production (api.hotdata.dev, where /v1/files is confirmed
404) -- examples 01, 03, 04, 05, and the vector-search example all pass
end to end, including the create_database -> create_table -> query ->
drop_database write/read cycle.
@rohan-hotdata
rohan-hotdata requested a review from a team as a code owner July 22, 2026 04:33
@rohan-hotdata
rohan-hotdata requested review from shefeek-jinnah and removed request for a team July 22, 2026 04:33
claude[bot]
claude Bot previously approved these changes Jul 22, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, focused fix. The presigned-flow migration is correct: UploadError and ApiException both map onto HotdataAPIError (preserving the Ibis error-translation chain), the upload_id field rename matches FinalizeUploadResponse, and the test fixtures meaningfully exercise all three upload stages. LGTM.

The conflict resolution merging #40 dropped the blank line between the
Fixed and Added sections under Unreleased.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migration to the presigned upload flow is correct and cleanly scoped. Verified: no missed upload["id"] call sites, new UploadError path handled alongside ApiException, and the updated tests exercise the real SDK upload_file (only HTTP endpoints mocked) so the kwarg names, model_dump return, and upload_id key are all covered.

@rohan-hotdata
rohan-hotdata merged commit 4eb583c into main Jul 22, 2026
8 checks passed
@rohan-hotdata
rohan-hotdata deleted the fix/presigned-upload-flow branch July 22, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant