fix(pandas): version-conditional timestamp resolution for to_pandas() (ns on v2, ms on v3) - #2724
fix(pandas): version-conditional timestamp resolution for to_pandas() (ns on v2, ms on v3)#2724haakonvt wants to merge 15 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request standardizes the to_pandas export behavior across the SDK to use millisecond precision (datetime64[ms]) for timestamp and datetime columns, aligning with the Cognite API and ensuring compatibility with pandas v3. To eliminate duplicate conversion logic, CogniteResource.to_pandas now delegates to its corresponding list class via a new _LIST_CLASS class variable, with a fallback helper for resource types without a list counterpart. Custom to_pandas overrides (such as in Group and Instances) have been simplified or removed in favor of this unified delegation pattern, and meta-tests have been added to enforce the back-references between resource and list classes. No review comments were provided, so there is no feedback to assess.
797e971 to
cf6d97b
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2724 +/- ##
==========================================
- Coverage 93.71% 93.69% -0.03%
==========================================
Files 504 504
Lines 51294 51476 +182
==========================================
+ Hits 48070 48229 +159
- Misses 3224 3247 +23
🚀 New features to boost your workflow:
|
1d686d4 to
ec5f01a
Compare
ec5f01a to
f0e27e9
Compare
Pandas v3 removed the implicit nanosecond default for datetimes, so
to_pandas()output would get arbitrary precision depending on input. This pins resolution explicitly: ns on pandas v2 (matching old behavior), ms on pandas v3 (matching CDF's native resolution), via shared helpers in_pandas_helpers.pyused across every timestamp-producing code path."Single" resource objects (aka not the list version), now delegate to the list-class
to_pandasmethod to avoid having duplicated logic. This relies on_LIST_CLASS/_RESOURCEback-references staying in sync, sotest_meta.pynow enforces it: everyCogniteResourceList._RESOURCEmust point to a resource class whose_LIST_CLASSpoints right back. A class added without wiring up its counterpart now fails CI instead of silently duplicating conversion logic later. Also adds atest_standalone_to_pandas_allowlisttest documenting the deliberate exceptions (e.g.Datapoint,RowCore) that don't follow this pattern.For the pandas versioning itself, added
tests/tests_unit/test_utils/test_pandas_helpers.pywith aforce_pandas_major_versionfixture that monkeypatches the version check, so both the ns and ms branches run in every CI invocation regardless of which pandas is actually installed.