Skip to content

fix: timezone parsing in str.to_datetime#4241

Merged
universalmind303 merged 7 commits intomainfrom
universalmind303/todatetime-fix
Apr 26, 2025
Merged

fix: timezone parsing in str.to_datetime#4241
universalmind303 merged 7 commits intomainfrom
universalmind303/todatetime-fix

Conversation

@universalmind303
Copy link
Copy Markdown
Member

Changes Made

changes the to_datetime kernel to try parsing with a timezone offset first as you can have a valid string with a timezone, but that would previously fail when trying to parse as seen in #4220

Related Issues

closes #4220

Checklist

  • Documented in API Docs (if applicable)
  • Documented in User Guide (if applicable)
  • If adding a new documentation page, doc is added to docs/mkdocs.yml navigation
  • Documentation builds and is formatted properly (tag @/ccmao1130 for docs review)

The commit adds explicit offset detection when parsing datetime strings to ensure
timezone-aware and naive datetimes are handled correctly.
Fix datetime parsing to check timezone offset first

The change is self-explanatory from the subject line and the logic change is straightforward, so no body is needed.
The changes improve handling of timezone offsets in datetime parsing:
- Convert dates with timezone offsets to UTC when no timezone specified
- Add helper to detect if format string contains offset characters
- Update tests to reflect UTC timezone coercion behavior
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 23, 2025

Codecov Report

Attention: Patch coverage is 81.69014% with 13 lines in your changes missing coverage. Please review.

Project coverage is 78.47%. Comparing base (a5060d4) to head (b2f09a1).
Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
src/daft-core/src/array/ops/utf8.rs 62.06% 11 Missing ⚠️
src/daft-functions/src/utf8/to_datetime.rs 83.33% 1 Missing ⚠️
src/daft-schema/src/time_unit.rs 97.22% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4241      +/-   ##
==========================================
+ Coverage   78.46%   78.47%   +0.01%     
==========================================
  Files         798      798              
  Lines      105192   105279      +87     
==========================================
+ Hits        82536    82616      +80     
- Misses      22656    22663       +7     
Files with missing lines Coverage Δ
src/daft-core/src/datatypes/mod.rs 40.00% <ø> (ø)
src/daft-functions/src/utf8/to_datetime.rs 69.23% <83.33%> (+1.14%) ⬆️
src/daft-schema/src/time_unit.rs 97.22% <97.22%> (ø)
src/daft-core/src/array/ops/utf8.rs 92.25% <62.06%> (-0.41%) ⬇️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@universalmind303 universalmind303 merged commit f56d50e into main Apr 26, 2025
55 checks passed
@universalmind303 universalmind303 deleted the universalmind303/todatetime-fix branch April 26, 2025 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

str.to_datetime ignores '%z'

2 participants