Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL: make date format functions more strict #112140

Merged
merged 3 commits into from
Aug 26, 2024

Conversation

luigidellaquila
Copy link
Contributor

Switching to CLDR locale provider (see #110222) exposed some problems related to date conversion functions.
In particular, these functions are defined to mimic MySQL and PostgreSQL behavior, and tests that compare results with the corresponding queries in MySQL and PostgreSQL show a slightly different behavior with CLDR, eg.

  • days of week are returned as Mon instead of Monday with the ROOT locale;
  • calculation of week of the year considers slightly different defaults (eg. minimum number of days in the year for the first week).

This change makes the calculation more strict by

The problem is not present with COMPAT locale provider, the intention of this change is to fix failures introduced by #110222, but I'm doing this as a separate PR to make sure that the new implementation is also good with COMPAT and we are not introducing regressions (ie. that it works fine with both COMPAT and CLDR)

.build(),
new Builder().pattern("%u").javaFormat(t -> String.format(Locale.ROOT, "%02d", t.get(WeekFields.ISO.weekOfYear()))).build(),
new Builder().pattern("%u")
.javaFormat(t -> String.format(Locale.ENGLISH, "%02d", t.get(WeekFields.of(DayOfWeek.MONDAY, 4).weekOfYear())))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly how MySQL spec defines %u

.javaFormat(t -> String.format(Locale.ENGLISH, "%02d", t.get(WeekFields.of(DayOfWeek.SUNDAY, 7).weekOfWeekBasedYear())))
.build(),
new Builder().pattern("%v")
.javaFormat(t -> String.format(Locale.ENGLISH, "%02d", t.get(WeekFields.of(DayOfWeek.MONDAY, 4).weekOfWeekBasedYear())))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly how MySQL spec defines %v

@luigidellaquila luigidellaquila marked this pull request as ready for review August 23, 2024 10:47
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Aug 23, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@luigidellaquila
Copy link
Contributor Author

Wait for #112196 before merging

Copy link
Contributor

@bpintea bpintea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏

@luigidellaquila
Copy link
Contributor Author

@elasticmachine update branch

@luigidellaquila luigidellaquila merged commit de73cda into elastic:main Aug 26, 2024
15 checks passed
@thecoop
Copy link
Member

thecoop commented Sep 4, 2024

@luigidellaquila Is it ok to backport this to 8.15 and 7.17?

@luigidellaquila
Copy link
Contributor Author

@thecoop For 8.15 I don't see any problems.
For 7.17 I think we'll have to do a partial backport (I'm pretty sure DATE_FORMAT function was never backported to 7.x)

@luigidellaquila
Copy link
Contributor Author

I can take care of it tomorrow

cbuescher pushed a commit to cbuescher/elasticsearch that referenced this pull request Sep 4, 2024
thecoop pushed a commit to thecoop/elasticsearch that referenced this pull request Sep 5, 2024
thecoop added a commit that referenced this pull request Sep 5, 2024
luigidellaquila added a commit to luigidellaquila/elasticsearch that referenced this pull request Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/SQL SQL querying >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants