gh-53502: add a new option aware_datetime in plistlib to loads or dumps aware datetime. #113363

aisk · 2023-12-21T15:49:15Z

Added a new option aware_datetime in load / dump and related functions, and the default value is False. In this case, there is no behavior changes from old versions of this library for compatible issue.

If set aware_datetime to True, the load and related functions will create datetime with tzinfo=UTC. The dump and related functions will convert the datetime to UTC bebore dump.

Issue: plistlib should create non-naïve datetime objects #53502

serhiy-storchaka

Please update the documentation, add versionchanged directives and a What's New entry.

Add tests for serializing a datatime object with UTC timezone (it should never be skipped), a naive datatime object with aware_datetime=True and an aware datatime object with aware_datetime=False (if it was not already covered).

ronaldoussoren · 2023-12-23T12:48:38Z

My first thought was: is the new parameter needed when writing plist files. And indeed, it is: we currently completely ignore the tzinfo.

That's something we can warn users about, but that's for a different PR.

aisk · 2023-12-24T13:38:19Z

Please update the documentation, add versionchanged directives and a What's New entry.

Add tests for serializing a datatime object with UTC timezone (it should never be skipped), a naive datatime object with aware_datetime=True and an aware datatime object with aware_datetime=False (if it was not already covered).

Tests and documents have been added, except for writing a aware datetime with aware_datetime=False in FMT_BINARY, because current implementation will call - operator with the input (a aware datetime) and a hard coded naive datetime in

cpython/Lib/plistlib.py

Line 781 in 0c57454

f = (value - datetime.datetime(2001, 1, 1)).total_seconds()

.This will cause a TypeError.

Therefore, the current code does not support write a aware datetime with FMT_BINARY. And I think maybe we shouldn't fix it, because serilizing a datetime by ignoring it's timezone information will often leads to bugs.

aisk · 2023-12-24T13:44:53Z

My first thought was: is the new parameter needed when writing plist files. And indeed, it is: we currently completely ignore the tzinfo.

That's something we can warn users about, but that's for a different PR.

Can we raise a DeprecatedWarning with messages like 'specify aware_datetime=False (the default value) will be deprecated in the future' if aware_datetime=False for both read and write operations?

serhiy-storchaka

Some tests are duplicated. Could you use a loop over ALL_FORMATS instead? TestPlistlib contains tests applicable for all formats, TestBinaryPlistlib only contains tests specific for binary format.

Doc/library/plistlib.rst

Lib/test/test_plistlib.py

serhiy-storchaka · 2023-12-24T14:22:17Z

Tests and documents have been added, except for writing a aware datetime with aware_datetime=False in FMT_BINARY, because current implementation will call - operator with the input (a aware datetime) and a hard coded naive datetime in

TypeError is also result. We should test that this is rejected in binary format rather than producing unexpected result.

Emitting a DeprecatedWarning is a good ide, but I think that it should be in other issue, and to be nice to users, it is better to wait few versions before adding runtime warning, so silencing it by passing aware_datetime=False would work on several maintained version.

aisk · 2023-12-26T09:28:12Z

@serhiy-storchaka thanks for the review, I updated all the codes with your comments.

ronaldoussoren

I have a minor textual nit with the documentation, otherwise the PR looks good to me.

Doc/library/plistlib.rst

Lib/test/test_plistlib.py

bedevere-app · 2023-12-27T10:34:43Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Co-authored-by: Ronald Oussoren <[email protected]>

Lib/test/test_plistlib.py

serhiy-storchaka · 2023-12-27T14:13:39Z

Lib/test/test_plistlib.py

+    def test_dump_aware_datetime(self):
+        dt = datetime.datetime(2345, 6, 7, 8, 9, 10,
+                               tzinfo=zoneinfo.ZoneInfo("America/Los_Angeles"))
+        for fmt in ALL_FORMATS:
+            s = plistlib.dumps(dt, fmt=fmt, aware_datetime=True)
+            self.assertEqual(plistlib.loads(s, fmt=fmt),
+                             datetime.datetime(2345, 6, 7, 15, 9, 10))


I do not like the results of this test. It silently ignores the time zone and returns a datetime not equal to the origin. It was expected that aware_datetime=True is a stricter option.

Updated to loads the result with aware_datetime=True, and compared the loads result with the original dt.

test_dump_utc_aware_datetime have the same issue and updated too.

Great. Now I see that it works as expected.

But it would be nice to test also that tzinfo is UTC in the loaded datetime.

Thanks for the review, updated.

serhiy-storchaka

LGTM.

1. Use 32-bit compatible date in test_dump_naive_datetime_with_aware_datetime_option 2. test_dump_naive_datetime_with_aware_datetime_option: Writing non-aware datetimes with aware_datetime==True leads to the time being off by twice the timezone offset from UTC

ronaldoussoren · 2024-01-01T19:38:05Z

#113627 fixes the issue with test_dump_naive_datetime_with_aware_datetime_option for me, test_dump_naive_datetime_with_aware_datetime_option on AIX should be fixed by using a date that's in range for 32-bit time_t.

bedevere-bot · 2024-01-01T19:42:18Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Clang Installed 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/531/builds/5068) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/531/builds/5068

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.13/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:42:57Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/469/builds/6909) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/469/builds/6909

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:43:18Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 LTO + PGO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/568/builds/5582) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/568/builds/5582

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:45:13Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 LTO + PGO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/442/builds/5621) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/442/builds/5621

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto-pgo/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:47:44Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 macOS 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/366/builds/6243) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/366/builds/6243

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 15, 0) != datetime.datetime(2345, 6, 7, 16, 0)

bedevere-bot · 2024-01-01T19:49:45Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/125/builds/5031) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/125/builds/5031

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 6, 0) != datetime.datetime(2345, 6, 7, 7, 0)

bedevere-bot · 2024-01-01T19:50:54Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/506/builds/6735) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/506/builds/6735

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:52:14Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 LTO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/64/builds/5832) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/64/builds/5832

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:52:54Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable Clang 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/234/builds/5053) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/234/builds/5053

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 6, 0) != datetime.datetime(2345, 6, 7, 7, 0)

bedevere-bot · 2024-01-01T19:55:45Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable Clang Installed 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/14/builds/5155) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/14/builds/5155

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 6, 0) != datetime.datetime(2345, 6, 7, 7, 0)

bedevere-bot · 2024-01-01T19:57:19Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora LTO + PGO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/545/builds/5038) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/545/builds/5038

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T19:57:55Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 LTO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/567/builds/5548) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/567/builds/5548

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T20:10:29Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora LTO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/55/builds/4292) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/55/builds/4292

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 12, 0) != datetime.datetime(2345, 6, 7, 13, 0)

bedevere-bot · 2024-01-01T20:27:12Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable LTO + PGO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/524/builds/5045) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/524/builds/5045

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto-pgo/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 6, 0) != datetime.datetime(2345, 6, 7, 7, 0)

* gh-53502: Fixes for tests in gh-113363 * Use 32-bit compatible date in test_dump_naive_datetime_with_aware_datetime_option * Saving non-aware datetimes will use the old behaviour regardless of the aware_datimetime setting

ronaldoussoren · 2024-01-01T20:42:02Z

The buildbot failures seem to be related to 32-bit time_t, in particular the test with a date in year 2345:

ERROR: test_dump_naive_datetime_with_aware_datetime_option (test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_plistlib.py", line 886, in test_dump_naive_datetime_with_aware_datetime_option
    s = plistlib.dumps(dt, fmt=fmt, aware_datetime=True)
        ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/plistlib.py", line 931, in dumps
    dump(value, fp, fmt=fmt, skipkeys=skipkeys, sort_keys=sort_keys,
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         aware_datetime=aware_datetime)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/plistlib.py", line 923, in dump
    writer.write(value)
    ~~~~~~~~~~~~^^^^^^^
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/plistlib.py", line 341, in write
    self.write_value(value)
    ~~~~~~~~~~~~~~~~^^^^^^^
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/plistlib.py", line 371, in write_value
    _date_to_string(value, self._aware_datetime))
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/plistlib.py", line 159, in _date_to_string
    d = d.astimezone(datetime.UTC)
        ~~~~~~~~~~~~^^^^^^^^^^^^^^
OverflowError: timestamp out of range for platform time_t

I'm working on a fix.

There are two problems:

test_dump_naive_datetime_with_aware_datetime_option as mentioned earlier this appears to be caused by using a year that's not compatible with 32-bit time_t in the test, fixed by changing the year in that test
test_dump_naive_datetime_with_aware_datetime_option: This is more annoying and is caused by the offset between expected and actual being different than expected (this affects most of the listed buildbots). I've for now committed a change that stores non-aware datetimes as-is in the plist file and that should fix the broken buildbots.

I'm not entirely happy about the second change, but don't quite understand what's causing the problem there. I'm leaving the issue open to further investigate what's going on here. At first I thought that the offset was double the expected offset, but's that not correct either.

bedevere-bot · 2024-01-01T20:42:35Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable LTO 3.x has failed when building commit b4b2cc1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/336/builds/4479) and take a look at the build logs.
Check if the failure is related to this commit (b4b2cc1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/336/builds/4479

Failed tests:

test_plistlib

Failed subtests:

test_dump_naive_datetime_with_aware_datetime_option - test.test_plistlib.TestPlistlib.test_dump_naive_datetime_with_aware_datetime_option

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.lto/build/Lib/test/test_plistlib.py", line 889, in test_dump_naive_datetime_with_aware_datetime_option
    self.assertEqual(parsed, expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: datetime.datetime(2345, 6, 7, 6, 0) != datetime.datetime(2345, 6, 7, 7, 0)

serhiy-storchaka · 2024-01-01T23:29:53Z

I'm not entirely happy about the second change, but don't quite understand what's causing the problem there.

DST. datetime(2003, 6, 7, 8) is in summer. We are very lucky that caught this in tests.

serhiy-storchaka · 2024-01-01T23:34:43Z

expected = dt.astimezone(datetime.UTC).replace(tzinfo=None)

serhiy-storchaka · 2024-01-02T09:54:48Z

See #113645.

…or dumps aware datetime. (python#113363) * add options to loads and dumps aware datetime in plistlib

* pythongh-53502: Fixes for tests in pythongh-113363 * Use 32-bit compatible date in test_dump_naive_datetime_with_aware_datetime_option * Saving non-aware datetimes will use the old behaviour regardless of the aware_datimetime setting

…or dumps aware datetime. (python#113363) * add options to loads and dumps aware datetime in plistlib

* pythongh-53502: Fixes for tests in pythongh-113363 * Use 32-bit compatible date in test_dump_naive_datetime_with_aware_datetime_option * Saving non-aware datetimes will use the old behaviour regardless of the aware_datimetime setting

…or dumps aware datetime. (python#113363) * add options to loads and dumps aware datetime in plistlib

* pythongh-53502: Fixes for tests in pythongh-113363 * Use 32-bit compatible date in test_dump_naive_datetime_with_aware_datetime_option * Saving non-aware datetimes will use the old behaviour regardless of the aware_datimetime setting

aisk added 2 commits December 21, 2023 23:46

add options to loads and dumps aware datetime in plistlib

ae160b4

add news entry

94631ec

bedevere-app bot added the awaiting review label Dec 21, 2023

bedevere-app bot mentioned this pull request Dec 21, 2023

plistlib should create non-naïve datetime objects #53502

Closed

aisk added 2 commits December 22, 2023 00:13

skip tests when zoneinfo not avialble

484e5af

fix test

a204de8

aisk changed the title ~~gh-53502: add a new option aware_datetime in to loads or dumps aware datetime.~~ gh-53502: add a new option aware_datetime in plistlib to loads or dumps aware datetime. Dec 21, 2023

serhiy-storchaka requested review from ronaldoussoren and serhiy-storchaka December 22, 2023 10:52

serhiy-storchaka reviewed Dec 22, 2023

View reviewed changes

update docs

9dd6d7a

add more tests

9cd2670

serhiy-storchaka reviewed Dec 24, 2023

View reviewed changes

Doc/library/plistlib.rst Outdated Show resolved Hide resolved

Lib/test/test_plistlib.py Show resolved Hide resolved

aisk added 2 commits December 26, 2023 16:08

update tests

e9a0e99

update document

12cedcd

using keyword-only instead of keyword in document

195234e

ronaldoussoren requested changes Dec 27, 2023

View reviewed changes

Doc/library/plistlib.rst Outdated Show resolved Hide resolved

Lib/test/test_plistlib.py Show resolved Hide resolved

bedevere-app bot removed the awaiting review label Dec 27, 2023

bedevere-app bot added the awaiting changes label Dec 27, 2023

Update Doc/library/plistlib.rst

7cf6c1f

Co-authored-by: Ronald Oussoren <[email protected]>

serhiy-storchaka reviewed Dec 27, 2023

View reviewed changes

aisk added 2 commits December 27, 2023 22:55

update tests

9a987c0

update tests to test loaded dt's tzinfo is UTC

a40cf85

serhiy-storchaka approved these changes Dec 28, 2023

View reviewed changes

aisk deleted the plist-aware-datetime branch January 15, 2024 11:18

aisk mentioned this pull request Jan 15, 2024

Add aware_datetime option to plistlib functions python/typeshed#11276

Merged

kulikjak pushed a commit to kulikjak/cpython that referenced this pull request Jan 22, 2024

pythongh-53502: add a new option aware_datetime in plistlib to loads …

1682251

…or dumps aware datetime. (python#113363) * add options to loads and dumps aware datetime in plistlib

aisk added a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-53502: add a new option aware_datetime in plistlib to loads …

9cf768c

…or dumps aware datetime. (python#113363) * add options to loads and dumps aware datetime in plistlib

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-53502: add a new option aware_datetime in plistlib to loads …

783f83c

…or dumps aware datetime. (python#113363) * add options to loads and dumps aware datetime in plistlib

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-53502: add a new option aware_datetime in plistlib to loads or dumps aware datetime. #113363

gh-53502: add a new option aware_datetime in plistlib to loads or dumps aware datetime. #113363

aisk commented Dec 21, 2023 •

edited

Loading

serhiy-storchaka left a comment

ronaldoussoren commented Dec 23, 2023

aisk commented Dec 24, 2023

aisk commented Dec 24, 2023

serhiy-storchaka left a comment

serhiy-storchaka commented Dec 24, 2023

aisk commented Dec 26, 2023

ronaldoussoren left a comment

bedevere-app bot commented Dec 27, 2023

serhiy-storchaka Dec 27, 2023

aisk Dec 27, 2023

serhiy-storchaka Dec 27, 2023

aisk Dec 28, 2023

serhiy-storchaka left a comment

ronaldoussoren commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

ronaldoussoren commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

serhiy-storchaka commented Jan 1, 2024

serhiy-storchaka commented Jan 1, 2024

serhiy-storchaka commented Jan 2, 2024

gh-53502: add a new option aware_datetime in plistlib to loads or dumps aware datetime. #113363

gh-53502: add a new option aware_datetime in plistlib to loads or dumps aware datetime. #113363

Conversation

aisk commented Dec 21, 2023 • edited Loading

serhiy-storchaka left a comment

Choose a reason for hiding this comment

ronaldoussoren commented Dec 23, 2023

aisk commented Dec 24, 2023

aisk commented Dec 24, 2023

serhiy-storchaka left a comment

Choose a reason for hiding this comment

serhiy-storchaka commented Dec 24, 2023

aisk commented Dec 26, 2023

ronaldoussoren left a comment

Choose a reason for hiding this comment

bedevere-app bot commented Dec 27, 2023

serhiy-storchaka Dec 27, 2023

Choose a reason for hiding this comment

aisk Dec 27, 2023

Choose a reason for hiding this comment

serhiy-storchaka Dec 27, 2023

Choose a reason for hiding this comment

aisk Dec 28, 2023

Choose a reason for hiding this comment

serhiy-storchaka left a comment

Choose a reason for hiding this comment

ronaldoussoren commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

ronaldoussoren commented Jan 1, 2024

bedevere-bot commented Jan 1, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

serhiy-storchaka commented Jan 1, 2024

serhiy-storchaka commented Jan 1, 2024

serhiy-storchaka commented Jan 2, 2024

aisk commented Dec 21, 2023 •

edited

Loading