[utils] Backport traverse_obj (etc) from yt-dlp

[lebdron]

Please follow the guide below

• You will be asked some questions, please read them carefully and answer honestly
• Put an x into all the boxes [ ] relevant to your pull request (like that [x])
• Use Preview tab to see how your pull request will actually look like

---

Before submitting a pull request make sure you have:

• Searched the bugtracker for similar pull requests
• Read adding new extractor tutorial
• Read youtube-dl coding conventions and adjusted the code to meet them
• Covered the code with tests (note that PRs without tests will be REJECTED)
• Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

• I am the original author of this code and I am willing to release it under Unlicense, except for code from yt-dlp for which this or the below has been asserted
• I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

• Bug fix
• Improvement
• New extractor
• New feature

---

Description of your pull request and other information

Backport traverse_obj from yt-dlp required for #31097.

Thanks @pukkandan (orignal code), @Grub4K (rewrite & tests).

[dirkf]

Please use my full port in PR #30713 as a basis. Feel free to define variadic() in this PR too, instead of locally within traverse_obj(), and to define IDENTITY(), get_first() and join_nonempty() as well.

The from_iterable() function in my code is needed because the library function used in yt-dlp isn't always available.

The Nonlocal thing is to simulate the nonlocal syntax for older Pythons. As you know, assigning to a previously undefined variable creates a variable binding in the current scope, unless the variable has been defined global. nonlocal tells Python to look for the variable on the call stack too, so that a new binding is not created when assigning to the variable. By making the variable that is to be nonlocal an instance value of an object, an assignment to the variable doesn't create a new binding, but updates the existing value.

listish() defines an expression used in several places so that the list of listish things can be updated with one change.

As you've done, ... has to be replaced with Ellipsis as the ... syntax is only in recent Pythons.

Finally, any str should be compat_str.

Even more finally, should we ask @pukkandan to write tests for it?

The code in PR #30713 was correct enough for that extractor, but as was commented extractors only use a subset of traverse_obj() functionality.

[lebdron]

@dirkf Thank you, done. I looked at your commit, ported some features which improve readability, like Nonlocal. I also used the new code from yt-dlp/yt-dlp@1797b07 and yt-dlp/yt-dlp@e6f868a .

I would suggest to merge this PR as is, without extensive test coverage, as it will be useful to port Panopto sooner, and it does not break any existing code. Tests can be added later in a separate pull request.

[pukkandan]

Even more finally, should we ask @pukkandan to write tests for it?

I honestly don't have too much incentive to write tests for it. Because of yt-dlp's extensive use of traverse_obj in "output template", test_prepare_outtmpl_and_filename kinda acts like a decent test suite for traverse_obj. But if anyone wants to contribute tests, it is welcome.

[pukkandan]

Complete rewrite and tests: yt-dlp/yt-dlp#5024

[dirkf]

Good news: there are tests
Bad news: the back-ported code should match the new code

[dirkf]

Recast with yt-dlp's new traverse_obj() implementation and tests.