GH-107803: double linked list implementation for asyncio tasks #107804
Conversation
kumaraditya303
commented
Aug 9, 2023
•
kumaraditya303
commented
- Issue: Double linked list implementation for asyncio tasks #107803
kumaraditya303
force-pushed
the
linked-list
branch
2 times, most recently
from
8d57b5a
to
af0280a
Compare
kumaraditya303
added
performance
|
🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit af0280a 🤖 If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
bedevere-bot
removed
the
🔨 test-with-buildbots
kumaraditya303
requested review from
1st1,
asvetlov,
gvanrossum and
willingc
as code owners
|
I'll add Whats's New entry for this later together with other performance improvements so skipping news for now. |
|
Is there someone else who could review this? I am currently scrambling. |
| TaskObj *tail = &state->asyncio_tasks.tail; | ||
| while (head != tail) | ||
| { | ||
| if (add_one_task(state, tasks, (PyObject *)head, loop) < 0) { |
There was a problem hiding this comment.
if I understand the refcounting semantics correctly, the linked list contains borrowed references to tasks that may be already freed. (probably shouldn't happen if unregister_task is called properly, but who knows)
if that's the case, this can fail when trying to call done() in add_one_task, right?
should we add some logic here to remove freed pointers? or, if this should never happen - add an assert to check for it?
There was a problem hiding this comment.
That should never happen
There was a problem hiding this comment.
is there an existing assert (or a new one we can add) that would catch such a scenario?
improved linked-list logic and more comments
| TaskObj *tail = &state->asyncio_tasks.tail; | ||
| while (head != tail) | ||
| { | ||
| if (add_one_task(state, tasks, (PyObject *)head, loop) < 0) { |
There was a problem hiding this comment.
is there an existing assert (or a new one we can add) that would catch such a scenario?
|
@kumaraditya303 @itamaro I'm going to mark this "Awaiting changes" due to the merge conflict and the amount of time that has passed since it was opened. If after fixing the changes, you would like to have this reviewed again, please ping. Thanks! |
kumaraditya303
added
the
🔨 test-with-buildbots
|
🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit c7b604c 🤖 If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
bedevere-bot
removed
the
🔨 test-with-buildbots
There was a problem hiding this comment.
Changes look good to me @kumaraditya303. Let's merge this and thanks for rebooting the PR.
|
@kumaraditya303 I wasn't sure if we wanted to backport. Please do so if it makes sense. Thanks! |
…ythonGH-107804) * linked list * add tail optmiization to linked list * wip * wip * wip * more fixes * finally it works * add tests * remove weakreflist * add some comments * reduce code duplication in _asynciomodule.c * address some review comments * add invariants about the state of the linked list * add better explanation * clinic regen * reorder branches for better branch prediction * Update Modules/_asynciomodule.c * Apply suggestions from code review Co-authored-by: Itamar Oren <[email protected]> * fix capturing of eager tasks * add comment to task finalization * fix tests and couple c implmentation to c task improved linked-list logic and more comments * fix test --------- Co-authored-by: Itamar Oren <[email protected]>
…ythonGH-107804) * linked list * add tail optmiization to linked list * wip * wip * wip * more fixes * finally it works * add tests * remove weakreflist * add some comments * reduce code duplication in _asynciomodule.c * address some review comments * add invariants about the state of the linked list * add better explanation * clinic regen * reorder branches for better branch prediction * Update Modules/_asynciomodule.c * Apply suggestions from code review Co-authored-by: Itamar Oren <[email protected]> * fix capturing of eager tasks * add comment to task finalization * fix tests and couple c implmentation to c task improved linked-list logic and more comments * fix test --------- Co-authored-by: Itamar Oren <[email protected]>
…ythonGH-107804) * linked list * add tail optmiization to linked list * wip * wip * wip * more fixes * finally it works * add tests * remove weakreflist * add some comments * reduce code duplication in _asynciomodule.c * address some review comments * add invariants about the state of the linked list * add better explanation * clinic regen * reorder branches for better branch prediction * Update Modules/_asynciomodule.c * Apply suggestions from code review Co-authored-by: Itamar Oren <[email protected]> * fix capturing of eager tasks * add comment to task finalization * fix tests and couple c implmentation to c task improved linked-list logic and more comments * fix test --------- Co-authored-by: Itamar Oren <[email protected]>
