Skip to content

Commit

Permalink
io_uring: pin context while queueing deferred tw
Browse files Browse the repository at this point in the history
Unlike normal tw, nothing prevents deferred tw to be executed right
after an tw item added to ->work_llist in io_req_local_work_add(). For
instance, the waiting task may get waken up by CQ posting or a normal
tw. Thus we need to pin the ring for the rest of io_req_local_work_add()

Cc: [email protected]
Fixes: c0e0d6b ("io_uring: add IORING_SETUP_DEFER_TASKRUN")
Signed-off-by: Pavel Begunkov <[email protected]>
Link: https://lore.kernel.org/r/1a79362b9c10b8523ef70b061d96523650a23344.1672795998.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <[email protected]>
  • Loading branch information
isilence authored and axboe committed Jan 4, 2023
1 parent af82425 commit 9ffa13f
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion io_uring/io_uring.c
Original file line number Diff line number Diff line change
Expand Up @@ -1236,13 +1236,18 @@ static void io_req_local_work_add(struct io_kiocb *req)
{
struct io_ring_ctx *ctx = req->ctx;

if (!llist_add(&req->io_task_work.node, &ctx->work_llist))
percpu_ref_get(&ctx->refs);

if (!llist_add(&req->io_task_work.node, &ctx->work_llist)) {
percpu_ref_put(&ctx->refs);
return;
}
/* need it for the following io_cqring_wake() */
smp_mb__after_atomic();

if (unlikely(atomic_read(&req->task->io_uring->in_idle))) {
io_move_task_work_from_local(ctx);
percpu_ref_put(&ctx->refs);
return;
}

Expand All @@ -1252,6 +1257,7 @@ static void io_req_local_work_add(struct io_kiocb *req)
if (ctx->has_evfd)
io_eventfd_signal(ctx);
__io_cqring_wake(ctx);
percpu_ref_put(&ctx->refs);
}

void __io_req_task_work_add(struct io_kiocb *req, bool allow_local)
Expand Down

0 comments on commit 9ffa13f

Please sign in to comment.