Skip to content
This repository has been archived by the owner on Aug 4, 2024. It is now read-only.

some question about kernel.io.directio #46

Open
mapleFU opened this issue Oct 3, 2023 · 3 comments
Open

some question about kernel.io.directio #46

mapleFU opened this issue Oct 3, 2023 · 3 comments
Labels
invalid This doesn't seem right

Comments

@mapleFU
Copy link

mapleFU commented Oct 3, 2023

Question

Direct IO, usally means O_DIRECT [1] and related IO, which could bypass some IO stack [2]. Usally it needs to align the size of io, and can bypass some page buffer/cache

Should we rename this to Local IO or other?

Besides, read using current reader is limited to single thread, since read is only support single thread. should we support a readAt (pread [3]) for possible multi-thread read?

impl Read for DirectIoReader {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        self.fs.read(buf)
    }
}

[1] https://stackoverflow.com/questions/41257656/what-does-o-direct-really-mean
[2] https://www.thomas-krenn.com/en/wiki/Linux_Storage_Stack_Diagram
[3] https://man7.org/linux/man-pages/man2/pwrite.2.html

@KKould
Copy link
Member

KKould commented Oct 3, 2023

Thank you very much for your suggestion.
In fact, according to the current code implementation, Local IO may be more suitable. In fact, this reader was originally scheduled to use direct io to implement reading, but since the implementation of direct io is different on various systems, it is temporarily The implementation is simplified and represented by File. I think it might be better if I comment on it.
I don't know much about readAt. I'll do some research.

@KKould KKould added the invalid This doesn't seem right label Oct 3, 2023
@mapleFU
Copy link
Author

mapleFU commented Oct 3, 2023

To be short, a file may has an offset, and read might change that. So if there are multiple readers read on same fd, it might cause some concurrency problem.

pread means a positional-read, which is "thread safe".

@lewiszlw
Copy link
Member

lewiszlw commented Oct 9, 2023

https://github.com/vasi/positioned-io this may helpful.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
invalid This doesn't seem right
Projects
None yet
Development

No branches or pull requests

3 participants