Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not acquire lock for file.Sync() fsync call #67

Closed
wants to merge 1 commit into from

Conversation

1978629634
Copy link

In the Flush() function, a lock is acquired before calling file.flush and file.sync. file.flush writes the glog buffer into the kernel buffer, while file.sync writes the kernel buffer to disk.

We encountered a problem where when disk I/O is heavy(the cloud service provider offers poor cloud disk performance), all glog logs would be blocked due to waiting for file.sync.

file.sync ensures that logs are flushed to disk, but I believe that acquiring a lock before calling file.sync is unnecessary. The kernel has an efficient buffered I/O mechanism, and file.sync does not block kernel buffered I/O. We only need to request that logs are flushed to disk and do not need to acquire a lock and wait for the kernel to complete the flush. Acquiring a lock and waiting for logs to be flushed to disk cannot make good use of the kernel's buffered I/O, and it can lead to performance issues.

Copy link

google-cla bot commented Apr 3, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@1978629634
Copy link
Author

Hi @chressie @stapelberg , Could you please take a look at this pull request when you have a chance? I'd appreciate your review and feedback.

@chressie
Copy link
Contributor

chressie commented Apr 4, 2024

Hi @1978629634.

Thanks for your PR and for taking the time to explain and draft a fix. While the change you're requesting is reasonable, we cannot merge it directly into this repository. We have to make an internal change first, that we'll then be able to push upstream.

Quote from the README:

The repository contains an open source version of the log package used inside Google. The master copy of the source lives inside Google, not here. The code in this repo is for export only and is not itself under development. Feature requests will be ignored.

Send bug reports to [email protected].

I'm closing this PR and will soon merge a new version with a fix. Thanks for your understanding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants