Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unbounded cache #1651

Merged
merged 1 commit into from
Oct 11, 2022
Merged

Conversation

jonnystoten
Copy link
Contributor

TUFMetaStorage uses an in-memory cache to store all retrieved metadata files. This cache has a few problems:

  1. It doesn't have any sort of limit on its size so it will grow indefinitely as more content is retrieved
  2. Entries in the cache are not deleted when Delete is calls on a TUFMetaStorage instance. This means the cache can hold entries that have been deleted from the underlying storage (there seem to be tests for this behavior so could this be for a reason?)
  3. It isn't safe for concurrent use, see Fix a concurrency bug in TUFMetaStorage by using sync.Map #1613.

I've taken the scorched-earth approach of simply removing this cache. After #1639 it should be fast to get these files from the database each time. I'll test this on Docker's production instance before merging it to check the performance is still OK.

@jonnystoten
Copy link
Contributor Author

Looks good on Docker's notary server after a few days. The performance seems fine and more consistent - the average is a little bit slower (~3ms -> ~5ms) but the worst case is much better (>100ms -> ~8ms).

I'm assuming the tests that were testing that deleting from the TUF store doesn't delete from the cache were just wrong - anything relying on this will have other issues because of course if the server is restarted the cache will be cleared and if more than one server is running each one will have different cache contents etc.

This can also close #1613.

@jonnystoten jonnystoten marked this pull request as ready for review September 16, 2022 14:24
Signed-off-by: Jonny Stoten <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants