Skip to content

Commit

Permalink
Revert "kasan: drop skip_kasan_poison variable in free_pages_prepare"
Browse files Browse the repository at this point in the history
This reverts commit 487a32e.

should_skip_kasan_poison() reads the PG_skip_kasan_poison flag from
page->flags.  However, this line of code in free_pages_prepare():

	page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;

clears most of page->flags, including PG_skip_kasan_poison, before calling
should_skip_kasan_poison(), which meant that it would never return true as
a result of the page flag being set.  Therefore, fix the code to call
should_skip_kasan_poison() before clearing the flags, as we were doing
before the reverted patch.

This fixes a measurable performance regression introduced in the reverted
commit, where munmap() takes longer than intended if HW tags KASAN is
supported and enabled at runtime.  Without this patch, we see a
single-digit percentage performance regression in a particular
mmap()-heavy benchmark when enabling HW tags KASAN, and with the patch,
there is no statistically significant performance impact when enabling HW
tags KASAN.

Link: https://lkml.kernel.org/r/[email protected]
Fixes: 487a32e ("kasan: drop skip_kasan_poison variable in free_pages_prepare")
  Link: https://linux-review.googlesource.com/id/Ic4f13affeebd20548758438bb9ed9ca40e312b79
Signed-off-by: Peter Collingbourne <[email protected]>
Reviewed-by: Andrey Konovalov <[email protected]>
Cc: Andrey Ryabinin <[email protected]>
Cc: Catalin Marinas <[email protected]> [arm64]
Cc: Evgenii Stepanov <[email protected]>
Cc: Vincenzo Frascino <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: <[email protected]>	[6.1]
Signed-off-by: Andrew Morton <[email protected]>
  • Loading branch information
pcc authored and akpm00 committed Mar 24, 2023
1 parent cc2a978 commit f446883
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion mm/page_alloc.c
Original file line number Diff line number Diff line change
Expand Up @@ -1398,6 +1398,7 @@ static __always_inline bool free_pages_prepare(struct page *page,
unsigned int order, bool check_free, fpi_t fpi_flags)
{
int bad = 0;
bool skip_kasan_poison = should_skip_kasan_poison(page, fpi_flags);
bool init = want_init_on_free();

VM_BUG_ON_PAGE(PageTail(page), page);
Expand Down Expand Up @@ -1470,7 +1471,7 @@ static __always_inline bool free_pages_prepare(struct page *page,
* With hardware tag-based KASAN, memory tags must be set before the
* page becomes unavailable via debug_pagealloc or arch_free_page.
*/
if (!should_skip_kasan_poison(page, fpi_flags)) {
if (!skip_kasan_poison) {
kasan_poison_pages(page, order, init);

/* Memory is already initialized if KASAN did it internally. */
Expand Down

0 comments on commit f446883

Please sign in to comment.