Skip to content
/ linux Public
forked from torvalds/linux

Commit

Permalink
lib: zstd: Don't inline functions in zstd_opt.c
Browse files Browse the repository at this point in the history
`zstd_opt.c` contains the match finder for the highest compression
levels. These levels are already very slow, and are unlikely to be used
in the kernel. If they are used, they shouldn't be used in latency
sensitive workloads, so slowing them down shouldn't be a big deal.

This saves 188 KB of the 288 KB regression reported by Geert Uytterhoeven [0].
I've also opened an issue upstream [1] so that we can properly tackle
the code size issue in `zstd_opt.c` for all users, and can hopefully
remove this hack in the next zstd version we import.

Bloat-o-meter output on x86-64:

```
> ../scripts/bloat-o-meter vmlinux.old vmlinux
add/remove: 6/5 grow/shrink: 1/9 up/down: 16673/-209939 (-193266)
Function                                     old     new   delta
ZSTD_compressBlock_opt_generic.constprop       -    7559   +7559
ZSTD_insertBtAndGetAllMatches                  -    6304   +6304
ZSTD_insertBt1                                 -    1731   +1731
ZSTD_storeSeq                                  -     693    +693
ZSTD_BtGetAllMatches                           -     255    +255
ZSTD_updateRep                                 -     128    +128
ZSTD_updateTree                               96      99      +3
ZSTD_insertAndFindFirstIndexHash3             81       -     -81
ZSTD_setBasePrices.constprop                  98       -     -98
ZSTD_litLengthPrice.constprop                138       -    -138
ZSTD_count                                   362     181    -181
ZSTD_count_2segments                        1407     938    -469
ZSTD_insertBt1.constprop                    2689       -   -2689
ZSTD_compressBlock_btultra2                19990     423  -19567
ZSTD_compressBlock_btultra                 19633      15  -19618
ZSTD_initStats_ultra                       19825       -  -19825
ZSTD_compressBlock_btopt                   20374      12  -20362
ZSTD_compressBlock_btopt_extDict           29984      12  -29972
ZSTD_compressBlock_btultra_extDict         30718      15  -30703
ZSTD_compressBlock_btopt_dictMatchState    32689      12  -32677
ZSTD_compressBlock_btultra_dictMatchState   33574      15  -33559
Total: Before=6611828, After=6418562, chg -2.92%
```

[0] https://lkml.org/lkml/2021/11/14/189
[1] facebook/zstd#2862

Link: https://lore.kernel.org/r/[email protected]/
Link: https://lore.kernel.org/r/[email protected]/

Reported-by: Geert Uytterhoeven <[email protected]>
Tested-by: Geert Uytterhoeven <[email protected]>
Reviewed-by: Geert Uytterhoeven <[email protected]>
Signed-off-by: Nick Terrell <[email protected]>
  • Loading branch information
terrelln committed Nov 18, 2021
1 parent ae8d67b commit 1974990
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 0 deletions.
7 changes: 7 additions & 0 deletions lib/zstd/common/compiler.h
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
*********************************************************/
/* force inlining */

#if !defined(ZSTD_NO_INLINE)
#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */
# define INLINE_KEYWORD inline
#else
Expand All @@ -24,6 +25,12 @@

#define FORCE_INLINE_ATTR __attribute__((always_inline))

#else

#define INLINE_KEYWORD
#define FORCE_INLINE_ATTR

#endif

/*
On MSVC qsort requires that functions passed into it use the __cdecl calling conversion(CC).
Expand Down
12 changes: 12 additions & 0 deletions lib/zstd/compress/zstd_opt.c
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,18 @@
* You may select, at your option, one of the above-listed licenses.
*/

/*
* Disable inlining for the optimal parser for the kernel build.
* It is unlikely to be used in the kernel, and where it is used
* latency shouldn't matter because it is very slow to begin with.
* We prefer a ~180KB binary size win over faster optimal parsing.
*
* TODO(https://github.com/facebook/zstd/issues/2862):
* Improve the code size of the optimal parser in general, so we
* don't need this hack for the kernel build.
*/
#define ZSTD_NO_INLINE 1

#include "zstd_compress_internal.h"
#include "hist.h"
#include "zstd_opt.h"
Expand Down

0 comments on commit 1974990

Please sign in to comment.