diff options
| author | Chengming Zhou <zhouchengming@bytedance.com> | 2025-09-07 10:56:27 -0400 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2025-09-09 18:56:34 +0200 |
| commit | ad2df864033e12c699e9f1457030c23f1e1bbc85 (patch) | |
| tree | d6441859d7ab642bd53f094a56224618e59448e1 | |
| parent | c866c40d07c444e2d847519e64b1d96ccc9f38bc (diff) | |
| download | linux-ad2df864033e12c699e9f1457030c23f1e1bbc85.tar.gz linux-ad2df864033e12c699e9f1457030c23f1e1bbc85.tar.bz2 linux-ad2df864033e12c699e9f1457030c23f1e1bbc85.zip | |
slub: Reflow ___slab_alloc()
[ Upstream commit 24c6a097b5a270e05c6e99a99da66b91be81fd7d ]
The get_partial() interface used in ___slab_alloc() may return a single
object in the "kmem_cache_debug(s)" case, in which we will just return
the "freelist" object.
Move this handling up to prepare for later changes.
And the "pfmemalloc_match()" part is not needed for node partial slab,
since we already check this in the get_partial_node().
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Stable-dep-of: 850470a8413a ("mm: slub: avoid wake up kswapd in set_track_prepare")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
| -rw-r--r-- | mm/slub.c | 31 |
1 files changed, 15 insertions, 16 deletions
diff --git a/mm/slub.c b/mm/slub.c index 391f9db1c8f3..b2fb63ebfda1 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3228,8 +3228,21 @@ new_objects: pc.slab = &slab; pc.orig_size = orig_size; freelist = get_partial(s, node, &pc); - if (freelist) - goto check_new_slab; + if (freelist) { + if (kmem_cache_debug(s)) { + /* + * For debug caches here we had to go through + * alloc_single_from_partial() so just store the + * tracking info and return the object. + */ + if (s->flags & SLAB_STORE_USER) + set_track(s, freelist, TRACK_ALLOC, addr); + + return freelist; + } + + goto retry_load_slab; + } slub_put_cpu_ptr(s->cpu_slab); slab = new_slab(s, gfpflags, node); @@ -3265,20 +3278,6 @@ new_objects: inc_slabs_node(s, slab_nid(slab), slab->objects); -check_new_slab: - - if (kmem_cache_debug(s)) { - /* - * For debug caches here we had to go through - * alloc_single_from_partial() so just store the tracking info - * and return the object - */ - if (s->flags & SLAB_STORE_USER) - set_track(s, freelist, TRACK_ALLOC, addr); - - return freelist; - } - if (unlikely(!pfmemalloc_match(slab, gfpflags))) { /* * For !pfmemalloc_match() case we don't load freelist so that |