diff options
| author | Jann Horn <jannh@google.com> | 2025-05-27 23:23:54 +0200 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2025-06-27 11:07:38 +0100 |
| commit | b7754d3aa7bf9f62218d096c0c8f6c13698fac8b (patch) | |
| tree | 149872a529d6c055d50afad132ad7bfcb5c3de41 | |
| parent | 02333ac1c35370517a19a4a131332a9690c6a5c7 (diff) | |
| download | linux-b7754d3aa7bf9f62218d096c0c8f6c13698fac8b.tar.gz linux-b7754d3aa7bf9f62218d096c0c8f6c13698fac8b.tar.bz2 linux-b7754d3aa7bf9f62218d096c0c8f6c13698fac8b.zip | |
mm/hugetlb: fix huge_pmd_unshare() vs GUP-fast race
commit 1013af4f585fccc4d3e5c5824d174de2257f7d6d upstream.
huge_pmd_unshare() drops a reference on a page table that may have
previously been shared across processes, potentially turning it into a
normal page table used in another process in which unrelated VMAs can
afterwards be installed.
If this happens in the middle of a concurrent gup_fast(), gup_fast() could
end up walking the page tables of another process. While I don't see any
way in which that immediately leads to kernel memory corruption, it is
really weird and unexpected.
Fix it with an explicit broadcast IPI through tlb_remove_table_sync_one(),
just like we do in khugepaged when removing page tables for a THP
collapse.
Link: https://lkml.kernel.org/r/20250528-hugetlb-fixes-splitrace-v2-2-1329349bad1a@google.com
Link: https://lkml.kernel.org/r/20250527-hugetlb-fixes-splitrace-v1-2-f4136f5ec58a@google.com
Fixes: 39dde65c9940 ("[PATCH] shared page table for hugetlb page")
Signed-off-by: Jann Horn <jannh@google.com>
Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
| -rw-r--r-- | mm/hugetlb.c | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a3907edf2909..2dee7dcb3b18 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -7162,6 +7162,13 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, return 0; pud_clear(pud); + /* + * Once our caller drops the rmap lock, some other process might be + * using this page table as a normal, non-hugetlb page table. + * Wait for pending gup_fast() in other threads to finish before letting + * that happen. + */ + tlb_remove_table_sync_one(); atomic_dec(&virt_to_page(ptep)->pt_share_count); mm_dec_nr_pmds(mm); return 1; |