linux.git/mm/memory_hotplug.c, branch v2.6.35-rc4

mem-hotplug: fix potential race while building zonelist for new populated zone

2010-05-25T15:07:02+00:00

Add global mutex zonelists_mutex to fix the possible race:

     CPU0                                  CPU1                    CPU2
(1) zone->present_pages += online_pages;
(2)                                       build_all_zonelists();
(3)                                                               alloc_page();
(4)                                                               free_page();
(5) build_all_zonelists();
(6)   __build_all_zonelists();
(7)     zone->pageset = alloc_percpu();

In step (3,4), zone->pageset still points to boot_pageset, so bad
things may happen if 2+ nodes are in this state. Even if only 1 node
is accessing the boot_pageset, (3) may still consume too much memory
to fail the memory allocations in step (7).

Besides, atomic operation ensures alloc_percpu() in step (7) will never fail
since there is a new fresh memory block added in step(6).

[haicheng.li@linux.intel.com: hold zonelists_mutex when build_all_zonelists]
Signed-off-by: Haicheng Li 
Signed-off-by: Wu Fengguang 
Reviewed-by: Andi Kleen 
Cc: Christoph Lameter 
Cc: Mel Gorman 
Cc: Tejun Heo 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mem-hotplug: avoid multiple zones sharing same boot strapping boot_pageset

2010-05-25T15:07:01+00:00

For each new populated zone of hotadded node, need to update its pagesets
with dynamically allocated per_cpu_pageset struct for all possible CPUs:

    1) Detach zone->pageset from the shared boot_pageset
       at end of __build_all_zonelists().

    2) Use mutex to protect zone->pageset when it's still
       shared in onlined_pages()

Otherwises, multiple zones of different nodes would share same boot strapping
boot_pageset for same CPU, which will finally cause below kernel panic:

  ------------[ cut here ]------------
  kernel BUG at mm/page_alloc.c:1239!
  invalid opcode: 0000 [#1] SMP
  ...
  Call Trace:
   [] __alloc_pages_nodemask+0x131/0x7b0
   [] alloc_pages_current+0x87/0xd0
   [] __page_cache_alloc+0x67/0x70
   [] __do_page_cache_readahead+0x120/0x260
   [] ra_submit+0x21/0x30
   [] ondemand_readahead+0x166/0x2c0
   [] page_cache_async_readahead+0x80/0xa0
   [] generic_file_aio_read+0x364/0x670
   [] nfs_file_read+0xca/0x130
   [] do_sync_read+0xfa/0x140
   [] vfs_read+0xb5/0x1a0
   [] sys_read+0x51/0x80
   [] system_call_fastpath+0x16/0x1b
  RIP  [] get_page_from_freelist+0x883/0x900
   RSP 
  ---[ end trace 4bda28328b9990db ]

[akpm@linux-foundation.org: merge fix]
Signed-off-by: Haicheng Li 
Signed-off-by: Wu Fengguang 
Reviewed-by: Andi Kleen 
Reviewed-by: Christoph Lameter 
Cc: Mel Gorman 
Cc: Tejun Heo 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

cpu/mem hotplug: enable CPUs online before local memory online

2010-05-25T15:07:00+00:00

Enable users to online CPUs even if the CPUs belongs to a numa node which
doesn't have onlined local memory.

The zonlists(pg_data_t.node_zonelists[]) of a numa node are created either
in system boot/init period, or at the time of local memory online.  For a
numa node without onlined local memory, its zonelists are not initialized
at present.  As a result, any memory allocation operations executed by
CPUs within this node will fail.  In fact, an out-of-memory error is
triggered when attempt to online CPUs before memory comes to online.

This patch tries to create zonelists for such numa nodes, so that the
memory allocation for this node can be fallback'ed to other nodes.

[akpm@linux-foundation.org: remove unneeded export]
[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: minskey guo
Cc: Minchan Kim 
Cc: Yasunori Goto 
Cc: Andi Kleen 
Cc: Christoph Lameter 
Cc: Tejun Heo 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: introduce dump_page() and print symbolic flag names

2010-03-12T23:52:28+00:00

- introduce dump_page() to print the page info for debugging some error
  condition.

- convert three mm users: bad_page(), print_bad_pte() and memory offline
  failure.

- print an extra field: the symbolic names of page->flags

Example dump_page() output:

[  157.521694] page:ffffea0000a7cba8 count:2 mapcount:1 mapping:ffff88001c901791 index:0x147
[  157.525570] page flags: 0x100000000100068(uptodate|lru|active|swapbacked)

Signed-off-by: Wu Fengguang 
Cc: Ingo Molnar 
Cc: Alex Chiang 
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Mel Gorman 
Cc: Christoph Lameter 
Cc: KOSAKI Motohiro 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: create /sys/firmware/memmap entry for new memory

2010-03-06T19:26:25+00:00

A memmap is a directory in sysfs which includes 3 text files: start, end
and type.  For example:

start: 	0x100000
end:	0x7e7b1cff
type:	System RAM

Interface firmware_map_add was not called explicitly.  Remove it and add
function firmware_map_add_hotplug as hotplug interface of memmap.

Each memory entry has a memmap in sysfs, When we hot-add new memory, sysfs
does not export memmap entry for it.  We add a call in function add_memory
to function firmware_map_add_hotplug.

Add a new function add_sysfs_fw_map_entry() to create memmap entry, it
will be called when initialize memmap and hot-add memory.

[akpm@linux-foundation.org: un-kernedoc a no longer kerneldoc comment]
Signed-off-by: Shaohui Zheng 
Acked-by: Andi Kleen 
Acked-by: Yasunori Goto 
Reviewed-by: Wu Fengguang 
Cc: Dave Hansen 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: fix section mismatch in memory_hotplug.c

2009-12-15T16:53:20+00:00

__free_pages_bootmem() is a __meminit function - which has been called
from put_pages_bootmem thus causes a section mismatch warning.

 We were warned by the following warning:

  LD      mm/built-in.o
WARNING: mm/built-in.o(.text+0x26b22): Section mismatch in reference
from the function put_page_bootmem() to the function
.meminit.text:__free_pages_bootmem()
The function put_page_bootmem() references
the function __meminit __free_pages_bootmem().
This is often because put_page_bootmem lacks a __meminit
annotation or the annotation of __free_pages_bootmem is wrong.

Signed-off-by: Rakib Mullick 
Cc: Yasunori Goto 
Cc: Badari Pulavarty 
Cc: Johannes Weiner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: memory_hotplug: make offline_pages() static

2009-12-15T16:53:20+00:00

It has no references outside memory_hotplug.c.

Cc: "Rafael J. Wysocki" 
Cc: Andi Kleen 
Cc: Gerald Schaefer 
Cc: KOSAKI Motohiro 
Cc: Yasunori Goto 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ksm: memory hotremove migration only

2009-12-15T16:53:20+00:00

The previous patch enables page migration of ksm pages, but that soon gets
into trouble: not surprising, since we're using the ksm page lock to lock
operations on its stable_node, but page migration switches the page whose
lock is to be used for that.  Another layer of locking would fix it, but
do we need that yet?

Do we actually need page migration of ksm pages?  Yes, memory hotremove
needs to offline sections of memory: and since we stopped allocating ksm
pages with GFP_HIGHUSER, they will tend to be GFP_HIGHUSER_MOVABLE
candidates for migration.

But KSM is currently unconscious of NUMA issues, happily merging pages
from different NUMA nodes: at present the rule must be, not to use
MADV_MERGEABLE where you care about NUMA.  So no, NUMA page migration of
ksm pages does not make sense yet.

So, to complete support for ksm swapping we need to make hotremove safe.
ksm_memory_callback() take ksm_thread_mutex when MEM_GOING_OFFLINE and
release it when MEM_OFFLINE or MEM_CANCEL_OFFLINE.  But if mapped pages
are freed before migration reaches them, stable_nodes may be left still
pointing to struct pages which have been removed from the system: the
stable_node needs to identify a page by pfn rather than page pointer, then
it can safely prune them when MEM_OFFLINE.

And make NUMA migration skip PageKsm pages where it skips PageReserved.
But it's only when we reach unmap_and_move() that the page lock is taken
and we can be sure that raised pagecount has prevented a PageAnon from
being upgraded: so add offlining arg to migrate_pages(), to migrate ksm
page when offlining (has sufficient locking) but reject it otherwise.

Signed-off-by: Hugh Dickins 
Cc: Izik Eidus 
Cc: Andrea Arcangeli 
Cc: Chris Wright 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: clear node in N_HIGH_MEMORY and stop kswapd when all memory is offlined

2009-12-15T16:53:13+00:00

When memory is hot-removed, its node must be cleared in N_HIGH_MEMORY if
there are no present pages left.

In such a situation, kswapd must also be stopped since it has nothing left
to do.

Signed-off-by: David Rientjes 
Signed-off-by: Lee Schermerhorn 
Cc: Christoph Lameter 
Cc: Yasunori Goto 
Cc: Mel Gorman 
Cc: Rafael J. Wysocki 
Cc: Rik van Riel 
Cc: KAMEZAWA Hiroyuki 
Cc: Lee Schermerhorn 
Cc: Mel Gorman 
Cc: Randy Dunlap 
Cc: Nishanth Aravamudan 
Cc: Andi Kleen 
Cc: David Rientjes 
Cc: Adam Litke 
Cc: Andy Whitcroft 
Cc: Eric Whitney 
Cc: Christoph Lameter 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: move inc_zone_page_state(NR_ISOLATED) to just isolated place

2009-12-15T16:53:12+00:00

Christoph pointed out inc_zone_page_state(NR_ISOLATED) should be placed
in right after isolate_page().

This patch does it.

Reviewed-by: Christoph Lameter 
Signed-off-by: KOSAKI Motohiro 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds