linux.git/mm/percpu.c, branch v2.6.35-rc4

percpu: fix first chunk match in per_cpu_ptr_to_phys()

2010-06-18T13:07:23+00:00

per_cpu_ptr_to_phys() determines whether the passed in @addr belongs
to the first_chunk or not by just matching the address against the
address range of the base unit (unit0, used by cpu0).  When an adress
from another cpu was passed in, it will always determine that the
address doesn't belong to the first chunk even when it does.  This
makes the function return a bogus physical address which may lead to
crash.

This problem was discovered by Cliff Wickman while investigating a
crash during kdump on a SGI UV system.

Signed-off-by: Tejun Heo 
Reported-by: Cliff Wickman 
Tested-by: Cliff Wickman 
Cc: stable@kernel.org

percpu: fix trivial bugs in pcpu_build_alloc_info()

2010-06-17T08:07:25+00:00

Fix the following two trivial bugs in pcpu_build_alloc_info()

* we should memset group_cnt to 0 by size of group_cnt, not size of
  group_map (both are of the same size, so the bug isn't dangerous)

* we can delete useless variable group_cnt_max.

Signed-off-by: Pavel V. Panteleev 
Signed-off-by: Tejun Heo

percpu: implement kernel memory based chunk allocation

2010-05-01T06:30:50+00:00

Implement an alternate percpu chunk management based on kernel memeory
for nommu SMP architectures.  Instead of mapping into vmalloc area,
chunks are allocated as a contiguous kernel memory using
alloc_pages().  As such, percpu allocator on nommu will have the
following restrictions.

* It can't fill chunks on-demand page-by-page.  It has to allocate
  each chunk fully upfront.

* It can't support sparse chunk for NUMA configurations.  SMP w/o mmu
  is crazy enough.  Let's hope no one does NUMA w/o mmu.  :-P

* If chunk size isn't power-of-two multiple of PAGE_SIZE, the
  unaligned amount will be wasted on each chunk.  So, archs which use
  this better align chunk size.

For instructions on how to use this, read the comment on top of
mm/percpu-km.c.

Signed-off-by: Tejun Heo 
Reviewed-by: David Howells 
Cc: Graff Yang 
Cc: Sonic Zhang

percpu: move vmalloc based chunk management into percpu-vm.c

2010-05-01T06:30:50+00:00

Separate out and move chunk management (creation/desctruction and
[de]population) code into percpu-vm.c which is included by percpu.c
and compiled together.  The interface for chunk management is defined
as follows.

 * pcpu_populate_chunk		- populate the specified range of a chunk
 * pcpu_depopulate_chunk	- depopulate the specified range of a chunk
 * pcpu_create_chunk		- create a new chunk
 * pcpu_destroy_chunk		- destroy a chunk, always preceded by full depop
 * pcpu_addr_to_page		- translate address to physical address
 * pcpu_verify_alloc_info	- check alloc_info is acceptable during init

Other than wrapping vmalloc_to_page() inside pcpu_addr_to_page() and
dummy pcpu_verify_alloc_info() implementation, this patch only moves
code around.  This separation is to allow alternate chunk management
implementation.

Signed-off-by: Tejun Heo 
Reviewed-by: David Howells 
Cc: Graff Yang 
Cc: Sonic Zhang

percpu: misc preparations for nommu support

2010-05-01T06:30:50+00:00

Make the following misc preparations for percpu nommu support.

* Remove refernces to vmalloc in common comments as nommu percpu won't
  use it.

* Rename chunk->vms to chunk->data and make it void *.  Its use is
  determined by chunk management implementation.

* Relocate utility functions and add __maybe_unused to functions which
  might not be used by different chunk management implementations.

This patch doesn't cause any functional change.  This is to allow
alternate chunk management implementation for percpu nommu support.

Signed-off-by: Tejun Heo 
Reviewed-by: David Howells 
Cc: Graff Yang 
Cc: Sonic Zhang

percpu: reorganize chunk creation and destruction

2010-05-01T06:30:50+00:00

Reorganize alloc/free_pcpu_chunk() such that chunk struct alloc/free
live in pcpu_alloc/free_chunk() and the rest in
pcpu_create/destroy_chunk().  While at it, add missing error handling
for chunk->map allocation failure.

This is to allow alternate chunk management implementation for percpu
nommu support.

Signed-off-by: Tejun Heo 
Reviewed-by: David Howells 
Cc: Graff Yang 
Cc: Sonic Zhang

percpu: factor out pcpu_addr_in_first/reserved_chunk() and update per_cpu_ptr_to_phys()

2010-05-01T06:30:49+00:00

Factor out pcpu_addr_in_first/reserved_chunk() from
pcpu_chunk_addr_search() and use it to update per_cpu_ptr_to_phys()
such that it handles first chunk differently from the rest.

This patch doesn't cause any functional change and is to prepare for
percpu nommu support.

Signed-off-by: Tejun Heo 
Reviewed-by: David Howells 
Cc: Graff Yang 
Cc: Sonic Zhang

percpu, module: implement and use is_kernel/module_percpu_address()

2010-03-29T14:07:12+00:00

lockdep has custom code to check whether a pointer belongs to static
percpu area which is somewhat broken.  Implement proper
is_kernel/module_percpu_address() and replace the custom code.

On UP, percpu variables are regular static variables and can't be
distinguished from them.  Always return %false on UP.

Signed-off-by: Tejun Heo 
Acked-by: Peter Zijlstra 
Cc: Rusty Russell 
Cc: Ingo Molnar

percpu: add __percpu sparse annotations to core kernel subsystems

2010-02-17T02:17:38+00:00

Add __percpu sparse annotations to core subsystems.

These annotations are to make sparse consider percpu variables to be
in a different address space and warn if accessed without going
through percpu accessors.  This patch doesn't affect normal builds.

Signed-off-by: Tejun Heo 
Reviewed-by: Christoph Lameter 
Acked-by: Paul E. McKenney 
Cc: Jens Axboe 
Cc: linux-mm@kvack.org
Cc: Rusty Russell 
Cc: Dipankar Sarma 
Cc: Peter Zijlstra 
Cc: Andrew Morton 
Cc: Eric Biederman

Merge branch 'master' into percpu

2010-02-02T05:38:15+00:00