// SPDX-License-Identifier: GPL-2.0
/* kernel/rwsem.c: R/W semaphores, public implementation
*
* Written by David Howells (dhowells@redhat.com).
* Derived from asm-i386/semaphore.h
*
* Writer lock-stealing by Alex Shi <alex.shi@intel.com>
* and Michel Lespinasse <walken@google.com>
*
* Optimistic spinning by Tim Chen <tim.c.chen@intel.com>
* and Davidlohr Bueso <davidlohr@hp.com>. Based on mutexes.
*
* Rwsem count bit fields re-definition and rwsem rearchitecture by
* Waiman Long <longman@redhat.com> and
* Peter Zijlstra <peterz@infradead.org>.
*/
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/sched/rt.h>
#include <linux/sched/task.h>
#include <linux/sched/debug.h>
#include <linux/sched/wake_q.h>
#include <linux/sched/signal.h>
#include <linux/sched/clock.h>
#include <linux/export.h>
#include <linux/rwsem.h>
#include <linux/atomic.h>
#include "lock_events.h"
/*
* The least significant 3 bits of the owner value has the following
* meanings when set.
* - Bit 0: RWSEM_READER_OWNED - The rwsem is owned by readers
* - Bit 1: RWSEM_RD_NONSPINNABLE - Readers cannot spin on this lock.
* - Bit 2: RWSEM_WR_NONSPINNABLE - Writers cannot spin on this lock.
*
* When the rwsem is either owned by an anonymous writer, or it is
* reader-owned, but a spinning writer has timed out, both nonspinnable
* bits will be set to disable optimistic spinning by readers and writers.
* In the later case, the last unlocking reader should then check the
* writer nonspinnable bit and clear it only to give writers preference
* to acquire the lock via optimistic spinning, but not readers. Similar
* action is also done in the reader slowpath.
* When a writer acquires a rwsem, it puts its task_struct pointer
* into the owner field. It is cleared after an unlock.
*
* When a reader acquires a rwsem, it will also puts its task_struct
* pointer into the owner field with the RWSEM_READER_OWNED bit set.
* On unlock, the owner field will largely be left untouched. So
* for a free or reader-owned rwsem, the owner value may contain
* information about the last reader that acquires the rwsem.
*
* That information may be helpful in debugging cases where the system
* seems to hang on a reader owned rwsem especially if only one reader
* is involved. Ideally we would like to track all the readers that own
* a rwsem, but the overhead is simply too big.
*
* Reader optimistic spinning is helpful when the reader critical section
* is short and there aren't that many readers around. It makes readers
* relatively more preferred than writers. When a writer times out spinning
* on a reader-owned lock and set the nospinnable bits, there are two main
* reasons for that.
*
* 1) The reader critical section is long, perhaps the task sleeps after
* acquiring the read lock.
* 2) There are just too many readers contending the lock causing it to
* take a while to service all of them.
*
* In the former case, long reader critical section will impede the progress
* of writers which is usually more important for system performance. In
* the later case, reader optimistic spinning tends to make the reader
* groups that contain readers that acquire the lock together smaller
* leading to more of them. That may hurt performance in some cases. In
* other words, the setting of non
| 