MESSAGE
| DATE | 2015-05-02 |
| FROM | Ruben Safir
|
| SUBJECT | Subject: [NYLXS - HANGOUT] Great Article on Software Concordance program writing
|
http://www.linuxjournal.com/article/5833
just the first page
Kernel Locking Techniques
Issue 100
From Issue #100
August 2002
Aug 01, 2002 By Robert Love
in
* SysAdmin
Robert explains the various locking primitives in the Linux kernel, why
you need them and how kernel developers can use them to write safe code.
Proper locking can be tough—real tough. Improper locking can result in
random crashes and other oddities. Poorly designed locking can result in
code that is hard to read, performs poorly and makes your fellow kernel
developers cringe. In this article, I explain why kernel code requires
locking, provide general rules for proper kernel locking semantics and
then outline the various locking primitives in the Linux kernel.
Why Do We Need Locking in the Kernel?
The fundamental issue surrounding locking is the need to provide
synchronization in certain code paths in the kernel. These code paths,
called critical sections, require some combination of concurrency or
re-entrancy protection and proper ordering with respect to other events.
The typical result without proper locking is called a race condition.
Realize how even a simple i++ is dangerous if i is shared! Consider the
case where one processor reads i, then another, then they both increment
it, then they both write i back to memory. If i were originally 2, it
should now be 4, but in fact it would be 3!
This is not to say that the only locking issues arise from SMP
(symmetric multiprocessing). Interrupt handlers create locking issues,
as does the new preemptible kernel, and any code can block (go to
sleep). Of these, only SMP is considered true concurrency, i.e., only
with SMP can two things actually occur at the exact same time. The other
situations—interrupt handlers, preempt-kernel and blocking
methods—provide pseudo concurrency as code is not actually executed
concurrently, but separate code can mangle one another's data.
These critical regions require locking. The Linux kernel provides a
family of locking primitives that developers can use to write safe and
efficient code.
SMP Locks in a Uniprocessor Kernel
Whether or not you have an SMP machine, people who use your code may.
Further, code that does not handle locking issues properly is typically
not accepted into the Linux kernel. Finally, with a preemptible kernel
even UP (uniprocessor) systems require proper locking. Thus, do not
forget: you must implement locking.
Thankfully, Linus made the excellent design decision of keeping SMP and
UP kernels distinct. This allows certain locks not to exist at all in a
UP kernel. Different combinations of CONFIG_SMP and CONFIG_PREEMPT
compile in varying lock support. It does not matter, however, to the
developer: lock everything appropriately and all situations will be covered.
Atomic Operators
We cover atomic operators initially for two reasons. First, they are the
simplest of the approaches to kernel synchronization and thus the
easiest to understand and use. Second, the complex locking primitives
are built off them. In this sense, they are the building blocks of the
kernel's locks. Atomic operators are operations, like add and subtract,
which perform in one uninterruptible operation. Consider the previous
example of i++. If we could read i, increment it and write it back to
memory in one uninterruptible operation, the race condition discussed
above would not be an issue. Atomic operators provide these
uninterruptible operations. Two types exist: methods that operate on
integers and methods that operate on bits. The integer operations work
like this:
atomic_t v;
atomic_set(&v, 5); /* v = 5 (atomically) */
atomic_add(3, &v); /* v = v + 3 (atomically) */
atomic_dec(&v); /* v = v - 1 (atomically) */
printf("This will print 7: %d\n", atomic_read(&v));
They are simple. There are, however, little caveats to keep in mind when
using atomics. First, you obviously cannot pass an atomic_t to anything
but one of the atomic operators. Likewise, you cannot pass anything to
an atomic operator except an atomic_t. Finally, because of the
limitations of some architectures, do not expect atomic_t to have more
than 24 usable bits. See the “Function Reference” Sidebar for a list of
all atomic integer operations.
Function Reference
The next group of atomic methods is those that operate on individual
bits. They are simpler than the integer methods because they work on the
standard C data types. For example, consider void set_bit(int nr, void
*addr). This function will atomically set to 1 the “nr-th” bit of the
data pointed to by addr. The atomic bit operators are also listed in the
“Function Reference” Sidebar.
Spinlocks
For anything more complicated than trivial examples like those above, a
more complete locking solution is needed. The most common locking
primitive in the kernel is the spinlock, defined in
include/asm/spinlock.h and include/linux/spinlock.h. The spinlock is a
very simple single-holder lock. If a process attempts to acquire a
spinlock and it is unavailable, the process will keep trying (spinning)
until it can acquire the lock. This simplicity creates a small and fast
lock. The basic use of the spinlock is:
spinlock_t mr_lock = SPIN_LOCK_UNLOCKED;
unsigned long flags;
spin_lock_irqsave(&mr_lock, flags);
/* critical section ... */
spin_unlock_irqrestore(&mr_lock, flags);
The use of spin_lock_irqsave() will disable interrupts locally and
provide the spinlock on SMP. This covers both interrupt and SMP
concurrency issues. With a call to spin_unlock_irqrestore(), interrupts
are restored to the state when the lock was acquired. With a UP kernel,
the above code compiles to the same as:
unsigned long flags;
save_flags(flags);
cli();
/* critical section ... */
restore_flags(flags);
which will provide the needed interrupt concurrency protection without
unneeded SMP protection. Another variant of the spinlock is
spin_lock_irq(). This variant disables and re-enables interrupts
unconditionally, in the same manner as cli() and sti(). For example:
spinlock_t mr_lock = SPIN_LOCK_UNLOCKED;
spin_lock_irq(&mr_lock);
/* critical section ... */
spin_unlock_irq(&mr_lock);
This code is only safe when you know that interrupts were not already
disabled before the acquisition of the lock. As the kernel grows in size
and kernel code paths become increasingly hard to predict, it is
suggested you not use this version unless you really know what you are
doing.
All of the above spinlocks assume the data you are protecting is
accessed in both interrupt handlers and normal kernel code. If you know
your data is unique to user-context kernel code (e.g., a system call),
you can use the basic spin_lock() and spin_unlock() methods that acquire
and release the specified lock without any interaction with interrupts.
A final variation of the spinlock is spin_lock_bh() that implements the
standard spinlock as well as disables softirqs. This is needed when you
have code outside a softirq that is also used inside a softirq. The
corresponding unlock function is naturally spin_unlock_bh().
Note that spinlocks in Linux are not recursive as they may be in other
operating systems. Most consider this a sane design decision as
recursive spinlocks encourage poor code. This does imply, however, that
you must be careful not to re-acquire a spinlock you already hold, or
you will deadlock.
Spinlocks should be used to lock data in situations where the lock is
not held for a long time—recall that a waiting process will spin, doing
nothing, waiting for the lock. (See the “Rules” Sidebar for guidelines
on what is considered a long time.) Thankfully, spinlocks can be used
anywhere. You cannot, however, do anything that will sleep while holding
a spinlock. For example, never call any function that touches user
memory, kmalloc() with the GFP_KERNEL flag, any semaphore functions or
any of the schedule functions while holding a spinlock. You have been
warned.
If you need a lock that is safe to hold for longer periods of time, safe
to sleep with or capable of allowing concurrency to do more than one
process at a time, Linux provides the semaphore.
|
|
