Re: read-write locks

[Torvald Riegel]

On Fri, 2017-01-06 at 15:25 +0100, Bruno Haible wrote:
> Torvald Riegel wrote:
> > > The rwlock writer starvation problem is not solved by throttling to a 
> > > fixed
> > > percentage of CPU time: If every reader thread spends 10% of its time 
> > > with the
> > > rwlock held, it will work fine with 4 CPUs, but it will hang with 24 CPUs.
> > 
> > Well, obviously, you need to throttle in such a way that all work can be
> > performed eventually before new work arrives.  For example, don't accept
> > new work for a while if old work hasn't been done yet.
> 
> You suggest to implement throttling, but in order to make it work reliably,
> you have to implement a system of dynamic control (of multiple parameters)
> around it. Such systems are *very hard* to build. Just two examples:
>   - It took a long long time until the OOM killer in Linux could reliably
>     prevent fork bombs from taking down a system.
>   - The Mac OS X 10.5 memory management example from my other mail.
> I don't want to get into this area, unless absolutely absolutely necessary.

All sequential execution "throttles" in the sense of ensuring that all
work is performed before new work is allowed to complete.  Fork/join
parallelism does that when joining.

> > The examples you
> > bring up, such as GC, suggest that you want abstractions at a much
> > higher level than explicit threading and locks, as provided by POSIX and
> > C11.  However, you previously also said that you want to make POSIX and
> > ISO C easier to use, so there's a gap there.
> 
> You're probably right here. I am dissatisfied with how hard it is to make
> multithreaded software work in a verifiably reliable way. I don't know what
> the answer is, i.e. what abstractions will provide this reliability.

Focus on simple forms of parallelism first.  When you have to deal with
actual concurrency (eg, a web server that has to serve http requests
concurrently), transform it into a simple parallel problem, if possible
(eg, put all requests into a bounded work queue, from which parallel
tasks pull work).