samba.git/source3/lib/pthreadpool, branch talloc-2.1.2

pthreadpool: Slightly serialize jobs

2014-08-22T22:24:18+00:00

Using the new msg_source program with 1.500 instances against a single
msg_sink I found the msg_source process to spawn two worker threads for
synchronously sending the data towards the receiving socket. This should
not happen: Per destination node we only create one queue. We strictly
only add pthreadpool jobs one after the other, so a single helper thread
should be perfectly sufficient.

It turned out that under heavy overload the main sending thread was
scheduled before the thread that just had finished its send() job. So
the helper thread was not able to increment the pool->num_idle variable
indicating that we don't have to create a new thread when the new job
is added.

This patch moves the signalling write under the mutex. This means that
indicating readiness via the pipe and the pool->num_idle variable happen both
under the same mutex lock and thus are atomic. No superfluous threads anymore.

Signed-off-by: Volker Lendecke 
Reviewed-by: Jeremy Allison

pthreadpool: Allow multiple jobs to be received

2014-03-27T05:06:11+00:00

This can avoid syscalls when multiple jobs are finished simultaneously

Signed-off-by: Volker Lendecke 
Reviewed-by: Jeremy Allison

pthreadpool: Avoid a malloc/free per job

2014-03-27T05:06:11+00:00

pthreadpool_add_job is in our hottest code path for r/w intensive workloads, so
we should avoid anything CPU-intensive. pthreadpool used to malloc each job and
free it in the worker thread. This patch adds a FIFO queue for jobs that helper
threads copy from, avoiding constant malloc/free. This cuts user space
CPU in the local-bench-pthreadpool benchmark by roughly 10% on my system.

Signed-off-by: Volker Lendecke 
Reviewed-by: Jeremy Allison

pthreadpool: Add test for fork crash

2014-03-03T13:31:33+00:00

Signed-off-by: Volker Lendecke 
Reviewed-by: Kamen Mazdrashki 
Reviewed-by: Simo Sorce 
Reviewed-by: Michael Adam

pthreadpool: Fix pthreadpools with fork

2014-03-03T13:29:24+00:00

The current could would crash if a pthreadpool was created, deleted and the
process then fork()s. "pthreadpools" is NULL in this case, but the
pthread_atfork handlers are in place. This fixes walking the pthreadpools list
in reverse.

Signed-off-by: Volker Lendecke 
Reviewed-by: Kamen Mazdrashki 
Reviewed-by: Simo Sorce 
Reviewed-by: Michael Adam

pthreadpool: Fix a comment, "quit"->"shutdown"

2014-01-28T18:06:40+00:00

Signed-off-by: Volker Lendecke 
Reviewed-by: David Disseldorp 

Autobuild-User(master): David Disseldorp 
Autobuild-Date(master): Tue Jan 28 19:06:40 CET 2014 on sn-devel-104

pthreadpool: Fix CID 710828 Sizeof not portable

2013-05-12T13:56:32+00:00

Signed-off-by: Volker Lendecke 
Reviewed-by: Ira Cooper

build: Do not install testing binaries

2012-11-22T11:00:36+00:00

These binaries are for developer or selftest use, and are not
supported for installation onto the system.  The autoconf build does
not install these binaries, and so neither should the waf build.

Andrew Bartlett

Reviewed-by: Andreas Schneider 

Autobuild-User(master): Andreas Schneider 
Autobuild-Date(master): Thu Nov 22 12:00:36 CET 2012 on sn-devel-104

s3: Fix compilation of pthreadpool_sync.c on platforms without pthread.h

2012-06-28T14:43:13+00:00

Signed-off-by: Bjoern Jacke 

Autobuild-User(master): Björn Jacke 
Autobuild-Date(master): Thu Jun 28 16:43:13 CEST 2012 on sn-devel-104

s3: Add a dummy implementation for the pthreadpool API

2012-06-19T17:27:24+00:00

Signed-off-by: Jeremy Allison