samba.git/source3/utils/status_profile.c, branch talloc-2.3.4

s3:utils: Fix buffer size for snprintf and format string

2017-08-09T11:37:47+00:00

GCC 7.1 produces an error:
‘snprintf’ output between 47 and 66 bytes into a destination of size 40

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12930

Signed-off-by: Andreas Schneider 
Reviewed-by: Ralph Boehme 

Autobuild-User(master): Ralph Böhme 
Autobuild-Date(master): Wed Aug  9 13:37:47 CEST 2017 on sn-devel-144

s3:smbprofile: Replace sysv shmem with tdb

2015-03-06T11:31:10+00:00

What?

This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.

Why?

At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.

The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.

But isn't it slow?

Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.

There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.

The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.

Pair-Programmed-With: Stefan Metzmacher 

Signed-off-by: Volker Lendecke 
Signed-off-by: Stefan Metzmacher 
Reviewed-by: Ralph Boehme

s3:smbprofile: specify SMBPROFILE_STATS_SECTION_START() with name vs. display[name]

2015-03-06T11:31:10+00:00

Signed-off-by: Stefan Metzmacher 
Reviewed-by: Volker Lendecke 
Reviewed-by: Ralph Boehme

s3:smbprofile: rewrite the internal macros

2014-11-19T19:51:37+00:00

We now autogenerate a lot of code using
SMBPROFILE_STATS_ALL_SECTIONS macro which expands to
different SMBPROFILE_STATS_{COUNT,BASIC,BYTES,IOBYTES} macros.

This also allows async profiling using:

   struct mystate {
       ...

       SMBPROFILE_BASIC_ASYNC_STATE(profile_state);
       ...
   };

   ...

   SMBPROFILE_BASIC_ASYNC_START(SMB2_negotiate, profile_p, mystate->profile_state);

   ...

   SMBPROFILE_BYTES_ASYNC_SET_IDLE(mystate->profile_state);

   ...

   SMBPROFILE_BYTES_ASYNC_SET_BUSY(mystate->profile_state);

   ...

   SMBPROFILE_BASIC_ASYNC_END(mystate->profile_state);

The current START_PROFILE*()/END_PROFILE*() are implemented as legacy wrappers.

Signed-off-by: Stefan Metzmacher

s3:smbprofile: Make "status_profile.h" a proper header

2014-11-19T19:51:37+00:00

Signed-off-by: Volker Lendecke 
Reviewed-by: Stefan Metzmacher 
Reviewed-by: Jeremy Allison

s3:smbd: improve writecache profiling

2014-11-19T19:51:37+00:00

In order to have useful profiling counters should never be decremented.
We need a separate counter for deallocation events.

The current value can be calculated by allocations - deallocations.

We also use better names and avoid having an array for the flush reasons.
This will simplify further profiling improvements a lot.

The value writecache_num_write_caches (this was similar to writecache_allocations)
is replaced by writecache_cached_writes, which counts the amount of writes which
were completely handled by the cache.

Signed-off-by: Stefan Metzmacher 
Reviewed-by: Jeremy Allison

s3:smbprofile: remove unused nmbd related counters

2014-11-19T19:51:36+00:00

Signed-off-by: Stefan Metzmacher 
Reviewed-by: Jeremy Allison

status: 80 chars per line

2014-10-07T12:44:05+00:00

Signed-off-by: Volker Lendecke 
Reviewed-by: David Disseldorp

profiling: Only compile utils/status_profile.c if profiling is enabled

2014-10-03T17:55:09+00:00

This conditional compile avoids some #ifdef WITH_PROFILE, which makes the code
more readable

Signed-off-by: Volker Lendecke 
Reviewed-by: Jeremy Allison

Rename the profile enums with a SAMBA_ prefix to avoid conflict with system files.

2013-11-22T16:56:38+00:00

WRITE_FLUSH is defined in fs.h in Linux.

Signed-off-by: Jeremy Allison 
Reviewed-by: David Disseldorp