samba.git/python/samba/drs_utils.py, branch talloc-2.3.2

drs_utils: Avoid invalid dereference of v8 requests

2018-11-06T06:15:33+00:00

req.more_flags only exists for v10 requests, so we throw an exception if
we try to dereference that field on a v8 (or v5) request. Unfortunately,
we were checking that we support v10 *after* we had tried to access the
more_flags. This patch fixes up the order of the checks.

This may be a problem trying to replicate with an older Windows DC
(pre-2008R2), and was reported on the samba mailing-list at one point:
https://lists.samba.org/archive/samba/2018-June/216541.html

Unfortunately this patch doesn't help the overall situation at all (the
join will fail because we can't resolve the link target and we can't use
GET_TGT). But it now gives you a more meaningful error, i.e.

  ERROR(runtime): uncaught exception - (8639, "Failed to process 'chunk'
    of DRS replicated objects: DOS code 0x000021bf"
instead of:
  ERROR(): uncaught exception -
    'drsuapi.DsGetNCChangesRequest8' object has no attribute 'more_flags'

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 

Autobuild-User(master): Tim Beale 
Autobuild-Date(master): Tue Nov  6 07:15:33 CET 2018 on sn-devel-144

drs_utils: Fix some long lines

2018-11-06T02:39:11+00:00

Tweak the code slightly to avoid some 80+ character lines.

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett

drs_util: Improve memory usage when joining large DB

2018-10-17T06:56:42+00:00

drs_Replicate.replicate() could consume a large amount of memory when
replicating a large DB. This is not a leak - the memory gets freed when
the function returns (i.e. once the partition is fully replicated).
However, while the partition is in the process of being replicated, it
accumulates memory for each replication chunk it receives. This can have
considerable overhead with 1000s of objects/links in the partition.

This was exhausting memory when joining a VM with 1Gb RAM to a DC with
25K users (average ~15 group memberships per user).

It seems that by storing a reference to something that's on the ctr's
talloc tree, it doesn't free up the memory for each ctr message (until
the function actually returns and req is destroyed).

With 10K users (and average 15 group memberships per user), .replicate()
consumed 211Mb of memory, according to talloc.report_full(). With this
patch, it goes down to just the current ctr message (1-2Mb).

Signed-off-by: Tim Beale 
Reviewed-by: Garming Sam 
Reviewed-by: Andrew Bartlett 

Autobuild-User(master): Andrew Bartlett 
Autobuild-Date(master): Wed Oct 17 08:56:42 CEST 2018 on sn-devel-144

s4/torture/drs: PY3 port for samba4.drs.replica_sync_rodc

2018-09-06T13:51:35+00:00

Signed-off-by: Noel Power 
Reviewed-by: Andrew Bartlett abartlet@samba.org

PEP8: fix E703: statement ends with a semicolon

2018-08-24T05:49:30+00:00

Signed-off-by: Joe Guo 
Reviewed-by: Andrew Bartlett 
Reviewed-by: Douglas Bagnall

PEP8: fix E401: multiple imports on one line

2018-08-24T05:49:30+00:00

Signed-off-by: Joe Guo 
Reviewed-by: Andrew Bartlett 
Reviewed-by: Douglas Bagnall

PEP8: fix E302: expected 2 blank lines, found 1

2018-08-24T05:49:29+00:00

Signed-off-by: Joe Guo 
Reviewed-by: Andrew Bartlett 
Reviewed-by: Douglas Bagnall

PEP8: fix E128: continuation line under-indented for visual indent

2018-08-24T05:49:27+00:00

Signed-off-by: Joe Guo 
Reviewed-by: Andrew Bartlett 
Reviewed-by: Douglas Bagnall

drs_utils: Always set the GET_TGT flag for clone renames

2018-07-05T02:01:25+00:00

The DCCloneAndRenameContext replication was a little inefficient, in
that it would essentially replicate the entire DB twice. This was due to
resolving the link targets - it finds a target object it doesn't know
about, so retries the entire replication again with the GET_TGT flag set
this time.

Normally, the repl_meta_data code will use the target object's GUID,
however, it can't do this for cross-partition links (if it hasn't
replicated the target partition yet). The repl_md code can normally
detect that the link is a cross-parition link by checking the base-DN,
however, this doesn't work in the DCCloneAndRenameContext case because
we have renamed the base-DN.

This is not a big deal - it just means extra work. However, because the
domains being backed up could potentially be quite large, it probably
makes sense to just always set the GET_TGT in the rename case and skip
this extra work.

Signed-off-by: Tim Beale 
Reviewed-by: Gary Lockyer 
Reviewed-by: Andrew Bartlett

drs_utils: Add infrastructure to support 'clone with rename'

2018-07-03T08:39:14+00:00

Our end goal is to create a backup clone of a DB, but rename the
domain/realm so we can startup the backup DC without interferring with
the existing Samba network. The basic strategy to do this is to leverage
DRS replication - by renaming the first object in the partition, all
subsequent objects will automatically be renamed.

This patch adds the infrastructure to do this. I've used object
inheritance to handle the special case of renaming the partition
objects. This means the domain-rename special case doesn't really
pollute the existing DRS replication code. All it needs is a small
refactor to create a new 'process_chunk()' function that the new
sub-class can then override.

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 
Reviewed-by: Gary Lockyer