| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
These tests are not affected by the reserved_usn change, so there is
no need to run them twice.
The test_repl_get_tgt_multivalued_links fails with or without
reserved_usn set to zero, but it fails differently in either case.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>
|
|
We don't need a timeout failure any more, since replication should
always work. Leaving the timeout in might sometimes cause a flapping
test if replication is being slow for some reason.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>
|
|
This emulates the behaviour of Azure AD.
As this is quite slow we will later reduce the test load in this case,
but for now we want to run all the getncchanges tests this way.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>
|
|
In the next commit we are going to add tests in which the client
modifies the highwatermark in a way that resets replication (on Samba
only). After that we'll fix it.
If we leave the test in an eternal loop, the commit history will not
be bisectable, so we are temporarily going to turn long waits into
failures.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>
|
|
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
leads and tmp_highest_usn moves
The NC root, on any replication when it appears, is the first object to be
replicated, including for all subsequent chunks in the replication.
However the tmp_highest_usn is not updated by that USN, it must
only be updated for the non-NC changes (to match Windows exactly),
or at least only updated with the non-NC changes until it would
naturally appear.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
|
|
reset the replication cookie
This demonstrates the behaviour used by the "Azure AD Connect" cloud sync tool.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
|
|
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
|
|
It is always better to keep the testing OUs unique if possible.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
|
|
This object is used to hold one of many possible connections and
it is helpful for debugging and uniqueness to know which DC is being
connected to.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15401
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
|
|
and real GUID
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10635
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
|
|
This made Python 2's print behave like Python 3's print().
In some cases, where we had:
from __future__ import print_function
"""Intended module documentation..."""
this will have the side effect of making the intended module documentation
work as the actual module documentation (i.e. becoming __doc__), because
it is once again the first statement in the module.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
TestCase.assertEquals() is an alias for TestCase.assertEqual() and
has been deprecated since Python 2.7.
When we run our tests with in python developer mode (`PYTHONDEVMODE=1
make test`) we get 580 DeprecationWarnings about this.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
|
|
This adds a test-case to highlight a bug in the client side GetNCChanges
handling.
These tests mostly exercise the server-side behaviour of sending the
GetNCChanges, however, there's a bug in the client-side code when we try
to handle a missing cross-partition link target *in combination* with
the GET_TGT flag already having been set.
The test is exercising the client-side code by using the 'samba-tool drs
replicate' command. By adding a one-way link to a deleted target object,
we force the client code to retry with the GET_TGT flag set.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14022
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
|
|
Signed-off-by: Olly Betts <olly@survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
Mostly involves splitting up long strings or comments so that they
span multiple lines. Some place-holder variables have been added in a
few places to avoid exceeding 80 chars.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
|
|
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
|
|
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
|
|
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
|
|
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
|
|
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
Several tests hang all the objects they create off a unique OU.
Having a common OU makes cleanup easier, and having a unique OU (i.e.
adding some randomness) helps protect against one-off test failures
(Replication between testenvs is happening in the background.
Occasionally, when a test finishes on one testenv and moves onto the
next testenv, that testenv may have received the replicated test
objects from the first testenv, but has not received their deletion
yet).
Rather than copy-n-pasting this code yet again, split it out into a
helper function.
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
|
|
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
|
|
In addition to converting the except line another line is also added
for each except to extract the tuple contents.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
|
|
While running the selftests, I noticed a case where DC replication
unexpectedly sends a linked attribute for a deleted object (created in
the drs.ridalloc_exop tests). The problem is due to the
msDS-NC-Replica-Locations attribute, which is a (known) one-way link.
Because it is a one-way link, when the test demotes the DC and deletes
the link target, there is no backlink to delete the link from the source
object.
After much debate and head-scratching, we decided that there wasn't an
ideal way to resolve this problem. Any automated intervention could
potentially do the wrong thing, especially if the link spans partitions.
Running dbcheck will find this problem and is able to fix it (providing
the deleted object is still a tombstone). So the recommendation is to
run dbcheck on your DCs every 6 months (or more frequently if using a
lower tombstone lifetime setting).
However, it does highlight a problem with the current GET_TGT
implementation. If the tombstone object had been expunged and you
upgraded to 4.8, then you would be stuck - replication would fail
because the target object can't be resolved, even with GET_TGT, and
dbcheck would not be able to fix the hanging link. The solution is to
not fail the replication for an unknown target if GET_TGT has already
been set (i.e. the dsdb_repl_flags contains
DSDB_REPL_FLAG_TARGETS_UPTODATE).
It's debatable whether we should add a hanging link in this case or
ignore/drop the link. Some cases to consider:
- If you're talking to a DC that still sends all the links last, you
could still get object deletion between processing the source object's
links and sending the target (GET_TGT just restarts the replication
cycle from scratch). Adding a hanging link in this case would be
incorrect and would add spurious information to the DB.
- Suppose there's a bug in Samba that incorrectly results in an object
disappearing. If other DCs then remove any links that pointed to that
object, it makes recovering from the problem harder. However, simply
ignoring the link shouldn't result in data loss, i.e. replication won't
remove the existing link information from other DCs. Data loss in this
case would only occur if a new DC were brought online, or if it were a
new link that was affected.
Based on this, I think ignoring the link does the least harm.
This problem also highlights that we should really be using the same
logic in both the unknown target and the deleted target cases.
Combining the logic and moving it into a common
replmd_allow_missing_target() function fixes the problem. (This also has
the side-effect of fixing another logic flaw - in the deleted object
case we would unnecessarily retry with GET_TGT if the target object was
in another partition. This is pointless work, because GET_TGT won't
resolve the target).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
Add a test where a source object links to multiple different targets.
First we do the replication without GET_TGT and check that the server
can handle sending a chunk containing only links (in the middle of the
replication). Then we repeat the replication forcing GET_TGT to be used.
To avoid having to create 1500 objects/links, I've lowered the 'max
link sync' setting on the vampire_dc testenv to 250.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
Samba would drop linked attributes that span partitions if it didn't
know about the target object. This patch adds a test that exposes the
problem.
I've re-used the code from the previous re-animation test to do this.
I've also added a very basic DcConnection helper class that basically
stores the connection state information the drs_base.py uses for
replication. This allows us to switch the DC we want to replicate from
easily. This approach could potentially be retro-fitted to some of the
existing test cases, as it allows us to test both the DRS client code
and server code at the same time.
Note this test case relates to the code change for commit
fae5df891c11f642cb.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
Reading between the lines, this scenario seems to be the main reason
that Microsoft added the GET_TGT flag. MS AD can handle getting links
for unknown targets OK, but if it receives links for a deleted/recycled
target then it would tend to drop the received links. Samba client also
used to drop the links if talking to a Microsoft DC (or a Samba server
with GET_TGT support).
The specific scenario is the client side already knows about a deleted
object. That object is then re-animated and used as the target for a
linked attribute. *Then* the target object gets updated again so it gets
sent in a later replication chunk to the linked attribute, i.e. the
client receives the link before it learns that the target object has
been re-animated.
In this test we're interested in particular at how the client behaves
when it receives a linked attribute for a deleted object. (It *should*
retry with GET_TGT to make sure the target is up-to-date. However, it
was just dropping the linked attribute).
To exercise the client-side, we disable replication, setup the
links/objects on one DC the way we want them, then force a replication
to the second DC. We then check that when we query each DC, they both
tell us about the links/objects we're expecting (i.e. no links got
lost).
Note that this wasn't a problem with older versions of Samba-to-Samba
because sending the links last guaranteed that the target objects were
always up-to-date.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
Add tests that delete the source and target objects for linked
attributes in the middle of a replication cycle.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
The code has to handle needing GET_ANC and GET_TGT in combination, i.e.
where we fetch the target object for the linked attribute and the target
object's parent is unknown as well. This patch adds a test case to
exercise this code path.
The second part of this test exercises GET_ANC/GET_TGT for an
incremental replication, where the objects are getting filtered by an
uptodateness-vector/HWM.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
We have identified a case where the Samba server can send linked
attributes but not the target object. In this case, the Samba DRS client
would hit the "Failed to re-resolve GUID" case in replmd and silently
discard the linked attribute.
However, Samba will resend the linked attribute in the next cycle
(because its USN is still higher than the committed HWM), so it should
recover OK. On older releases, this may have caused problems if the
first error resulting in a hanging link (which might mean the second
time it's processed it still fails to be added).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
test_repl_get_tgt:
- Adds 2 sets of objects
- Links one set to the other
- Changes the order so the target object comes last in the
replication (which means the client has to use GET_TGT)
- Checks that when GET_TGT is used that we have received all target
objects we need to resolve the linked attibutes
- Checks that we expect to receive the linked attributes *before*
the last chunk is sent (by default, Samba sends all the links at
the end, so this fails)
- Checks that we eventually receive all expected objects, and all
links we receive match what is expected
test_repl_get_tgt_chain:
This adds the linked attributes in a more complicated chain. We add
300 objects, but the links for 100 objects will point to a linked
chain of 200 objects.
This was mainly to determine whether or not Windows follows the
target object (i.e. whether it sends all the links for the target
object as well). It turns out Windows maintains its own linked
attribute DB, so it sends the links based on USN.
Note that the 2 testenvs fail for different reasons. promoted_dc fails
because it is sending all the linked attributes last. vampire_dc fails
because it doesn't support GET_TGT yet, so it sends the link before the
peer knows about the target object.
Note that to test against vampire_dc (rather than the ad_dc_ntvfs DC),
we need to send the GetNCChanges requests to DC2 instead of DC1.
I've left the DC numbering scheme as is, but I've addeed a test_ldb_dc
handle to drs_base.py - it defaults to DC1, but tests can override it
easily and still have everything work.
While running the new tests through autobuild, I noticed an intermittent
LDAP_ENTRY_ALREADY_EXISTS failure in the test setup(). This appears to
be due to a timing issue in the background replication between the
multiple testenvs. Adding some randomness so that the test base OU is
unique seems to avoid the problem.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
|
|
Add a basic test that when we use GET_ANC and the parents have linked
attributes, then we receive all the expected links and all the expected
objects by the end of the test.
This extends the test code to track what linked attributes get received
and check whether they match what's present on the DC.
Also made some minor cleanups to store the received objects/links each
time we successfully receive a GETNCChanges response (this saves the
test case having to repeat this code every time).
Note that although this test involves linked attributes, it shouldn't
exercise the GET_TGT case at all.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
|
|
This test:
- creates blocks of parent/child objects
- modifies the parents, so the child gets received first in the
replication (which means the client has to use GET_ANC)
- checks that we always receive the parent before the child (if not, it
either retries with GET_ANC, or asserts if GET_ANC is already set)
- modifies the parent objects to change their USN while the
replication is in progress
- checks that all expected objects are received by the end of the
test
I've added a repl_get_next() function to help simulate a client's
behaviour - if it encounters an object it doesn't know the parent of,
then it retries with GET_ANC.
Also added some debug to drs_base.py that developers can turn on to make
it easier to see what objects we're actually receiving in the
responses.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
|
|
This adds a new test to check that if objects are modified during a
replication, then those objects don't wind up missing from the
replication data.
Note that when this scenario occurs, samba returns the objects in a
different order to Windows. This test doesn't care what order the
replicated objects get returned in, so long as they all have been
received by the end of the test.
As part of this, I've refactored _check_replication() in drs_base.py so
it can be reused in new tests. In these cases, the objects are split up
over multiple different chunks. So asserting that the objects are returned
in a specific order makes it difficult to run the same test on both Samba
and Windows.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
|
