summaryrefslogtreecommitdiff
path: root/python/samba/emulate
AgeCommit message (Collapse)AuthorFilesLines
2023-06-23python:samba:emulate: Fix code spellingAndreas Schneider1-5/+5
Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2023-02-23python: fix mutable default argumentsRob van der Linde1-1/+3
Signed-off-by: Rob van der Linde <rob@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Thu Feb 23 23:33:46 UTC 2023 on atb-devel-224
2022-05-10python: Remove unnecessary 'pass' statementsJoseph Sutton1-2/+0
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz> Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-28python: remove all 'from __future__ import division'Douglas Bagnall1-1/+0
This made '//' and '/' in Python 2 behave as in Python 3. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-04-28python: remove all 'from __future__ import print_function'Douglas Bagnall1-1/+1
This made Python 2's print behave like Python 3's print(). In some cases, where we had: from __future__ import print_function """Intended module documentation...""" this will have the side effect of making the intended module documentation work as the actual module documentation (i.e. becoming __doc__), because it is once again the first statement in the module. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-10-02python2 reduction: Merge remaining compat code into commonDavid Mulder1-1/+1
The remaining compat code (get_string, get_bytes, cmp) are useful helper routines which we should simply merge into common (especially since there is some duplication here). Signed-off-by: David Mulder <dmulder@suse.com> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Autobuild-User(master): David Mulder <dmulder@samba.org> Autobuild-Date(master): Fri Oct 2 14:49:36 UTC 2020 on sn-devel-184
2020-04-27traffic_packets: fix SyntaxWarning: "is" with a literalDavid Disseldorp1-1/+1
Python 3.8 adds this warning via https://bugs.python.org/issue34850: the "is" and "is not" operator sometimes is used with string and numerical literals. This code "works" on CPython by accident, because of caching on different levels (small integers and strings caches, interned strings, deduplicating constants at compile time). But it shouldn't work on other implementations, and can not work even on early or future CPython versions. Reported-by: L. van Belle <belle@samba.org> Signed-off-by: David Disseldorp <ddiss@samba.org> Reviewed-by: Noel Power <noel.power@suse.com> Autobuild-User(master): David Disseldorp <ddiss@samba.org> Autobuild-Date(master): Mon Apr 27 12:19:59 UTC 2020 on sn-devel-184
2019-09-24traffic.py: avoid inefficient string concatenationsBjörn Jacke1-3/+1
Signed-off-by: Bjoern Jacke <bjacke@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24traffic_replay: Avoid DB full scans in LDAP searchesTim Beale2-0/+63
When generating LDAP search traffic, a full DB scan can be very costly. Avoiding full-scan LDAP searches means that we can run traffic_replay against a 100K user DB and get some sane results. Because the traffic_learner doesn't record the LDAP search filter at all, the traffic_replay LDAP searches default to being full scans. Doing full scans meant that the LDAP search was usually the first packet type to exceed the max latency and fail the test. It could also skew results for the other packet types by creating big demands on memory/CPU/ DB-lock-time. It's hard to know for sure exactly what real-world LDAP searches will look like, but let's assume full scan searches will be fairly rare. In traffic-model files we've collected previously, some of the attributes are fairly unique (e.g. pKIExtendedKeyUsage), and as there are some LDAP queries specified in MS specs (such as MS-GPOL and MS-WCCE), it allows us to infer what the search filter might be. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24traffic replay: Store the instance id in the replay contextGary Lockyer1-1/+3
Store the traffic runner instance id in the replay context. Will be used in subsequent commits. Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24traffic_replay: Make use of SCOPE_BASE explicitTim Beale1-2/+2
i.e. avoid hard-coded numbers. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24traffic_replay: Store total conversations on the replay contextTim Beale1-0/+3
This is useful info to know, and will be used in subsequent commits. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-01traffic: load dns query from file and write stats to fileJoe Guo1-13/+69
Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed May 1 01:10:42 UTC 2019 on sn-devel-184
2019-04-30traffic: define kerberos_state to simplify codeJoe Guo1-25/+10
Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
2019-04-30traffic: make code more pythonicJoe Guo1-9/+3
Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
2019-01-08traffic_replay: use packets per second as primary scaleDouglas Bagnall1-6/+14
The old -S/--scale-traffic is relative to the original model, which made its relationship to true traffic volumes quite opaque Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: add option to reanimate dying conversationsDouglas Bagnall1-4/+20
The traffic model is generated from a window in time, which makes conversations appear to start and stop unnaturally at the window boundaries. When the window is short compared to the traffic replay time and the true expected conversation length, this has a significant distorting effect, leading to more conversations than would be expected to generate a given number of packets. To offset this slightly we add the --conversation-persistence option which tries to convert apparent death into a longish wait. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: Conversation.add_short_packet is discerning about packetsDouglas Bagnall1-1/+4
If the packets really wouldn't do anything, we might as well not add them. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: remove useless branch in stats reportDouglas Bagnall1-25/+12
This completes the work of 68c64c634ad7576b2ab5ef838c203320afd90f44, but differs from that in that it makes no actual change because isatty was not being called so was always evaluated as true. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: generate more statistics in replayDouglas Bagnall1-30/+69
Add more "header" values indicating the progress of the run as a whole. The new fields are: Max sleep miss - the longest sleep() oversleep. Indicates client load. Maximum lag - the longest gap between a planned packet time and its actual time. Start lag - the longest gap between intended and actual conversation start. Planned conversations - how many conversations we meant to have. Planned packets - how many "packets" we thought we were making. Not all "packets" result in actual operations or packets. Unfinished conversations - how many conversations had not finished when they were killed. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: avoid bare except: clausesDouglas Bagnall1-2/+2
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: simplify tests for badpassword_frequencyDouglas Bagnall1-3/+4
x <= 0 will fail one or both of the other test clauses. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: assign context domain at startDouglas Bagnall1-7/+1
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: traffic_replay --latency-timeout to control final waitDouglas Bagnall1-1/+2
Conversations that haven't finished within some acceptable margin of on-time can be said to have failed. This is where you specify that margin. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic_replay: --stop-on-any-error option to not ignore client troubleDouglas Bagnall1-0/+3
Sometimes you want to know if any client is crashing for any reason. In those times use --stop-on-any-error for an early exit. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: evoke OS error codes by name, not numberDouglas Bagnall1-2/+3
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: reinitialise RNG in each childDouglas Bagnall1-2/+8
Otherwise they all replay using the same random sequence. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: avoid generating conversations without packetsDouglas Bagnall1-0/+28
Some "packets" don't generate any actual traffic. If we have a conversation consisting only of those, we can avoid forking a client for it. This *slightly* increases the load over that which would be generated otherwise for a given traffic rate, but that's OK. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: rework conversation generation to better use memoryDouglas Bagnall1-149/+173
Use less memory altogether and don't allocated shared mutable before the fork. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: generate sparser descriptions of conversationsDouglas Bagnall1-4/+3
Rather than building all the packets at this point, we stick to the barest details of the packets (which is all the model gives us anyway). The advantage is that will take a lot less memory, which matters because this process forks into many clients that were sharing and mutate the conversation list. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: new version of model with packet_rate, version numberDouglas Bagnall1-46/+72
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: fix hash non-determinism when loading JSONDouglas Bagnall1-0/+2
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Tim Beale <timbeale@samba.org>
2019-01-08traffic: Conversation gets given id, list of short packetsDouglas Bagnall1-8/+8
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: rename packet_rate -> replay_speed for accuracy and roomDouglas Bagnall1-5/+6
We are soon going to have a self.packet_rate, and replay_speed is more accurate in this case. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: initialise conversation specific lists after the forkDouglas Bagnall1-12/+12
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: split is_a_real_packet() function out of classDouglas Bagnall1-17/+21
So we can use it to determine whether a packet should be a Packet before making the leap. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: give Packet __slots__ for reduced memoryDouglas Bagnall1-1/+10
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: make random_colour_print more deterministic, silent on demandDouglas Bagnall1-9/+21
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: use namedtuple for ConversationAccounts namespace classDouglas Bagnall1-9/+10
Saving memory, which reduces fork overhead. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: fix mk_masked_dir doc and commentsDouglas Bagnall1-1/+2
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08traffic: Packet.from_line classmethod uses cls varDouglas Bagnall1-3/+3
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-11various: Remove references to about to be deleted thirdparty/dnspythonNoel Power1-1/+0
Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-04traffic_replay: Add a max-members option to cap group sizeTim Beale1-4/+30
traffic_replay tries to distribute the users among the groups in a realistic manner - some groups will have almost all users in them. However, this becomes a problem when testing a really large database, e.g. we may want 100K users, but no more than 5K users in each group. This patch adds a max-member option so we can limit how big the groups actually get. If we detect that a group exceeds the max-members, we reset the group's probability (of getting selected) to zero, and then recalculate the cumulative distribution. The means that the group should no longer get selected by generate_random_membership(). (Note we can't completely remove the group from the list because that changes the list-index-to-group-ID mapping). Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Tue Dec 4 12:22:50 CET 2018 on sn-devel-144
2018-12-04traffic: Rework how assignments are generated slightlyTim Beale1-19/+14
We want to cap the number of members that can be in a group. But first, we need to tweak how the assignment dict gets generated, so that we get rid of the intermediary set. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07traffic_replay: Rework machine accounts to remove redundant codeTim Beale1-55/+5
generate_users_and_groups() now generates the machine acounts as well as the user accounts, so it seems there's no need to also have generate_traffic_accounts(), which does the same job. Instead, we can just pass through the number of machine acounts to generate_users_and_groups() and delete the other function. Also updated generate_users_and_groups() so that machine_accounts is no longer optional (we want to create machine accounts in all cases). Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07traffic_replay: Make packet generation work on a pre-populated DB againTim Beale1-3/+13
Generate separate machine accounts for populating a large DB vs replaying network traffic. We want to use different userAccountControl flags in each of the above cases (i.e. commit 3338a3e257fa9f28). However, this means that once you use the --generate-users-only option, you can't replay network packets against the machine accounts. We can avoid this problem by creating separate machine accounts for each of 2 different cases, e.g. STGM-0-x machines for traffic-replay, and PC-0-x machines for padding out the database. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07traffic_replay: Make sure naming assumptions are in a single placeTim Beale1-7/+11
The traffic_replay group/user/machine account names follow a standard format. This adds a function to generate the machine-name. It also makes sure the existing user_name() function gets called in all applicable places. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07traffic_replay: Move 'traffic account' flag up a levelTim Beale1-4/+7
We create machine accounts for 2 different purposes: 1). For traffic generation, i.e. testing realistic network packets. 2). For generating a realistic large DB. Unfortunately, we want to use different userAccountControl flags for the 2 different cases. Commit 3338a3e257fa9f28 changed the flags used for case #2, but this breaks case #1. The problem is generate_users_and_groups() is called in both cases, so we want the 'traffic account' flag passed into that function. This ensures that the machine accounts get created with the appropriate userAccountControl flags for the particular case you want to test. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07traffic_replay: Move machine account creationTim Beale1-7/+7
I was assuming that generate_users_and_groups() only gets called in the --generate-users-only case. However, it also gets called in the default traffic replay case. This patch reworks the code so that the number of machine accounts to create gets passed in, and the 'create 25% more computers than users' assumption only applies to the --generate-users-only case. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-05traffic: Machine accounts were generated as critical objectsTim Beale1-5/+16
Due to the userAccountControl flags we were specifying, the machine accounts were all created as critical objects. When trying to populate 1000s of machine accounts in a DB, this makes replication unnecessarily slow (because it has to replicate them all twice). This patch changes it so when we're just creating machine accounts for the purpose of populating a semi-realistic DB, we jsut use the default WORKSTATION_TRUST_ACCOUNT flag. Note that for the accounts used for traffic-replay, we apparently need the existing flags in order for the DC to accept certain requests. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Autobuild-User(master): Tim Beale <timbeale@samba.org> Autobuild-Date(master): Mon Nov 5 03:43:24 CET 2018 on sn-devel-144
generated by cgit v1.2.3 (git 2.25.1) at 2026-05-07 07:30:34 +0000