Autobuild-User: Kai Blin <kai@samba.org>
Autobuild-Date: Tue Mar  6 21:17:19 CET 2012 on sn-devel-104

Our packet layer relies on the event system reliably telling us when a
packet is available. When we are using a socket layer like TLS then
things get a bit trickier, as there may be bytes in the encryption
buffer which could be read even if there are no bytes at the socket
level. The GNUTLS library is supposed to prevent this happening by
always leaving some data at the socket level when there is data to be
processed in its buffers, but it seems that this is not always
reliable.

To work around this I have added a new packet option
packet_set_unreliable_select() which tells the packet layer to not
assume that the socket layer has a reliable select, and to instead
keep trying to read from the socket until it gets back no data. This
option is set for the ldap client and server when TLS is negotiated.

This seems to fix the problems with the ldaps tests.

metze

list=""
list="$list event_context:tevent_context"
list="$list fd_event:tevent_fd"
list="$list timed_event:tevent_timer"

for s in $list; do
	o=`echo $s | cut -d ':' -f1`
	n=`echo $s | cut -d ':' -f2`
	r=`git grep "struct $o" |cut -d ':' -f1 |sort -u`
	files=`echo "$r" | grep -v source3 | grep -v nsswitch | grep -v packaging4`
	for f in $files; do
		cat $f | sed -e "s/struct $o/struct $n/g" > $f.tmp
		mv $f.tmp $f
	done
done

metze

We should only include events.h where we really need it
and prefer forward declarations of 'struct event_context'

metze

There are still a few tidyups of old FSF addresses to come (in both s3
and s4). More commits soon.
(This used to be commit fcf38a38ac691abd0fa51b89dc951a08e89fdafa)

contexts from the application layer into the socket layer.

This improves a number of correctness aspects, as we now allow LDAP
packets to cross multiple SASL packets.  It should also make it much
easier to write async LDAP tests from windows clients, as they use SASL
by default.  It is also vital to allowing OpenLDAP clients to use GSSAPI
against Samba4, as it negotiates a rather small SASL buffer size.

This patch mirrors the earlier work done to move TLS into the socket
layer.

Unusual in this pstch is the extra read callback argument I take.  As
SASL is a layer on top of a socket, it is entirely possible for the
SASL layer to drain a socket dry, but for the caller not to have read
all the decrypted data.  This would leave the system without an event
to restart the read (as the socket is dry).

As such, I re-invoke the read handler from a timed callback, which
should trigger on the next running of the event loop.  I believe that
the TLS code does require a similar callback.

In trying to understand why this is required, imagine a SASL-encrypted
LDAP packet in the following formation:

+-----------------+---------------------+
| SASL  Packet #1 | SASL Packet #2      |
----------------------------------------+
| LDAP Packet #1       | LDAP Packet #2 |
----------------------------------------+

In the old code, this was illegal, but it is perfectly standard
SASL-encrypted LDAP.  Without the callback, we would read and process
the first LDAP packet, and the SASL code would have read the second SASL
packet (to decrypt enough data for the LDAP packet), and no data would
remain on the socket.

Without data on the socket, read events stop.  That is why I add timed
events, until the SASL buffer is drained.

Another approach would be to add a hack to the event system, to have it
pretend there remained data to read off the network (but that is ugly).

In improving the code, to handle more real-world cases, I've been able
to remove almost all the special-cases in the testnonblock code.  The
only special case is that we must use a deterministic partial packet
when calling send, rather than a random length.  (1 + n/2).  This is
needed because of the way the SASL and TLS code works, and the 'resend
on failure' requirements.

Andrew Bartlett
(This used to be commit 5d7c9c12cb2b39673172a357092b80cd814850b0)

This reduces caller complexity, because the TLS code is now called
just like any other socket.  (A new socket context is returned by the
tls_init_server and tls_init_client routines).

When TLS is not available, the original socket is returned.

Andrew Bartlett
(This used to be commit 09b2f30dfa7a640f5187b4933204e9680be61497)

enabling of packet serialisation
(This used to be commit 6a47cd65a8b588f9ddd375c57caaba08281e7cbb)