Establishing server connections

Introduction
------------

This paper is part of the "Zorp Anatomy" series of articles and as such it
gives some background information about one or more Zorp subsystems. It is
intended for advanced Zorp users, those interested in what behind the scene
is.

Proxy startup
-------------

As soon as the proxy starts up as described in the article "Incoming
connection handling" it takes control of the incoming connection and
continues processing according to the corresponding protocol specification.

Some protocols immediately require a server side connection (for example
POP3 or FTP), others only need the server side after the first protocol
element is read (for example HTTP).

Connecting to the server is initiated by the proxy when it sees fit, but the
connection is established by the policy layer, more specifically the
Proxy.setServerAddress and Proxy.connectServer methods.

The first method (setServerAddress) is used by the proxy to hint the policy
layer about its own idea of the destination. This value is used when the
policy does not specify a destination. For example non-transparent HTTP
proxying specifies the server name in the HTTP request.

The second method (connectServer) actually establishes the connection based
on information previously set by the Router and setServerAddress.

Establishing the actual connection
----------------------------------

As described in the previous section the proxy calls connectServer method
does the following things in order:

1) it checks whether this proxy is a stacked proxy in which case the server
   side connection is already prepared. If it was a stacked proxy the
   prepared connection is returned and processing ends here.
  
NOTE: if the proxy is not stacked, the service must have an associated Chainer
      object. A Chainer class is derived from AbstractChainer and defaults to a
      ConnectChainer instance if the service definition does not specify otherwise.

NOTE: the router and the proxy instances must have placed their
      source/destination address hints by this time. This information is
      stored in the session object.

2) the chainParent method of the chainer object is called, which in turn
   performs address translation (see below for more details about NAT) and
   starts connecting to the server by calling
   ConnectChainer.establishConnection

3) establishConnection resolves the server zone and performs access control
   checks.

4) a connection is initiated and processing blocks while the connection
   either succeeds or fails.
   
5) the return status is propagated back to the calling proxy

Chainer classes
---------------

Connection establishment relies heavily on various Chainer classes. While
the default ConnectChainer is fine for most purposes, some extra
functionality can easily be added at this processing point.

For example FailoverChainer uses a list of server addresses and connects
them in a round-robin fashion. While FailoverChainer is very simple, it does
not remember failed hosts and will try connecting to them again and again,
it is very easy to add state. 

A more complex example is SideStackChainer which is used to chain proxies
side-by-side, it is detailed in the article "Stacking explained".

NAT functions
-------------

Network address translation in the context of proxy firewalls differs
from the NAT functions a usual packet filtering solution gives us. For
example a proxy firewall without extra processing masquerades the client
showing its own IP address to the server. The explanation is very simple:
proxies use a separate connection to connect to the server.

There are some cases however when this masquerading should be turned off,
for example the webserver in our DMZ might want to implement some kind of
access control based on client IP address, and this is not possible when the
firewall masquerades the client. The original IP address must be shown in
this case: the forge_addr parameter of all router classes enable this
function.

There is more than that, Zorp is able to perform general address translation
on either the source or the destination address just before the connection
is started, changing previous router/proxy decisions.

Each service can have an associated NATPolicy for SNAT and DNAT. A NATPolicy
describes address mappings like "map the public 5.6.7.0/24 address range to
the private 192.168.0.0/24". Example:

	NATPolicy("intra-inter", 
                  cacheable=TRUE,
                  nat=GeneralNAT([(InetDomain('5.6.7.0/24'), InetDomain('192.168.0.0/24'))]))

	NATPolicy("inter-intra", 
                  cacheable=TRUE,
                  nat=GeneralNAT([(InetDomain('192.168.0.0/24'), InetDomain('5.6.7.0/24'))]))

	Service("intra_HTTP_internet", HttpProxy,
		router=TransparentRouter(forge_addr=TRUE),
		snat_policy="intra-inter")

	Service("internet_HTTP_intra", HttpProxy,
		router=TransparentRouter(forge_addr=TRUE),
		dnat_policy="inter-intra")

The cacheable attribute specifies that the an IP address always maps to the
same IP address after NAT. By enabling caching this decision is made only
once for every IP address.

The nat argument of the NATPolicy class specifies a NAT mapping class to
use, in our example we used GeneralNAT which is a general NAT implementing
N:M mapping between addresses.