Previous Up Next

Chapter 8  Connection and Thread Management

This chapter describes how omniORB manages threads and network connections.

8.1  Background

In CORBA, the ORB is the `middleware' that allows a client to invoke an operation on an object without regard to its implementation or location. In order to invoke an operation on an object, a client needs to `bind' to the object by acquiring its object reference. Such a reference may be obtained as the result of an operation on another object (such as a naming service or factory object) or by conversion from a stringified representation. If the object is in a different address space, the binding process involves the ORB building a proxy object in the client's address space. The ORB arranges for invocations on the proxy object to be transparently mapped to equivalent invocations on the implementation object.

For the sake of interoperability, CORBA mandates that all ORBs should support IIOP as the means to communicate remote invocations over a TCP/IP connection. IIOP is usually1 asymmetric with respect to the roles of the parties at the two ends of a connection. At one end is the client which can only initiate remote invocations. At the other end is the server which can only receive remote invocations.

Notice that in CORBA, as in most distributed systems, remote bindings are established implicitly without application intervention. This provides the illusion that all objects are local, a property known as `location transparency'. CORBA does not specify when such bindings should be established or how they should be multiplexed over the underlying network connections. Instead, ORBs are free to implement implicit binding by a variety of means.

The rest of this chapter describes how omniORB manages network connections and the programming interface to fine tune the management policy.

8.2  The model

omniORB is designed from the ground up to be fully multi-threaded. The objective is to maximise the degree of concurrency and at the same time eliminate any unnecessary thread overhead. Another objective is to minimise the interference by the activities of other threads on the progress of a remote invocation. In other words, thread `cross-talk' should be minimised within the ORB. To achieve these objectives, the degree of multiplexing at every level is kept to a minimum by default.

Minimising multiplexing works well when the ORB is relatively lightly loaded. However, when the ORB is under heavy load, it can sometimes be beneficial to conserve operating system resources such as threads and network connections by multiplexing at the ORB level. omniORB has various options that control its multiplexing behaviour.

8.3  Client side behaviour

On the client side of a connection, the thread that invokes on a proxy object drives the GIOP protocol directly and blocks on the connection to receive the reply. The first time the client makes a call to a particular address space, the ORB opens a suitable connection to the remote address space (based on the client transport rule as described in section 8.7.179Client transport rulessubsection.8.7.1). After the reply has been received, the ORB caches the open network connection, ready for use by another call.

If two (or more) threads in a multi-threaded client attempt to contact the same address space simultaneously, there are two different ways to proceed. The default way is to open another network connection to the server. This means that neither the client or server ORB has to perform any multiplexing on the network connections---multiplexing is performed by the operating system, which has to deal with multiplexing anyway. The second possibility is for the client to multiplex the concurrent requests on a single network connection. This conserves operating system resources (network connections), but means that both the client and server have to deal with multiplexing issues themselves.

In the default one call per connection mode, there is a limit to the number of concurrent connections that are opened, set with the maxGIOPConnectionPerServer parameter. To tell the ORB to multiplex calls on a single connection, set the oneCallPerConnection parameter to zero.

Note that some server-side ORBs, including omniORB versions before version 4.0, are unable to deal with concurrent calls multiplexed on a single connection, so they serialise the calls. It is usually best to keep to the default mode of opening multiple connections.

8.3.1  Client side timeouts

omniORB can associate a timeout with a call, meaning that if the call takes too long a TRANSIENT exception is thrown. Timeouts can be set for the whole process, for a specific thread, or for a specific object reference.

Timeouts are set using this API:
namespace omniORB {
  void setClientCallTimeout(CORBA::ULong millisecs);
  void setClientCallTimeout(CORBA::Object_ptr obj, CORBA::ULong millisecs);
  void setClientThreadCallTimeout(CORBA::ULong millisecs);
setClientCallTimeout() sets either the global timeout or the timeout for a specific object reference. setClientThreadCallTimeout() sets the timeout for the calling thread. The calling thread must have an omni_thread associated with it. Setting any timeout value to zero disables it.

Accessing per-thread state is a relatively expensive operation, so per thread timeouts are disabled by default. The supportPerThreadTimeOut parameter must be set true to enable them.

To choose the timeout value to use for a call, the ORB first looks to see if there is a timeout for the object reference, then to the calling thread, and finally to the global timeout.

8.4  Server side behaviour

The server side has two primary modes of operation: thread per connection and thread pooling. It is able to dynamically transition between the two modes, and it supports a hybrid scheme that behaves mostly like thread pooling, but has the same fast turn-around for sequences of calls as thread per connection.

8.4.1  Thread per connection mode

In thread per connection mode (the default, and the only option in omniORB versions before 4.0), each connection has a single thread dedicated to it. The thread blocks waiting for a request. When it receives one, it unmarshals the arguments, makes the up-call to the application code, marshals the reply, and goes back to watching the connection. There is thus no thread switching along the call chain, meaning the call is very efficient.

As explained above, a client can choose to multiplex multiple concurrent calls on a single connection, so once the server has received the request, and just before it makes the call into application code, it marks the connection as `selectable', meaning that another thread should watch it to see if any other requests arrive. If they do, extra threads are dispatched to handle the concurrent calls. GIOP 1.2 actually allows the argument data for multiple calls to be interleaved on a connection, so the unmarshalling code has to handle that too. As soon as any multiplexing occurs on the connection, the aim of removing thread switching cannot be met, and there is inevitable inefficiency due to thread switching.

The maxServerThreadPerConnection parameter can be set to limit the number of threads that can be allocated to a single connection containing concurrent calls. Setting the parameter to 1 mimics the behaviour of omniORB versions before 4.0, that did not support multiplexed calls.

8.4.2  Thread pool mode

In thread pool mode, selected by setting the threadPerConnectionPolicy parameter to zero, a single thread watches all incoming connections. When a call arrives on one of them, a thread is chosen from a pool of threads, and set to work unmarshalling the arguments and performing the up-call. There is therefore at least one thread switch for each call.

The thread pool is not pre-initialised. Instead, threads are started on demand, and idle threads are stopped after a period of inactivity. The maximum number of threads that can be started in the pool is selected with the maxServerThreadPoolSize parameter. The default is 100.

A common pattern in CORBA applications is for a client to make several calls to a single object in quick succession. To handle this situation most efficiently, the default behaviour is to not return a thread to the pool immediately after a call is finished. Instead, it is set to watch the connection it has just served for a short while, mimicking the behaviour in thread per connection mode. If a new call comes in during the watching period, the call is dispatched without any thread switching, just as in thread per connection mode. Of course, if the server is supporting a very large number of connections (more than the size of the thread pool), this policy can delay a call coming from another connection. If the threadPoolWatchConnection parameter is set to zero, connection watching is disabled and threads return to the pool immediately after finishing a single request.

8.4.3  Policy transition

If the server is dealing with a relatively small number of connections, it is most efficient to use thread per connection mode. If the number of connections becomes too large, however, operating system limits on the number of threads may cause a significant slowdown, or even prevent the acceptance of new connections altogether.

To give the most efficient response in all circumstances, omniORB allows a server to start in thread per connection mode, and transition to thread pooling if many connections arrive. This is controlled with the threadPerConnectionUpperLimit and threadPerConnectionLowerLimit parameters. The former must always be larger than the latter. The upper limit chooses the number of connections at which time the ORB transitions to thread pool mode; the lower limit selects the point at which the transition back to thread per connection is made.

For example, setting the upper limit to 50 and the lower limit to 30 would mean that the first 49 connections would receive dedicated threads. The 50th to arrive would trigger thread pooling. All future connections to arrive would make use of threads from the pool. Note that the existing dedicated threads continue to service their connections until the connections are closed. If the number of connections falls below 30, thread per connection is reactivated and new connections receive their own dedicated threads (up to the limit of 50 again). Once again, existing connections in thread pool mode stay in that mode until they are closed.

8.5  Idle connection shutdown

It is wasteful to leave a connection open when it has been left unused for a considerable time. Too many idle connections could block out new connections when it runs out of spare communication channels. For example, most Unix platforms have a limit on the number of file handles a process can open. 64 is the usual default limit. The value can be increased to a maximum of a thousand or more by changing the `ulimit' in the shell.

Every so often, a thread scans all open connections to see which are idle. The scanning period (in seconds) is set with the scanGranularity parameter. The default is 5 seconds.

Outgoing connections (initiated by clients) and incoming connections (initiated by servers) have separate idle timeouts. The timeouts are set with the outConScanPeriod and inConScanPeriod parameters respectively. The values are in seconds, and must be a multiple of the scan granularity.

8.5.1  Interoperability Considerations

The IIOP specification allows both the client and the server to shutdown a connection unilaterally. When one end is about to shutdown a connection, it should send a CloseConnection message to the other end. It should also make sure that the message will reach the other end before it proceeds to shutdown the connection.

The client should distinguish between an orderly and an abnormal connection shutdown. When a client receives a CloseConnection message before the connection is closed, the condition is an orderly shutdown. If the message is not received, the condition is an abnormal shutdown. In an abnormal shutdown, the ORB should raise a COMM_FAILURE exception whereas in an orderly shutdown, the ORB should not raise an exception and should try to re-establish a new connection transparently.

omniORB implements these semantics completely. However, it is known that some ORBs are not (yet) able to distinguish between an orderly and an abnormal shutdown. Usually this is manifested as the client in these ORBs seeing a COMM_FAILURE occasionally when connected to an omniORB server. The work-around is either to catch the exception in the application code and retry, or to turn off the idle connection shutdown inside the omniORB server.

8.6  Transports and endpoints

omniORB can support multiple network transports. All platforms (usually) have a TCP transport available. Unix platforms support a Unix domain socket transport. Platforms with the OpenSSL library available can support an SSL transport.

Servers must be configured in two ways with regard to transports: the transports and interfaces on which they listen, and the details that are published in IORs for clients to see. Usually the published details will be the same as the listening details, but there are times when it is useful to publish different information.

Details are selected with the endPoint family of parameters. The simplest is plain endPoint, which chooses a transport and interface details, and publishes the information in IORs. End point parameters are in the form of URIs, with a scheme name of `giop:', followed by the transport name. Different transports have different parameters following the transport.

TCP end points have the format:
The host must be a valid host name for the server machine. It determines the network interface on which the server listens. The port selects the TCP port to listen on, which must be unoccupied. Either the host or port, or both can be left empty. If the host is empty, the ORB published the IP address of the first non-loopback network interface it can find (or the loopback if that is the only interface), but listens on all network interfaces. If the port is empty, the operating system chooses a port.

Multiple TCP end points can be selected, either to specify multiple network interfaces on which to listen, or (less usefully) to select multiple TCP ports on which to listen.

If no endPoint parameters are set, the ORB assumes a single parameter of giop:tcp::, meaning IORs contain the address of the first non-loopback network interface, the ORB listens on all interfaces, and the OS chooses a port number.

SSL end points have the same format as TCP ones, except `tcp' is replaced with `ssl'. Unix domain socket end points have the format:
where the filename is the name of the socket within the filesystem. If the filename is left blank, the ORB chooses a name based on the process id and a timestamp.

8.6.1  End point publishing

To publish an end point in IORs, without actually listening on that end point, the endPointNoListen parameter can be set. This can be useful in fault-tolerant applications where replicas of an object can be contacted at more than one server. endPointNoListen does not check that the transport specified is sensible for the current machine, so it allows the address of a different machine to be specified.

Similarly, but less likely to be useful, it is possible to ask the server to listen on an end point, but not publish the details in IORs, using the endPointNoPublish parameter. This should not be used for security by obscurity!

If a machine has multiple TCP network interfaces, it may be useful to publish all interfaces, instead of just the first one. This is necessary if different interfaces are on separate non-gatewayed subnets, for example. Publishing all addresses could be achieved with lots of endPoint parameters, but a short-hand is to set the endPointPublishAllIFs parameter to 1. That (in conjunction with a `giop:tcp::' transport selection without a specific hostname) causes all the machine's non-loopback interfaces to be published in IORs.

8.7  Connection selection and acceptance

In the face of IORs containing details about multiple different end points, clients have to know how to choose the one to use to connect a server. Similarly, servers may wish to restrict which clients can connect to particular transports. This is achieved with transport rules.

8.7.1  Client transport rules

The clientTransportRule parameter is used to filter and prioritise the order in which transports specified in an IOR are tried. Each rule has the form:
<address mask> [action]+
The address mask can be one of
1. localhost The address of this machine
2. w.x.y.z/m1.m2.m3.m4 An IPv4 address with the bits selected by the mask, e.g.
3. * Wildcard that matches any address

The action is one or more of the following:
1. none Do not use this address
2. tcp Use a TCP transport
3. ssl Use an SSL transport
4. unix Use a Unix socket transport
5. bidir Any connection to this address should be used bidirectionally (see section 8.881Bidirectional GIOPsection.8.8)

The transport-selecting actions form a prioritised list, so an action of `unix,tcp,ssl' means to use a Unix transport if there is one, failing that a TCP transport, failing that an SSL transport. In the absence of any explicit rules, the client uses the implicit rule of `* unix,tcp,ssl'.

If more than one rule is specified, they are prioritised in the order they are specified. For example, the configuration file might contain:
  clientTransportRule =  unix,tcp
  clientTransportRule =     unix,tcp
                      =       *                    none
This would be useful if there is a fast network ( which should be used in preference to another network (, and connections to other networks are not permitted at all.

In general, the result of filtering the end point specifications in an IOR with the client transport rule will be a prioritised list of transports and networks. (If the transport rules are do not prioritise one end point over another, the order the end points are listed in the IOR is used.) When trying to contact an object, the ORB tries its possible end points in turn, until it finds one with which it can contact the object. Only after it has unsuccessfully tried all permissible transports will it raise a TRANSIENT exception to indicate that the connect failed.

8.7.2  Server transport rules

The server transport rules gave the same format as client transport rules. Rather than being used to select which of a set of ways to contact a machine, they are used to determine whether or not to accept connections from particular clients. In this example, we only allow connections from our intranet:
  serverTransportRule = localhost                  unix,tcp,ssl
                      =     tcp,ssl
                      = *                          none
And in this one, we only accept SSL connections if the client is not on the intranet:
  serverTransportRule = localhost                  unix,tcp,ssl
                      =     tcp,ssl
                      = *                          ssl,bidir

8.8  Bidirectional GIOP

omniORB supports bidirectional GIOP, which allows callbacks to be made using a connection opened by the original client, rather than the normal model where the server opens a new connection for the callback. This is important for negotiating firewalls, since they tend not to allow connections back on arbitrary ports.

There are several steps required for bidirectional GIOP to be enabled for a callback. Both the client and server must be configured correctly. On the client side, these conditions must be met: On the server side, these conditions must be met:

8.9  SSL transport

omniORB 4.0 supports an SSL transport, using OpenSSL. It is only built if OpenSSL is available. On platforms using Autoconf, it is autodetected in many locations, or its location can be given with the --with-openssl= argument to configure. On other platforms, the OPEN_SSL_ROOT make variable must be set in the platform file.

To use the SSL transport, you must link your application with the omnisslTP library, and correctly set up certificates. See the src/examples/ssl_echo directory for an example. That directory contains a README file with more details.

GIOP 1.2 supports `bidirectional GIOP', which permits the rôles to be reversed.

Previous Up Next