Configure threads for peer communication
A DataSource application uses threads to process the communication with its DataSource peers. Here we explain how to configure the allocation of threads to peers.
You can configure how your DataSource application allocates threads to peer connections by setting the global configuration item
peer-thread-pool-size and the
thread-name option of the
add-peer configuration item. This helps to improve performance when the application is connected to several DataSource peers.
|You may need to experiment to determine the optimum configuration for the DataSource peer connection threads. You’ll need to balance the performance gains against memory usage, taking into account the number of CPUs on which those threads can run.|
thread-name option for a peer creates a named thread. You can then allocate the same named thread to other peers, so all the peers share this thread. For example:
add-peer remote-name DataSource_1 remote-label DataSource_1 ... thread-name DS_Thread_1 end-peer add-peer remote-name DataSource_2 remote-label DataSource_2 ... thread-name DS_Thread_1 end-peer
In this example, the connections to
DataSource_2 are handled on a single thread called
peer-thread-pool-size to define a pool of threads that are shared between peer connections that don’t have a defined
thread-name. For example:
peer-thread-pool-size 2 ... add-peer remote-name DataSource_3 remote-label DataSource_3 ... # No thread-name defined end-peer add-peer remote-name DataSource_4 remote-label DataSource_4 ... # No thread-name defined end-peer add-peer remote-name DataSource_5 remote-label DataSource_5 ... # No thread-name defined end-peer
In this example (continued from the previous one), a pool of two threads is allocated to three of the DataSource peer connections (
DataSource_5), so two of the connections must share a thread.
If you need to ensure that each peer connection that isn’t explicitly allocated a named thread nevertheless has its own unique thread (that is, not shared with any other connection), omit
peer-thread-pool-size from the configuration.
If neither of the
thread-name configuration items is defined, the DataSource application creates a dedicated thread for each configured peer.
|This is the same behaviour as in older versions of the DataSource libraries that don’t have these configuration items.|
Consider allocating pool threads to DataSource peer connections with low update rates (typically 100 updates/sec or lower). In this case, there’s no real performance advantage from dedicating a separate thread to each connection.
If you expect a particular peer connection to be heavily loaded (typically when it must handle 10,000 or more updates/sec), we recommend that you allocate a dedicated thread to that connection. For example:
peer-thread-pool-size 30 ... add-peer remote-name DataSource_1 remote-label DataSource_1 ... # No thread-name defined end-peer ... add-peer remote-name DataSource_90 remote-label DataSource_90 ... # No thread-name defined end-peer add-peer remote-name DataSource_91 remote-label DataSource_91 ... # This peer connection is heavily loaded # so we give it its own thread: thread-name DS_91_Thread end-peer
If your DataSource application has just one DataSource peer to which you send, or which sends you, a high rate of updates, you may be able to improve performance by configuring more than one connection to the peer, and ensuring each connection uses a separate thread. The updates are then spread across multiple threads.