TCP KeepAlive: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(33 intermediate revisions by the same user not shown)
Line 2: Line 2:


* http://zhefeng.wordpress.com/2009/06/15/oracle-connection-idle-timeout-with-firewall/
* http://zhefeng.wordpress.com/2009/06/15/oracle-connection-idle-timeout-with-firewall/
* http://www.tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/
* https://www.gnugk.org/keepalive.html
* http://coryklein.com/tcp/2015/11/25/custom-configuration-of-tcp-socket-keep-alive-timeouts.html


=Internal=
=Internal=
Line 7: Line 10:
* [[Keep-Alive]]
* [[Keep-Alive]]
* [[TCP#Keep-Alive|TCP]]
* [[TCP#Keep-Alive|TCP]]
* [[Kernel_Runtime_Configuration#net.ipv4.tcp_keepalive_time.2C_net.ipv4.tcp_keepalive_probes.2C_net.ipv4.tcp_keepalive_intvl|Linux Kernel Runtime Configuration]]


=Overview=
=Overview=


TCP Keep-Alive is a mechanism that insures small probe packets are periodically sent to the other end of the TCP connection, with the purpose of detecting whether the peer host crashed. Under normal circumstances, if the TCP Keep-Alive is enabled and if the TCP stacks at both ends are up, the keep-alive probe exchange will continue indefinitely if the application layers at both ends stay idle, not sending data - TCP keep-alive does not need application traffic to latch onto. For a discussion on how long a TCP stays up in presence or absence of application traffic, see [[TCP#Duration_of_a_TCP_Connection|Duration of a TCP Connection]]. Note that a TCP connection can stay alive indefinitely even in absence of keep-alive packets, if there are no network disruptions.. Note that the keep-alive mechanism has to be explicitly enabled, whether it is enabled by default or not is implementation dependent. The keep-alive probe logic is implemented in the TCP stack: if the keep-alive option is enabled, and no application data has been exchanged across the socket in either direction for [[#Keep-Alive_Time|keep-alive time]], by default 7,200 seconds, TCP automatically sends a keep-alive probe to the peer. This probe is a TCP segment which contains null data. In an Ethernet network, a keepalive frame length is 60 bytes, while the server response to this, also a null data frame, is 54 bytes. The peer must respond to it. One of three responses is expected:  
TCP Keep-Alive is a mechanism that insures small probe packets are periodically sent to the other end of the TCP connection, with the purpose of detecting whether the peer host crashed. Under normal circumstances, if the TCP Keep-Alive is enabled and if the TCP stacks at both ends are up, the keep-alive probe exchange will continue indefinitely if the application layers at both ends stay idle, not sending data - TCP keep-alive does not need application traffic to latch onto. For a discussion on how long a TCP stays up in presence or absence of application traffic, see [[TCP#Duration_of_a_TCP_Connection|Duration of a TCP Connection]]. Note that the keep-alive mechanism has to be explicitly enabled, whether it is enabled by default or not is implementation dependent. The keep-alive probe logic is implemented in the TCP stack: if the keep-alive option is enabled, and no application data has been exchanged across the socket in either direction for [[#Keep-Alive_Time|keep-alive time]], by default 7,200 seconds, TCP automatically sends a keep-alive probe to the peer. This probe is a TCP segment which contains null data. In an Ethernet network, a keepalive frame length is 60 bytes, while the server response to this, also a null data frame, is 54 bytes. If a peer receives a keep-alive probe, then it must respond to it. One of three responses is expected:  
# The peer responds with the expected ACK. The application is not notified, since everything is OK. TCP will send another probe following another [[#Keep-Alive_Time|keep-alive time]] seconds of inactivity.  
# The peer responds with the expected ACK. The application is not notified, since everything is OK. TCP will send another probe following another [[#Keep-Alive_Time|keep-alive time]] seconds of inactivity.  
# The peer responds with an RST, which tells the local TCP that the peer host has crashed and rebooted. The socket is closed.  
# The peer responds with an RST, which tells the local TCP that the peer host has crashed and rebooted. As result, the socket will be closed.  
# There is no response from the peer. The OS will close the socket and will release the associated resources. The application listening on that particular socket will receive an error from the OS.
# There is no response from the peer. The OS will close the socket and will release the associated resources. The application listening on that particular socket will receive an error from the OS.


Aside from application notification in case of connection failure, another benefit of enabling TCP Keep-Alive is that it keeps the connection "active" so if the connection goes over a firewall that watches for inactivity, that will prevent the firewall from dropping the connection.
Aside from application notification in case of connection failure, another benefit of enabling TCP Keep-Alive is that it keeps the connection "active" so if the connection goes over a firewall that watches for inactivity, that will prevent the firewall from dropping the connection.
According to RFC 1122 4.2.3.6, responding to and/or relaying TCP Keep-Alive packets is optional:
::<font color=teal>Implementors MAY include “keep-alives” in their TCP implementations, although this practice is not universally accepted. If keep-alives are included, the application MUST be able to turn them on or off for each TCP connection, and they MUST default to off. [...] It is extremely important to remember that ACK segments that contain no data are not reliably transmitted by TCP.</font>
If a Java application's socket detects a failed keep-alive exchange, it throws:
<syntaxhighlight lang='java'>
java.io.IOException: Operation timed out
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
        ...
</syntaxhighlight>
or
<syntaxhighlight lang='java'>
java.io.IOException: Connection timed out
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
        ...
</syntaxhighlight>
=Keep-Alive and Blocking Read=
If the application executes a blocking read, keep-alive is not turned on the socket, and the other side fails for any reason without sending a RST, the tread that is performing the blocking read can block potentially forever.


=Configuration=
=Configuration=


Java networking API will report whether a certain socket has its keep-alive mechanism turned on or off, and allows configuring it programmatically. See [[Socket_SO_KEEPALIVE#Overview|Socket SO KEEPALIVE]].
Keep-alive can be turned on or off on an individual socket basis. Moreover, either side of the connection can have its keep-alive status configured independently. Java networking API will report whether a certain socket has its keep-alive mechanism turned on or off, and allows configuring it programmatically. For details, see [[Socket_SO_KEEPALIVE#Overview|Socket SO KEEPALIVE]].
 
However, the numeric parameters dictating how TCP Keep-Alive behaves can only be changed at the system level, in the kernel:


<font color=darkgray>Both sides of the connection can apparently have their keep-alive options configured independently.</font>
==<span id='Keepalive_time'></span>Keep-Alive Time==


There are three parameters related to keepalive:
The time of ''connection inactivity'' after which the first keep-alive request is sent. In other words, is the duration between two keepalive transmissions in idle condition. The default value on Linux is 2 hours (7,200 seconds). On Linux, it is available as "net.ipv4.tcp_keepalive_time" kernel configuration parameter.


==<span id='Keepalive_time'></span>Keep-Alive Time==
==<span id='Keepalive_interval'></span>Keep-Alive Interval==


The time of ''connection inactivity'' after which the first keep-alive request is sent. In other words, is the duration between two keepalive transmissions in idle condition. The default value on Linux is 2 hours (7,200 seconds). More details [[TCP KeepAlive on Linux]].
The duration between two successive keepalive retransmissions, if acknowledgement to the previous keepalive transmission is not received. On Linux, it is available as "net.ipv4.tcp_keepalive_intvl" kernel configuration parameter.


==Keepalive interval==
==<span id='Keepalive_retry'></span>Kee-Alive Retry==


The duration between two successive keepalive retransmissions, if acknowledgement to the previous keepalive transmission is not received.
The number of retransmissions to be carried out before declaring that remote end is not available.  On Linux, it is available as "net.ipv4.tcp_keepalive_probes" kernel configuration parameter.


==Keepalive retry==
==Keep-Alive Init==


The number of retransmissions to be carried out before declaring that remote end is not available.
Macs seem to have an extra parameter "net.inet.tcp.keepinit", which is "a per-socket timeout argument  for new, non-established TCP connections.


=O/S Specific Details=
=O/S Specific Details=


The fact that TCP KeepAlive is enabled or not, and how it is configured, it is OS-dependent
The fact that TCP KeepAlive is enabled or not, and how it is configured, it is OS-dependent. This is how the configuration values can be obtained and changed on various systems:
 
==Linux==
 
To get the values:
 
sysctl net.ipv4.tcp_keepalive_time
sysctl net.ipv4.tcp_keepalive_probes
sysctl net.ipv4.tcp_keepalive_intvl
 
or read them from /proc/sys/net/ipv4/tcp_keepalive_time,  /proc/sys/net/ipv4/tcp_keepalive_probes and /proc/sys/net/ipv4/tcp_keepalive_intvl.
 
The values for tcp_keepalive_time and tcp_keepalive_intvl are expressed in seconds.
 
To change them to survive reboot:
 
sysctl -w net.ipv4.tcp_keepalive_time=180 net.ipv4.tcp_keepalive_probes=3 net.ipv4.tcp_keepalive_intvl=10
 
For more details see [[Kernel Runtime Configuration#Overview|Kernel Runtime Configuration]].


==TCP Keepalive on Linux==
==Mac==


<font color=darkgray>
To get the values:
* The TCP KeepAlive Source of Record http://www.tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/
* Using TCP Keepalive to Detect Network Errors http://www.gnugk.org/keepalive.html
</font>


=Questions and TODO=
sysctl net.inet.tcp | grep -E "keepidle|keepintvl|keepcnt"
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.keepcnt: 8


<font color=red>
Note that values of keepidle and keepintvl are specified in milliseconds, as opposed to Linux that uses seconds.


* Can keepalive be set per TCP connection, or is a system-wide setting (all TCP/IP connections)?
To set the values to survive reboot:
* So it is true that if I don't have keep alive, my write can block forever if I power off the other end suddenly.


sysctl -w net.inet.tcp.keepidle=180000 net.inet.tcp.keepcnt=3 net.inet.tcp.keepintvl=10000


</font>
For more methods to set Mac kernel parameters, see [[Mac_Kernel_Parameter_Configuration|Mac Kernel Parameter Configuration]].

Latest revision as of 16:07, 27 July 2018

External

Internal

Overview

TCP Keep-Alive is a mechanism that insures small probe packets are periodically sent to the other end of the TCP connection, with the purpose of detecting whether the peer host crashed. Under normal circumstances, if the TCP Keep-Alive is enabled and if the TCP stacks at both ends are up, the keep-alive probe exchange will continue indefinitely if the application layers at both ends stay idle, not sending data - TCP keep-alive does not need application traffic to latch onto. For a discussion on how long a TCP stays up in presence or absence of application traffic, see Duration of a TCP Connection. Note that the keep-alive mechanism has to be explicitly enabled, whether it is enabled by default or not is implementation dependent. The keep-alive probe logic is implemented in the TCP stack: if the keep-alive option is enabled, and no application data has been exchanged across the socket in either direction for keep-alive time, by default 7,200 seconds, TCP automatically sends a keep-alive probe to the peer. This probe is a TCP segment which contains null data. In an Ethernet network, a keepalive frame length is 60 bytes, while the server response to this, also a null data frame, is 54 bytes. If a peer receives a keep-alive probe, then it must respond to it. One of three responses is expected:

  1. The peer responds with the expected ACK. The application is not notified, since everything is OK. TCP will send another probe following another keep-alive time seconds of inactivity.
  2. The peer responds with an RST, which tells the local TCP that the peer host has crashed and rebooted. As result, the socket will be closed.
  3. There is no response from the peer. The OS will close the socket and will release the associated resources. The application listening on that particular socket will receive an error from the OS.

Aside from application notification in case of connection failure, another benefit of enabling TCP Keep-Alive is that it keeps the connection "active" so if the connection goes over a firewall that watches for inactivity, that will prevent the firewall from dropping the connection.

According to RFC 1122 4.2.3.6, responding to and/or relaying TCP Keep-Alive packets is optional:

Implementors MAY include “keep-alives” in their TCP implementations, although this practice is not universally accepted. If keep-alives are included, the application MUST be able to turn them on or off for each TCP connection, and they MUST default to off. [...] It is extremely important to remember that ACK segments that contain no data are not reliably transmitted by TCP.

If a Java application's socket detects a failed keep-alive exchange, it throws:

java.io.IOException: Operation timed out
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
        ...

or

java.io.IOException: Connection timed out
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
        ...

Keep-Alive and Blocking Read

If the application executes a blocking read, keep-alive is not turned on the socket, and the other side fails for any reason without sending a RST, the tread that is performing the blocking read can block potentially forever.

Configuration

Keep-alive can be turned on or off on an individual socket basis. Moreover, either side of the connection can have its keep-alive status configured independently. Java networking API will report whether a certain socket has its keep-alive mechanism turned on or off, and allows configuring it programmatically. For details, see Socket SO KEEPALIVE.

However, the numeric parameters dictating how TCP Keep-Alive behaves can only be changed at the system level, in the kernel:

Keep-Alive Time

The time of connection inactivity after which the first keep-alive request is sent. In other words, is the duration between two keepalive transmissions in idle condition. The default value on Linux is 2 hours (7,200 seconds). On Linux, it is available as "net.ipv4.tcp_keepalive_time" kernel configuration parameter.

Keep-Alive Interval

The duration between two successive keepalive retransmissions, if acknowledgement to the previous keepalive transmission is not received. On Linux, it is available as "net.ipv4.tcp_keepalive_intvl" kernel configuration parameter.

Kee-Alive Retry

The number of retransmissions to be carried out before declaring that remote end is not available. On Linux, it is available as "net.ipv4.tcp_keepalive_probes" kernel configuration parameter.

Keep-Alive Init

Macs seem to have an extra parameter "net.inet.tcp.keepinit", which is "a per-socket timeout argument for new, non-established TCP connections.

O/S Specific Details

The fact that TCP KeepAlive is enabled or not, and how it is configured, it is OS-dependent. This is how the configuration values can be obtained and changed on various systems:

Linux

To get the values:

sysctl net.ipv4.tcp_keepalive_time
sysctl net.ipv4.tcp_keepalive_probes
sysctl net.ipv4.tcp_keepalive_intvl

or read them from /proc/sys/net/ipv4/tcp_keepalive_time, /proc/sys/net/ipv4/tcp_keepalive_probes and /proc/sys/net/ipv4/tcp_keepalive_intvl.

The values for tcp_keepalive_time and tcp_keepalive_intvl are expressed in seconds.

To change them to survive reboot:

sysctl -w net.ipv4.tcp_keepalive_time=180 net.ipv4.tcp_keepalive_probes=3 net.ipv4.tcp_keepalive_intvl=10

For more details see Kernel Runtime Configuration.

Mac

To get the values:

sysctl net.inet.tcp | grep -E "keepidle|keepintvl|keepcnt"
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.keepcnt: 8

Note that values of keepidle and keepintvl are specified in milliseconds, as opposed to Linux that uses seconds.

To set the values to survive reboot:

sysctl -w net.inet.tcp.keepidle=180000 net.inet.tcp.keepcnt=3 net.inet.tcp.keepintvl=10000

For more methods to set Mac kernel parameters, see Mac Kernel Parameter Configuration.