Configuring Linux and FreeBSD TCP performance

For both Linux and FreeBSD we’re using the sysctl utility. For all of the following options, entering the command ‘sysctl variable=number’ should do the trick. To view the current settings use: ‘sysctl <variable name>’

  • Maximum window size:
    • FreeBSD:
      kern.ipc.maxsockbuf=262144
    • Linux:
      net.core.wmem_max=8388608
  • Default window size:
    • FreeBSD, sending and receiving:
      net.inet.tcp.sendspace=65536
      net.inet.tcp.recvspace=65536
    • Linux, sending and receiving:
      net.core.wmem_default = 65536
      net.core.rmem_default = 65536
  • RFC 1323:
    This enables the useful window scaling options defined in rfc1323, which allows the windows to dynamically get larger than we specified above.
    • FreeBSD:
      net.inet.tcp.rfc1323=1
    • Linux:
      net.ipv4.tcp_window_scaling=1
  • Buffers:
    When sending large amounts of data, we can run the operating system out of buffers. This option should be enabled before attempting to use the above settings. To increase the amount of „mbufs“ available:
    • FreeBSD:
      kern.ipc.nmbclusters=32768
    • Linux:
      net.ipv4.tcp_mem= 98304 131072 196608

These quick changes will skyrocket TCP performance. Afterwards we were able to run ttcp and attain around 895 Mb/s every time – quite an impressive data rate. There are other options available for adjusting the UDP datagram sizes as well, but we’re mainly focusing on TCP here.

Defaults for FreeBSD 7.0-CURRENT are:

  • kern.ipc.maxsockbuf: 2097152
  • net.inet.tcp.sendspace: 131072
    net.inet.sctp.sendspace: 233016
  • net.inet.tcp.rfc1323: 1
  • kern.ipc.nmbclusters: 33792