The TCP/IP protocol

The TCP/IP protocol sometimes shows its age.

It was invented in an era when networks were very slow and packet loss was high. So one of the main considerations in early protocol design was reliability.

The Transmission Control Protocol (TCP) has built-in mechanisms for reliability that include validating a checksum on every packet, as well as detection and retransmission of dropped or out-of-order packets.

These features were invented when WAN bandwidth of 56Kbps was fast and packet drop rates of 1% were not uncommon.

Today’s broadband networks are many orders of magnitude faster, as well as vastly more reliable. So it shouldn’t come as a surprise that the bulletproof reliability mechanisms designed into the protocol sometimes cause problems. One of them has to do with a feature called TCP windowing.

What’s a TCP window?

A key reliability feature of TCP is the acknowledgement (ACK) packet. Device A sends a bunch of data to device B. Then device B validates all the packet-level checksums and sends an ACK packet that says everything was received correctly.

If everything isn’t received, some or all of the data needs to be retransmitted. Each device maintains a buffer of all the data just in case it needs to send it again. Receiving the ACK packet means the device can flush that old data out of the buffer.

The TCP window is the maximum number of bytes that can be sent before the ACK must be received.

If the network is unreliable, it’s better to keep the window small. This way you don’t have to retransmit as much data if there’s a problem. But if the network is reliable, then the window can be quite large.

One of the most clever features of TCP is the concept of a sliding window. Devices can change the window size dynamically, making it smaller when there’s congestion and bigger when things are clear.

Each device sends the other a suggested window size that says how much data it wants to receive before getting an acknowledgement. You can even have different window sizes for the two traffic directions in the same conversation.

Where TCP windowing goes wrong

TCP windowing is a very clever mechanism. But consider what happens on a network with very high latency and high bandwidth.

You can easily transmit an entire window’s worth of data before the first packet is even received at the other end. Then the sender stops and waits for the acknowledgement. Eventually, the receiver receives the last packet in the burst and sends an acknowledgement—a single packet that has to cross the network—taking the same amount of time again.

Each device can only send packets in these relatively short bursts. Then it must wait for the acknowledgement from the other end.

The average amount of data getting through the network is a small fraction of the total bandwidth. In modern networks, drop rates are so low, this slow transmission rate isn’t justified. All it does is drag down network performance

How to fix TCP windowing

The TCP window size is controlled by the end devices, not by the routers, switches, or firewalls that happen to be in the middle. The devices actively and dynamically negotiate the window size throughout the session.

But as I mentioned earlier, the TCP mechanism was designed for network bandwidth that’s orders of magnitude slower than what we have today. So some implementations still enforce a maximum window size of 64KB. You can get around this by enabling TCP windows scaling, which allows windows of up to 1GB.

Windows scaling was introduced in RFC 1323 to solve the problem of TCP windowing on fast, reliable networks. It’s available as an option in any modern TCP implementation. The only question is whether it’s been enabled properly.

In all recent Microsoft Windows implementations, windows scaling is enabled by default. You‘ll find places on the Internet telling you to change registry values to increase your window size, but depending on the Windows version you’re using, these changes will have no effect. The values may no longer even exist. Bottom line, you don’t need to fix TCP windowing in Windows, either on clients or servers.

On Linux systems, you can check that full TCP window scaling is enabled by looking at the value in /proc/sys/net/ipv4/tcp_window_scaling.

On Cisco devices, you can adjust the window size using the global configuration command, “ip tcp window-size”. This command only affects sessions to the Cisco device itself. Network devices generally won’t change the parameters for sessions that merely pass through them.

Your Guide to Selling Managed Network Services

Get templates for network assessment reports, presentations, pricing & more—designed just for MSPs.

Ebook cover - The Ultimate Guide to Selling Managed Network Services
  1. Sooraj Avatar
    Sooraj

    Very good article. Well articulated

  2. Brian Berry Avatar
    Brian Berry

    Great article and as Sooraj has said, nicely articulated.
    One question I have is related to the Windows platforms that have adopted Windows Scaling as per RFC1323? Looking at the date of this RFC (1992) I’m assuming that Microsoft have introduced this on all Post-Windows XP platforms – I am using Windows 7 on a test network.

    1. Kevin Dooley Avatar
      Kevin Dooley

      Hi Brian – thanks for your question. RFC 1323 has been available on Windows Server platforms since Server 2000 and on workstations since Windows XP. It can be enabled and disabled through the Registry “Tcp1323Opts” parameter. So if you’re unsure if it’s on and working, that’s probably the best starting point.

  3. Vignesh Avatar
    Vignesh

    This is how anyone should explain basic Windowing and Window scaling to a network tech at any level!
    Thanks..

  4. paul Avatar
    paul

    nice clean explanation thanks Paul uk

  5. Ruben Pasillas Avatar
    Ruben Pasillas

    Excellent summary on TCP window scaling

  6. maria Avatar
    maria

    Very useful.. straight to the point…Thanks

  7. radha Avatar
    radha

    nice one just the explanation i need about window size in TCP.

    thanks

  8. Fikret Karim Avatar
    Fikret Karim

    Still we don’t know how to set the window size to a fixed value. We can disable sliding then what next?

  9. Isaac Munoz Avatar
    Isaac Munoz

    What about Android? Where/how can window size or window scaling be changed? Thanks.

    1. Kevin Dooley Avatar
      Kevin Dooley

      Hi Isaac,

      Android is a Linux variant under the covers. As such, it will have a default initial TCP window size, but should then automatically scale properly to the maximum allowed by the protocol if the throughput and latency are good. I don’t think there is any user accessible way of changing the behaviour.

      – Kevin

      1. Thalys Avatar
        Thalys

        Does Linux have TCP window size auto adjust?
        I know that windows have it since Windows XP. But what about Linux, does it come originally on Linux bases systems?

  10. James Bradley Avatar
    James Bradley

    Unfortunately this article only says windows scaling is enabled by default. How do we confirm that? On wireshark captures that I’m looking at windows scaling variable =1 so I’m assuming it’s not enabled.

    Can you explain the steps to enable on windows 10 as the article indicated by the title

  11. Ajay Avatar
    Ajay

    Thanks for explaining TCP window in simplified way. But I need to know, is it possible to scale TCP window on ESXI operating system ? I have two host in different datacenter (both are Gen 6) hardware but when I want to do clone of any VM, I get very low speed (4 MBPS) but we have WAN 10GBps
    Network team has remove the bandwidth cap but cant see any improvement.

    So I just want to know if I can scale TCP windows in esxi to get maximum speed ? Would be great, if you can guide

    1. Ryan LaFlamme Avatar

      We’ve checked in, and unfortunately ESXI is outside our expert’s area of knowledge. Good luck on your search!

  12. amer Avatar
    amer

    The send window size of TCP depends on the speed of the network ?

    1. Ryan LaFlamme Avatar

      For faster networks, you can consider “scaling” the window size. You might find this helpful! https://docs.microsoft.com/en-us/troubleshoot/windows-server/networking/description-tcp-features

      1. Tassadar Avatar
        Tassadar

        I’ve a SFTP server on a Windows 10 machine and I get 48 megabytes per second on a gigabit lan but when I move one of the computers to another place in the same city and connect them by two 600/600 or faster fiber connections speed reach only 8,5 megabytes/second.

        This is clearly due to latency since banchwith is enought to 40 megabytes/s and tcp window size is 64k.

        I want to “scale” the window size as explained on your link but I’m not able to. I’ve create “Tcp1323Opts” key in registry and it appears as enabled when I run “netsh interface tcp show global” on powershell.

        I think I should define the scale factor but I don’t know how to.

        Regards

Leave a Reply

Your email address will not be published. Required fields are marked *