Check if jumbo frames are working in LAN


To totally unlock this section you need to Log-in


Login

To begin...what are jumbo frames?

You may be wondering what are the jumbo frames, and what the hell I'm writing about these "things".

Well, jumbo frames are, in many computer mid-large networks, are used to improve data transfer performances and to let all of us to get data more quickly.

In computer networking, jumbo frames are Ethernet frames with more than 1500 bytes of payload. Conventionally, jumbo frames can carry up to 9000 bytes of payload, but variations exist and some care must be taken using the term.

Many Gigabit Ethernet switches and Gigabit Ethernet network interface cards support jumbo frames. Some Fast Ethernet switches and Fast Ethernet network interface cards also support jumbo frames, too.

Today, the vast majority of data networks use a default 1500-byte MTU size. This is because this is the default Ethernet MTU size for hosts and switches. Frequently, links between enterprise routers and the upstream ISP routers only support 1500-byte MTU.

These jumbo frames are sometimes also called "Giants". Also, the term "Baby Giants" refer to frames that are just slightly larger than the 1500 byte Ethernet MTU size (e.g. 1998 bytes).

In some situations, jumbo frames can be used to allow for much larger frame sizes if the networking hardware is capable of this configuration. Most modern routers and switches are capable of jumbo frames and much of the networking hardware within the data centers are capable of this.

How to check if Jumbo Frames are enabled

Fairly straight forward this time, you’ve configured your MTU/jumbo frames to be 9000 on your client and destination devices (say a laptop/desktop/server/san/nas) and on ALL your switching devices in between – you’ve done that right?

[tweet]

So the next step is, we want to test if our new 9000 byte MTU is actually working and we can reap the benefits of a larger packet size (whether it’s on iSCSI, LAN, whatever) being of course a higher latency but also higher throughput. This depends on the OS you are running.

Linux

On Linux using the ping command we'll have (in this case there's even the possible ping's output in the case that jumbo frames are disabled or not supported):

# ping -s 8000 -M do 172.16.64.75

PING 172.16.64.75 (172.16.64.75) 8000 (8192) bytes of data.
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)

NOTE: in this case, on Linux, the ping command use:

  • the -s switch, that specifies the number of data bytes to be sent.
  • the -M (Hint) switch, that select Path MTU Discovery strategy. Hint may be either do (prohibit fragmentation, even local one), want (do PMTU discovery, fragment locally when packet size is large), or dont (do not set DF flag).

In this case we've set a ping that will use 8000 bytes for single packet sent and a path MTU discovery strategy in "do" mode.

Windows

On Windows we'll can use a similar command, always using the ping command:

ping -f -l 9000 [destinationIP]

NOTE: in this case, on Windows, the ping command use:

  • the -l switch, that specifies the number of data bytes to be sent.
  • the -f switch, that set the "Don't fragment packet" flag in the frame.

Apple MacOSX

On Apple MacOSX we'll can use a similar command, always using the ping command:

ping -D -s 8184 [destinationIP]

NOTE: in this case, on MacOSX, the ping command use:

  • the -s switch, that specifies the number of data bytes to be sent (packet size).
  • the -D switch, that set the "Don't fragment packet" bit/flag in the frame.

The Output

If you don't have enabled jumbo frames in no nodes (no jumbo frames on switches, on clients and servers), you'll receive an output like the following (in which we can see that the system inform us that he need to fragment the packet to send it properly through the network):

ping -s 8000 -M do 172.16.64.75

PING 172.16.64.75 (172.16.64.75) 8000 (8192) bytes of data.
From 172.16.64.68 icmp_seq=1 Frag needed and DF set (mtu = 1500)

If you’ve forgotten to enable jumbo frames/9k MTU on your client device you’re sending the ping from you’ll see:

PING xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx): 8184 data bytes

ping: sendto: Message too long

If you have enabled jumbo frames on your client but not the destination (or a switch in between) you’ll see:

PING xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx): 8184 data bytes

Request timeout for icmp_seq 0

If you’ve done everything right and you’re set up ready to go you’ll get this:

PING xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx): 8184 data bytes

8192 bytes from xxx.xxx.xxx.xxx: icmp_seq=0 ttl=128 time=0.714 ms

Now rejoice in your lovely jumbo-frame goodness and a good 20-30% in sustained data throughput.