Thoughts on Spanned Layer 2 Networks

Introduction

A layer 2 network is by design intended to exist in a local area. A stretched layer 2 network is not considered networking best practice, but there is traction in the industry to provide for it, and make it a work.

There are various solutions out there, for example:

– Citrix NetScaler CloudBridge
– Cisco Pseudowire-Class (Layer 2 Tunnelling Protocol)
– F5 Big-IP
– Or even a straight-forward GRE tunnel

Most of the technologies above have been around for a while.

Considerations

The most important considerations in order to have a functional stretched layer 2 network are:

– Hardware
– Broadcast Traffic and Bandwidth
– Distance and number of hops
– MTU

Hardware:

Not all hardware will support running stretched VLANs, an example is that Cisco Metro switches are recommended if you are considering Cisco Pseudowire-class.

Broadcast Traffic and Bandwidth:

In a local layer 2 network, the broadcast traffic is typically contained across not many switches within a small area. Typically, this broadcast traffic will flow across Gigabit or better networks. When building a stretched layer 2 network, consideration has got to be given regards how much bandwidth will all the broadcast traffic take. A poorly designed stretched layer 2 network could soon find its WAN links saturated by broadcast traffic. Also, the cost of providing enough bandwidth to accommodate all the inter-VLAN and broadcast traffic needs to be carefully measured.

Usual networking best practices apply regards keeping the number of devices on the layer 2 networks down to a reasonable level, so as to reduce the amount of broadcast traffic, and where possible implement VLANs to break up broadcast domains.

Distance and number of hops:

The desired response times from devices in either side of the layer 2 network, will dictate how far it is possible to physically separate the local segments of the stretched layer 2 network.

Example using division by the speed of light:
1000km introduces a minimum round trip (round trip being 2000km) latency of 6.7ms

Every device (hop) in the path from side A to side B, will add an element of jitter, and additional latency.

MTU:

The minimum MTU of devices across the layer 2 network is very important. Every device should be set to the same MTU. If a device sends packets with a larger MTU than another device, this introduces fragmentation of packets, and more delay to their successful delivery.

It is possible to test maximum MTU using the DOS prompt, for example:

C:\>ping www.test.com -f -l 1473

Pinging www.test.com with 1473 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

C:\>ping www.test.com -f -l 1472

Pinging www.test.com [174.36.85.72] with 1472 bytes of data:
Reply from 174.36.85.72: bytes=1472 time=153ms TTL=53
Reply from 174.36.85.72: bytes=1472 time=152ms TTL=53
Reply from 174.36.85.72: bytes=1472 time=152ms TTL=53
Reply from 174.36.85.72: bytes=1472 time=153ms TTL=53

In the above example, we see that setting payload size to 1473 causes the sent packet to be fragmented and not delivered. Setting the payload to 1472 meets with success!

Remember that the MTU = max packet size from ping test + 28
(The 28 is for IP and ICMP headers)

A nice little MTU test tool is the free to download mtutest.exe and an example of its output is below:

Final Comment

The industry drive for stretched layer 2 networks comes mostly from the cloud and the desire for on premise networks to be stretched into the cloud for possible migration strategies or to elastically take advantage of provisioning additional resource. Another driver is for moving servers with legacy applications on, and which may not take kindly to having their or other device IP addresses changed.

Personally, I would avoid wherever possible!

Comments