Introduction to Fiber Channel Switching

Amr Ibrahim Enan is a Global Knowledge instructor who teaches and blogs from Global Knowledge Egypt.

With the introduction of I/O consolidation that I discussed in a previous post, network engineers found themselves responsible for one physical switch that provides both LAN and SAN access for servers in data centers since SAN traffic is now carried over Ethernet using FCOE technology.

Even if you do not have to configure the SAN features of your I/O consolidation switch, you should at least have a basic understanding of FC technologies. In this post, I will provide you with all you need to know about FC technologies as a network engineer. While doing so, I will try to relate — whenever possible — the FC feature to a network feature you already understand, which will make it much easier and fun to understand and memorize the relative new FC feature.

Before we discuss anything, let me begin with a quick comparison between the TCP/IP stack and the FC stack.

Before we dig deeply in this comparison, I just want to bring to your attention that for the TCP/IP stack, we have four layers. Each layer has a programming API or physical interfaces, but for FC it is called Levels. Theoretically speaking there is no difference, but from the implementation perspective Levels does mandate programming API or Physical interfaces between levels where levels implementation is vendor specific. So how this will makes FC different from Ethernet? In Ethernet, if you connected two Ethernet switches in which one is Cisco and one is Juniper, most probably it will work just fine as each switch functionality is strictly defined and implemented according to its layer definition. For FC switches that is not the case if you tried to connect a Cisco Switch to a Brocade switch for example. Because of this specific implementation, you have to configure both switches in what is called compatibility mode since each vendor has a specific FC level implementation.

Now let us dip deeper in the layers. As you can see in Figure 1, we have the TCP/IP stack with its four layers, and right next to it is the FC levels .

Application layer/FC-level 4 and FC-level 3

The application layer is where we run all of our application layer protocols like DNS SMTP,HTTP, etc. As you may already know, whenever any application layer protocol needs to put some data on the network, it relies on the services of the next layer which, in our case, is the Transport layer. This is exactly the same role of FC level 4. In this layer, we map the ULP protocols to the FC environment.

You can see from the above figure that inside FC-level 4 each ULP has it is own mapping. FC level 3 is a placeholder for future features that can be applied to all ULP protocols before the packets is passed to FC level 2

Transport layer /FC level 2

In the transport layer we have two main protocols: the TCP protocol and the UPD protocol. Both are available for the use of application layer protocols, but each provides a different set of services to the application layer. TCP is a connection oriented protocol, which means it needs to negotiate a specific set of parameters before exchanging any data between the two peers on the network. Thanks to this negotiation, TCP can recover lost frames while sending data and in times of congestion, provide the connection with flow control service. UDP is just a connectionless protocol that does not provide you with any of those services. FC level 2 provides FC level 4 with the same set of services that are provided by TCP to the Application layer while TCP is implemented inside the kernel of the OS. This places a lot of processing load on the CPU to drive a TCP connection. For example, to drive a 100 mb/s link TCP would consume 80% of your CPU cycles. FC is implemented in Hardware using smart Asics on the Storage adapter, meaning that FC level 2 has its own CPU instead of relying on the system CPU This is why storage adapters named HBA (Host bus adapters ) are more expensive than normal network adapters and require virtually no system CPU cycles to drive the FC link.

Again, the transport layer will rely on the Internetwork layer to put this data on the network. In the Internetwork layer, we have only one protocol called the IP protocol. The main function of the IP protocol is to provide a unique identity for your machine on the network so it can send and receive data from other hosts connected to the Internet. This address is used by routers to identify the path to this host through routes exchanged by the routers using any configuration routing protocol. This layer has no equal level in FC.

Data link /FC level 1

Finally, the Internetwork layer relies on the Data link layer too, but this piece of data on the network is actually two layers. The Data link layer is where we have the Ethernet protocol encoding that decodes the data as it moves to and from the host. This function is carried by FC level 1. For FC we use 10b/8b decoding scheme. Because of this decoding scheme, you will end up losing 20% of your BW. This means that if you use 1Gb/s, your effective BW is 8oo MB/s. How do we calculate it? It is very easy. Just multiply 1 G/s* 8/10, and the result is 0.8 G/s, which is the same as 800 MB/ s.

Physical layer / FC level 0

Both layers are responsible for serializing the data to the cable and deserializing the data off the cable. Take notice that FC can provide you with an error free line if the latency is no higher than 10^-13 .

In the next post we will the difference between Ethernet switching and FC switching concept


In this article

Join the Conversation