Ethernet Switching vs. Fiber Channel Switching

Now that we discussed the difference between the TCP/IP stack and the FC levels, it is time to examine the difference between Ethernet and FC switching concept But first, let me ask you what is the major difference between Ethernet switching and FC switching? Ethernet is loose, which means if a device sends a frame to an Ethernet switch and the receiving buffer of that switch is full, the switch silently drops the frame. It will not notify the sending host that it dropped the frame, so the question is how we can recover this frame? Well, actually Ethernet does not care about that. It leaves this job to the other layer in the TCP/IP stack: the Transport layer. If your application layer protocol relies on TCP to transport the data, TCP will detect the loss of the segment, and it will recover the segment by asking the sending host to resend this segment again. Also, TCP has a flow control mechanism. If the receiving host buffer becomes full then it can ask the sending host to stop sending frames for a while. This can be simply done by sending a TCP segment after setting the window size to zero. By doing so, I am asking the sending host to stop sending me any more segments until I notify it that I can receive more segments. Believe it or not, we have a similar feature in Ethernet called Flow control, but it is disabled by default on Ethernet switches. If you enabled this feature on an Ethernet switch and the receiving switch dropped a frame, it will send a pause frame to the sending switch asking it to stop sending frames for a specific amount of time. That’s cool, is it not? But no one uses this feature since TCP handles this job just great for ip traffic. Unlike Ethernet, FC is LOSSLESS, which means that there is no chance that a frame will be dropped when it is transmitted from a host to a switch or from a switch to another switch. This is why back in the days when they decided which network technology was better to handle SCSI traffic between the hosts and centralized storage, FC was the only choice at that time. But wait a minute, what is SCSI protocol? Oh, sorry, I forgot you’re the network guys. SCSI stands for Small Computer System Interface, and it is the standard protocol used to carry the data between the OS and the hard disk. SCSI used to be the definition of the protocol and the cable carrying this protocol, but as the need grew for more centralized storage, the SCSI cable was not fit for such a task for many reasons. We had two technologies at that time that satisfied our needs, which were:

• All servers should have access to the storage to efficiently utilize and manage it
• When I upgrade storage capacity, I do not have to shut down the servers introducing down time

Both technologies, Ethernet and FC, satisfied the two conditions at that time, but there was a catch. The SCSI protocol, when it was first designed, was meant to travel over very small distances. This means that it was very unlikely that we lost any SCSI blocks over such a small distance, which is why, in SCSI protocol, there is not much stress on frame recover mechanisms If SCSI lost a frame, it required one minute to recover that lost frame. ONE MINUTE! Yes, you read it right. A COMPLETE 60 seconds to recover one SCSI data unit. This means if we decided to send SCSI over the network, then the network protocol we use should be LOSSLESS. This is why, at that time, FC was the only choice. But how does FC achieve this lossless behavior? Well, FC is completely different from Ethernet. In Ethernet, once a host is connected to a switch or a switch is connected to another switch, they can start exchanging data immediately with no prior negotiations whatsoever. But in FC, it is different when you connect any device to your FC Switch. You should indicate to your FC switch what the type of the device is on the other end. Is it a switch, a host, or a storage array, because based on the type of the device on the other end, the switch will start a specific set of negotiations to ensure that while exchanging data with the device on the other end, no frames will be lost during data exchange. So how we can indicate to the switch what type of device is on the other end, and what is the set of negotiations between a switch and another switch or between a switch and a host? This is what we will discuss in the next post.