Imagine being able to manage scores of network switches as easily as scores of servers;
The stage is set for SDN (software-defined networking)
to change the way we push data through our infrastructures, with the
promises of more agile network provisioning and management, as well as
more affordable network hardware. But for many, the SDN concept is still
amorphous. What does SDN look like in practice?
To shed light on this question, I sat down with a few Dell Networking S6000 switches running Cumulus Linux 2.2.1. There are many approaches to an SDN solution, but one of the most significant is the advent of white-box switches and à la carte switch firmware. This is the essence of the solution offered by Cumulus Networks.
As with any other Debian Linux box, if you want to configure an Ethernet interface, you either do it from the shell prompt with the various ip tools, or add an entry to the /etc/network/interfaces file that provides details about that physical interface, then use
Cumulus makes use of Quagga for advanced routing protocols like OSPF and BGP; if you’ve used Quagga, you’ll be in familiar territory. Further, Quagga offers a Cisco-like interface that will make veteran Cisco admins feel right at home.
The upshot is that network admins who have Linux experience will be way ahead of the game, while network admins light on Linux experience will need to put in some time to get acclimated to a very different way of working with a network device.
However, keep in mind that the completely open framework of Cumulus Linux makes it possible to develop a custom command-line interface that can be used to manipulate the switch. This CLI could be as simple as some Bash scripts, a Python tool, or any other programming framework. It’s only Linux after all.
Running as a daemon on the switch, switchd monitors the Netlink communication channels for configuration changes to any part of the network stack, such as assigning an IP address, adding a port to a bridge group, and configuring a firewall rule. Whenever switchd detects a change, it configures the Ethernet controllers to match. Thus, the Linux layer is used as a template of sorts applied to the hardware.
Likewise, interface statistics and other data coming from the physical ports and switching internals are communicated back up through the chain and presented within the Linux layer. Thus, Linux interface inspection commands such as
That is the essence of Cumulus Linux. From the operator's standpoint, it’s merely Linux. The net result is the pairing of wire-speed network operation with a simple configuration layer.
That script could perform any type of specific configuration tasks, including adding local APT sources and installing custom packages, installing data center orchestration clients for Puppet, or distributing pre-shared SSH keys -- you name it. If you can script it, it can run. This procedure can include the application of the required license file, as the switch will not allow the use of any switching ports unless a valid license file is installed.
The Dell Networking S6000-ON switches that formed my test bed are built to a familiar spec. The S6000-ON has 32 ports of 40G Ethernet or 96 ports of 10G and eight line-rate 40G Ethernet ports in breakout mode, plus a single out-of-band Ethernet management port and a serial console port. The ports are referred to internally as swp1 through swp32, with a loopback at lo.
Let's say we want to create a two-link LACP-bonded connection to another switch, then place a few ports into VLAN10 with this switch. We do this by defining the bonds and bridge specifications in the /etc/network/interfaces file. For the bonded ports, we create the bond and assign the ports we need as follows:
auto swp2
The
above lines will place interfaces swp1 and swp2 into an LACP bond using
802.3ad. A similar configuration on a switch connected to those ports
will bring up two bonded interfaces.
Of course, adding these lines to the file doesn’t enable the ports themselves. We need to bring up the ports and the bond from the command line:
This
highlights a conceptual difference between Cumulus Linux and a
traditional switch. The configuration file is simply a file, and changes
made within the file do not happen immediately -- unlike changes made
using a standard switch command-line interface. You can make changes
that are immediately applied to the switch with the Linux ip tools, but
those will not be retained in the interfaces file. Thus, admins new to a
Linux-driven switch will need to understand the configuration
persistence rules.
If we have a large number of ports to enable, we don’t need to do them one at a time. Instead, we use the shell because it’s Linux. The following command enables ports 1 through 24:
We can check the status of the bond as you might expect, by viewing /proc/net/bonding/uplinkbond0.
Now that we have our bonded interface, let’s create a bridge, or VLAN, and assign our ports:
Here
we have created VLAN128, placing ports swp10 and swp11 on that VLAN,
with their traffic untagged. If I wanted to add a tagged port, such as a
trunk interface, I could add
So far, everything we've seen here is standard procedure for using interface bonding and bridging on a Debian Linux system.
To look at the interfaces, we use the Linux ip tools:
Likewise, we can use
When
working with routing protocols, the rules are a little different. As
noted above, Cumulus uses Quagga to handle the configuration of these
protocols, and several methods can be used to manipulate the
configuration. In addition to the standard Quagga
As an example, we could specify an OSPF area thusly:
Or we could do the same through
The companion to
There is also LLDP and CDP support via
Logging is handled with syslog as you would expect, with logs present in /var/log.
From a Linux admin perspective, Cumulus Linux is standard stuff, and most of what has been outlined in this article will seem remedial, basic operations. That’s the underlying principle and value of Cumulus Linux -- it's really just Linux. It can be scripted, adapted, adjusted, tweaked, and customized to suit nearly any network or administrative requirement, using commodity hardware switches.
The brilliance of Cumulus Linux is in the simplicity. For a significant number of seasoned Linux administrators and use cases, Cumulus Linux switches will carry no substantive learning curve to integrate into production networks. Paired with data center orchestration tools, Cumulus Linux offers a combination of flexibility, compatibility, and ease of use that will shake up traditional networks everywhere.
To shed light on this question, I sat down with a few Dell Networking S6000 switches running Cumulus Linux 2.2.1. There are many approaches to an SDN solution, but one of the most significant is the advent of white-box switches and à la carte switch firmware. This is the essence of the solution offered by Cumulus Networks.
Decidedly Debian
Cumulus Linux is based on the Debian distribution. You may be SSHing into a 1U box with 52 physical interfaces, but from the inside, a Cumulus Networks switch is like any other Linux box. I ran switches built on a Freescale PowerPC base with 2GB of RAM and a 394MB read-write overlay root partition with 390MB free, or what you might expect from a small but complete Linux system.As with any other Debian Linux box, if you want to configure an Ethernet interface, you either do it from the shell prompt with the various ip tools, or add an entry to the /etc/network/interfaces file that provides details about that physical interface, then use
ifup/ifdown
to control the interface. If you know how to do standard bridging,
routing, forwarding, and firewalling with Debian, then you know how to
configure Cumulus Linux for those tasks -- there is no real learning
curve.Cumulus makes use of Quagga for advanced routing protocols like OSPF and BGP; if you’ve used Quagga, you’ll be in familiar territory. Further, Quagga offers a Cisco-like interface that will make veteran Cisco admins feel right at home.
The upshot is that network admins who have Linux experience will be way ahead of the game, while network admins light on Linux experience will need to put in some time to get acclimated to a very different way of working with a network device.
However, keep in mind that the completely open framework of Cumulus Linux makes it possible to develop a custom command-line interface that can be used to manipulate the switch. This CLI could be as simple as some Bash scripts, a Python tool, or any other programming framework. It’s only Linux after all.
Behind the scenes
Although the configuration of the switch is handled entirely within Linux, the kernel itself is not directly involved in packet switching, routing, or firewalling. Instead, Cumulus has developed a translation service called switchd that communicates network configuration changes made in Linux to the hardware Ethernet controllers driving the physical ports and handle the internal switching and routing.Running as a daemon on the switch, switchd monitors the Netlink communication channels for configuration changes to any part of the network stack, such as assigning an IP address, adding a port to a bridge group, and configuring a firewall rule. Whenever switchd detects a change, it configures the Ethernet controllers to match. Thus, the Linux layer is used as a template of sorts applied to the hardware.
Likewise, interface statistics and other data coming from the physical ports and switching internals are communicated back up through the chain and presented within the Linux layer. Thus, Linux interface inspection commands such as
ip link show and ifconfig work as you'd expect.That is the essence of Cumulus Linux. From the operator's standpoint, it’s merely Linux. The net result is the pairing of wire-speed network operation with a simple configuration layer.
Booting, provisioning, and configuration
The first task with a new switch is to boot it and install an OS. Cumulus uses ONIE (Open Network Install Environment) for this, and it boots looking for an install image either on a local USB device or on the network via standard DHCP and TFTP options. Once the OS is loaded, it runs initial boot automation procedures that use DHCP options to locate further instructions -- by downloading and running a Python script, for example. Thus, with a properly configured network boot configuration, switch provisioning could be completely automated. Then, to configure the switch, we SSH into it.That script could perform any type of specific configuration tasks, including adding local APT sources and installing custom packages, installing data center orchestration clients for Puppet, or distributing pre-shared SSH keys -- you name it. If you can script it, it can run. This procedure can include the application of the required license file, as the switch will not allow the use of any switching ports unless a valid license file is installed.
The Dell Networking S6000-ON switches that formed my test bed are built to a familiar spec. The S6000-ON has 32 ports of 40G Ethernet or 96 ports of 10G and eight line-rate 40G Ethernet ports in breakout mode, plus a single out-of-band Ethernet management port and a serial console port. The ports are referred to internally as swp1 through swp32, with a loopback at lo.
Let's say we want to create a two-link LACP-bonded connection to another switch, then place a few ports into VLAN10 with this switch. We do this by defining the bonds and bridge specifications in the /etc/network/interfaces file. For the bonded ports, we create the bond and assign the ports we need as follows:
auto swp1iface swp1auto swp2
iface swp2
auto uplinkbond0iface uplinkbond0 bond-slaves swp1 swp2 bond-mode 802.3ad bond-miimon 100 bond-lacp-rate 1 bond-use-carrier 1 bond-min-links 1 bond-xmit-hash-policy layer3+4Of course, adding these lines to the file doesn’t enable the ports themselves. We need to bring up the ports and the bond from the command line:
$ sudo ifup swp1$ sudo ifup swp2$ sudo ifup uplinkbond0If we have a large number of ports to enable, we don’t need to do them one at a time. Instead, we use the shell because it’s Linux. The following command enables ports 1 through 24:
$ for i in {1..24}; do sudo ifup swp$i; doneNow that we have our bonded interface, let’s create a bridge, or VLAN, and assign our ports:
auto swp10iface swp10auto swp11iface swp11auto vlan128iface vlan128 bridge-ports swp10 swp11 bridge-stp onswp12.128, which would include traffic from port swp12 that was tagged 128.So far, everything we've seen here is standard procedure for using interface bonding and bridging on a Debian Linux system.
To look at the interfaces, we use the Linux ip tools:
$ ip -s link show dev swp13:
swp1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast master uplinkbond0 state UP mode DEFAULT qlen 500 link/ether 6c:64:1a:00:2e:16 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 39442 318 0 8 0 318 TX: bytes packets errors dropped carrier collsns 37952 29 0 0 0 0 brctl to show bridging information:$ brctl showbridge name bridge id STP enabled interfacesvlan128 8000.6c641a002e1e yes swp10 swp11vtysh utility that drives like Cisco IOS, Cumulus-specific commands such as cl-ospf and cl-bgp
achieve the same results without the need to work with the command-line
interface. These allow for one-line configuration changes and easy
scripting.As an example, we could specify an OSPF area thusly:
$ sudo cl-ospf network add 192.168.1.0/24 area 0.0.0.0vtysh:$ sudo vtysh
Hello, this is Quagga (version 0.99.21).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
q# configure terminal
q(config)# router ospf
q(config-router)# network 192.168.1.0/24 area 0.0.0.0cl-ospf is cl-bgp,
which manipulates the BGP daemon. Routing protocol availability via
Quagga is handled as usual, by specifying which protocols to enable in
the Quagga daemons file. Here it's important to note that changes made
in the Quagga.conf file persist over reboots but are not implemented
immediately unless Quagga is restarted, whereas changes made via the cl-ospf and cl-bgp
commands take place immediately but do not persist across reboots.
Changes made in the Quagga command-line interface are made immediately
and retained if a wr mem command is issued. Thus, you can drive the routing protocols either way for the best of both worlds.lldpcli:$ sudo lldpcli show neighbors ports swp1-----------------------------------------------------------LLDP neighbors:-----------------------------------------------------------Interface: swp1, via: LLDP, RID: 3, Time: 0 day, 00:16:54 Chassis: ChassisID: mac 6c:64:1a:00:2a:8f SysName: leaf1 SysDescr: Cumulus Linux MgmtIP: 192.168.0.11 Capability: Bridge, off Capability: Router, on Port: PortID: ifname swp1 PortDescr: swp1-----------------------------------------------------------Switch internals
Aside from networking functions, general switch maintenance is simple. There are provisions for two firmware images -- one primary, one alternate -- and command-line options to swap between them if needed. The file system layouts differ depending on the architecture of the switch. PowerPC-based switches have an overlay file system visible to each image, while x86-based switches have separate logical volumes. The separate file systems are nevertheless available to each image, so you can copy files to and from active and alternate image file systems if needed.Logging is handled with syslog as you would expect, with logs present in /var/log.
From a Linux admin perspective, Cumulus Linux is standard stuff, and most of what has been outlined in this article will seem remedial, basic operations. That’s the underlying principle and value of Cumulus Linux -- it's really just Linux. It can be scripted, adapted, adjusted, tweaked, and customized to suit nearly any network or administrative requirement, using commodity hardware switches.
The brilliance of Cumulus Linux is in the simplicity. For a significant number of seasoned Linux administrators and use cases, Cumulus Linux switches will carry no substantive learning curve to integrate into production networks. Paired with data center orchestration tools, Cumulus Linux offers a combination of flexibility, compatibility, and ease of use that will shake up traditional networks everywhere.
Hello, my name is Mohd Azahar. I'm a self-employed Pivrate from the India.
No comments:
Post a Comment