In a previous post we walked through Building VyOS Because…We Can as it is an open source routing platform and further, allows one to supply their own hardware. This is great as it can provide some additional flexibility that just isn’t present in the hardware provided by many vendors (such as upgradability, ability to easily virtualize, etc.). This post is going to walk through the configuration of OSPF on VyOS, how it compares to Mikrotik’s RouterOS and how to get the two routing solutions working together.
Hardware
As this will be a physical lab, it will require some physical hardware to accomplish.
Mikrotik Devices
In this case, it will be two Mikrotik RB450Gx4 RouterBoards running a current long term release of RouterOS (6.47.10). One will be named mt.vlan10 and the other mt.vlan20 and we will get into what the hostnames mean later on.
x86_64 PC or Server
In this case, it will be a Compulab Fitlet2 with the dual NIC FACET-card for a total of 4 Gigabit Ethernet interfaces. The Fitlet 2 in question is the model with the E3950 quad core CPU with 8 GB memory (way overkill for this application but it’s been collecting dust for awhile now so I figured I’d pull it out).
Managed Switch
Since VLANs will be used in this walkthrough, a managed switch will be used to provide access ports for the Mikrotik routers. Really any managed switch will work but this lab is really to load up the CPU of the Fitlet2 to get a better estimation of what it is actually capable of. The switch used here will be a Ubiquiti EdgeSwitch 8 running 1.8.5-Lite firmware with two ports setup in a 802.3ad LAG and an access port for each VLAN that will be assigned.
Configuration Overview
To start with, the basic configuration overview looks like the following:
The Fitlet2 will have an 802.3ad dual interface bond on Port 3 and Port 4 interfacing with the EdgeSwitch’s 0/7 and 0/8 interfaces (becomes 3/1 when the LAG is created) and pass tagged traffic on VLAN 10 and VLAN 20. 0/1 will be an access port on VLAN 10 and 0/2 an access port on VLAN 20. The Fitlet2 will also have Port 1 occupied with my home network just to get some more OSPF routes, remote access from my desktop via SSH and SNMP function. Each Mikrotik will simply be routers in the OSPF area with a client interface set on ether2 for performing throughput testing (this will come later). Technically speaking, the Fitlet2 is configured as a Router On a Stick (ROS) meaning traffic goes in and out on the same interface (the LAG in this case).
Fitlet2 Setup
Hostname: vyos.fitlet2
Interfaces: eth0 (Port 1) 10.1.10.3/24
eth1 (Port 3) bond0
eth3 (Port 2) bond0
bond0.10 172.16.10.1/30
bond0.20 172.16.20.1/30
lo 172.16.1.1/32
OSPF: Area 9 10.1.10.0/24
Area 13 172.16.1.1/32
172.16.10.0/30
172.16.20.0/30
Router ID: 172.16.1.1RB450Gx4-1 Setup
Hostname: mt.vlan10
Interfaces: ether1 172.16.10.2/30
ether2 172.16.15.254/24
lo 172.16.1.2/32
OSPF: Area 13 172.16.10.0/30
172.16.15.0/24
172.16.1.2/32
Router ID: 172.16.1.2RB450x4-2 Setup
Hostname: mt.vlan20
Interfaces: ether1 172.16.20.2/30
ether2 172.16.25.254/24
lo 172.16.1.3/32
OSPF: Area 13 172.16.20.0/30
172.16.25.0/24
172.16.1.3/32
Router ID: 172.16.1.3VyOS Configuration
Assuming a fairly default configuration (remember I have some extra stuff running on mine which is not needed for the lab and I will not be going over). Login into the VyOS machine (serial console, local console or IP based if you have it available) and enter configuration mode using the configure command. Start with creating the bond interface.
set interfaces bond bond0 mode 802.3ad
set interfaces bond bond0 hash-policy layer3-4
set interfaces bond bond0 description "Port 3 & 3 LACP"Next create the VLANs on bond0.
set interfaces bond bond0 vif 10
set interfaces bond bond0 vif 20Now assign IP addresses to the interfaces (remember, I have eth0 as an OOB management interface, it is not needed for this lab).
set interfaces bond0.10 address 172.16.10.1/30
set interfaces bond0.20 address 172.16.20.1/20
set interfaces loopback lo 172.16.1.1/32Now assign physical interfaces to bond0.
set interfaces eth1 description "Port 3"
set interfaces eth1 bond-group bond0
set interfaces eth3 description "Port 4"
set interfaces eth3 bond-group bond0Now configure OSPF (remember, I have a network for Area 9 as well but it is not necessary for this lab).
set protocols ospf area 0.0.0.13 network 172.16.1.1/32
set protocols ospf area 0.0.0.13 network 172.16.10.0/30
set protocols ospf area 0.0.0.13 network 172.16.20.0/30Finally set the system hostname and activate the config and save the running config to the startup config.
set system hostname vyos.fitlet2
commit
save
exitVyOS is not configured for basic OSPF function in a ROS setup .
Configure RouterOS
RB450GX4-1
For those who have never used RouterOS before, it may see a little strange. These configurations are assuming a no-defaults reset has been performed. In the case of a no-defaults configuration, the only way to get at a Mikrotik device is either through a console cable, a Mikrotik protocol called mac-telnet (it’s telnet but uses mac addresses instead of IP addresses) or through Mikrotik’s GUI client called Winbox using the mac address of the device.
Start with the hostname, creating a bridge for the loopback interface and adding IP addresses.
/system identity set name=mt.vlan10
/interface bridge add name=lo
/ip address add address=172.16.1.2 interface=lo
/ip address add address=172.16.10.2/30 interface=ether1
/ip address add address=172.16.15.254/24 interface=ether2Now, assuming ether1 is connected to the EdgeSwitch’s 0/1 (access on VLAN 10) and link activity is good…you should be able to ping the VyOS router using /ping 172.16.10.1. If you see timeouts, check your connection and switch configuration. Next, on to OSPF.
/routing ospf area add name=area13 area-id=0.0.0.13
/routing ospf instance set 0 router-id=172.16.1.2
/routing ospf network add area=area13 network=172.16.1.2/32
/routing ospf network add area=area13 network=172.16.10.0/30
/routing ospf network add area=area13 network=172.16.15.0/24It should be noted that the Router-ID should be set manually with Mikrotik where VyOS (and other routing platforms) will use the IP address of the loopback interface being advertised by OSPF as the Router-ID by default (unless it is otherwise overridden). At this point, you should be able to view the OSPF neighbors from the Mikrotik by issuing the following command /routing ospf neighbor print which should print something along the lines of this.
0 instance=default router-id=172.16.1.1 address=172.16.10.1 interface=ether1
priority=1 dr-address=172.16.10.2 backup-dr-address=172.16.10.1
state="Full" state-changes=4 ls-retransmits=0 ls-requests=0 db-summaries=0
adjacency=54m56s
Alternatively, viewing the neighbor list from the VyOS router using show ip ospf neighbor should show something similar. Finally we need to configure a DHCP server on the Mikrotik for use with testing later on. If nothing is actively plugged into ether2, it will not advertise via OSPF (dynamic entry in the OSPF Interface menu) so don’t be confused if you don’t see the 172.16.15.0/24 network from VyOS…it will show up eventually.
/ip pool add name=pool0 ranges=172.16.15.1-172.16.15.10
/ip dhcp-server add address-pool=pool0 name=dhcp0 interface=ether2 disabled=no
/ip dhcp-server network add address=172.16.15.0/24 gateway=172.16.15.254 dns-none=yesMikrotik will default to a 10 minute lease and since we are working on a small air-gapped setup here, DNS isn’t really necessary.
RB450Gx4-2
The configuration of this almost exactly the same as the previous…just different IPs, networks and a hostname.
/system identity set name=mt.vlan20
/interface bridge add name=lo
/ip address add address=172.16.1.3 interface=lo
/ip address add address=172.16.20.2/30 interface=ether1
/ip address add address=172.16.25.254/24 interface=ether2
/routing ospf area add name=area13 area-id=0.0.0.13
/routing ospf instance set 0 router-id=172.16.1.3
/routing ospf network add area=area13 network=172.16.1.3/32
/routing ospf network add area=area13 network=172.16.20.0/30
/routing ospf network add area=area13 network=172.16.25.0/24
/ip pool add name=pool0 ranges=172.16.25.1-172.16.25.10
/ip dhcp-server add address-pool=pool0 name=dhcp0 interface=ether2 disabled=no
/ip dhcp-server network add address=172.16.25.0/24 gateway=172.16.25.254 dns-none=yesAt this point, you should be able to plug a computer into ether2 of either Mirkotik, pull a DHCP address and ping the loopback address of any of the three routers. You can even run a traceroute to verify the path to each router.
Testing
This testing will be performed utilizing two identical Minix Z83 PCs with 4GB of memory and 32GB (fixed) storage. Both are running Red Hat 8.4 (though one is configured as a desktop and the other in a minimal install). Using iPerf3, we will test the throughput and CPU load capabilities of the Fitlet2. CPU load will be pulled from SNMP on the Fitlet2. The results are interesting.
| Packet Size | Average CPU Utilization | Average Throughput |
| 128 Byte Packets | 25.5% | 187 Mbps |
| 256 Byte Packets | 20.5% | 446 Mbps |
| 512 Byte Packets | 17% | 701 Mbps |
| 1024 Byte Packets | 4% | 888 Mbps |
| 1500 Byte Packets | 3% | 908.5 Mbps |
Working with smaller packets, the CPU clearly begins to have some issues while anything over 512 Bytes allows for relatively decent throughput (remember, computers that are being used for testing only have gigabit interfaces). Overall, I think I’m going to have to do some more testing in comparing the Fitlet2 to something like a RB450Gx4 as a quick test directly between the Minix PCs on the same Layer 2 segment with results varying by less than 1% suggests the Minixes to be the current limiting factor in this setup.
Pingback: VyOS vs RouterOS vs EdgeOS – Wireless Communications Technology