The following explanation is going to be a bit convoluted, so please bear with me.
For the router, I’m planning to use one of the MSI B550 Tomahawk motherboards and 5900X processors that I used for the bandwidth tests with its biggest advantage being that I already have it, so less I have to buy. This combination along with 16GB of RAM and a 256GB NVMe should provide sufficient oomph as a router. However, it only has one 2.5GbE and one 1GbE NIC. I’m going to have to add 2 more NIC’s: One to support the 40GbE LAN connection to the switch and one to support the 5GbE WAN connection to the ATT modem. The onboard 1GbE I’ll use as a management port and the onboard 2.5GbE is going unused.
As is typical with most AMD socket AM4 motherboards it has a total of 24 PCIe lanes that we need allocate appropriately. Here’s what I have to work with:
One x16 PCIe4 slot (PCI_E1) directly attached to the CPU. This is where you usually connect a GPU. It is possible to bifurcate this slot into two x8 connections but would require additional hardware to physically realize this.
One M.2 connector (M2_1) directly attached to the CPU. It’s equivalent to a x4 PCIe4 connection.
A x4 PCIe4 connection to the B550 chipset (PCH). Everything connected to the B550 chipset is multiplexed onto this connection to the CPU. This includes both onboard Ethernet ports, a second M.2 (M2_2), and all the remaining PCIe slots (PCI_E2, PCI_E3, and PCI_4). There is quite a bit of overlap in the usage of these slots which renders most of them useless if M2_2 is used.
The MCX353 NIC for the 40GbE LAN connection will require a x8 PCIe slot, while a 5GbE NIC can be managed with a 1x PCIe slot. For the best performance, both of these NIC’s should have a direct attachment to the CPU and not through the PCH.
Unless something goes awry, the plan is to put the MCX353 in the x16 PCIe slot and go with a 5GbE Ethernet adapter in the M2_1 NVMe slot. The 256GB NVMe drive will be in the M2_2 NVMe slot which shouldn’t be a problem since I’ve previously verified that the MB can boot from this slot and I really don’t need lightning-fast performance for the file system. The M.2 5GbE adapter I’ve decided to use is a generic RTL8126 network chip I was able to pick up from IO Crest via Amazon. It should work well, but I’ll have to test this to be sure.
Currently the only issue I’m seeing is that I have no way to get video out of this configuration as I’m using the PCIE_E1 slot for the MCX353. Not a showstopper, but if things go very wrong I like having a way to connect a video card and keyboard to the system as a backdoor of last resort. Here’s to hoping I don’t need it.
I’m sure someone is thinking that if I replace the 40GbE NIC with a 10GbE NIC that I could use a different configuration not using PCI_E1 thus allowing me to use a GPU if necessary. Umm no. A 10GbE NIC would still need to be connected directly to the CPU making the configuration I’m using the only configuration for this MB that will work regardless of the NIC speed.
One more thing to point out: a nice to have feature that this system does not support would be ECC RAM. One of the tradeoffs I’m going to except.
Next up is to talk about software… I’m planning to use VyOS. I have nothing against any of the other options (OpenSense, pfSense, OpenWRT, et. al.), I’m sure they would all work well in this application. I just like the idea of VyOS. This mainly comes from a bit of history with Vyatta based routers that I am comfortable with, and VyOS is the current heir to that legacy. The adventure in this project will be building VyOS with the correct drivers which is new to me; and then getting everything to fit in a 1U rack space.
Thanks for reading. As always, comments and questions are welcome.
Comments
Loading comments…