Mellanox NIC's

During my excursion into the switch search I learned quite a bit about Mellanox, particularly about their NIC’s. To be honest, prior to this Mellanox had never registered to me, but with what I’ve learned, I’ve decided I’m going to use Mellanox NIC’s and DAC cables where possible. ¹

My reasons for going Mellanox are fairly pedestrian:

1. Mellanox has a great reputation for building bulletproof equipment.
2. Less chance of compatibility issues vs mix and matching NIC’s and DAC’s. Not as big of a concern as it used to be, but it can still be a thing.
3. The drivers are built into the Linux kernel. It just works. Or at least it should.
4. ConnectX3, while 5 generations old now, gives really good performance at a great price. The biggest downside being power consumption as they are not as efficient as the CX4 and newer NIC’s.
5. And as long as I stick with 10GbE, there should be no reason to change unless something quits working. At some point I’ll buy a few extra to keep as spares. And I could always move up CX4 or CX5 as cost come down. As of now, CX4 cards are typically 2x-3x CX3 cards and CX5 is double that.

As of today, I’ve acquired 2 MCX353’s and 3 MCX311A’s. I needed to make sure everything is as up to date as possible, and I got a bit lucky with this as all the cards had the latest supported software, though I did have to change the MCX353 NIC’s from InfiniBand to Ethernet. The MCX353 are QSFP+ NIC’s for 40/56GbE; since I have QSFP+ ports on the SX1024, I picked up two of these just to play with. If you need to update Mellanox NIC’s or change them from InfiniBand to Ethernet, there are good videos on YT. There are two things that you need, one is the latest software, and the other is the update tools. Both can be had from the Nvidia/Mellanox support website for anything CX3 or newer. ²

Thanks for taking the time to read. As always, comments and questions are welcome.