Hello all,
I am about to build a new platform and now I have to answer a number of crucial issues to make the right choice for hardware.
3-Node S2D Setup based on Windows 2019 Datacenter:
3 x HPE DL360 Gen10 NVME with 5 x 6.4 TB NVME SSD with an effective 25.6 TB Storage Pool
In every server I have the option to do 2 x SFP28 10 / 25Gbit adapter with 2 ports of the brands:
• Mellanox ConnectX-4 Lx (640FLR-SFP28) prefered
• Broadcom BCM57414 (631FLR-SFP28)
• Marvell QL41401L-A2G (622FLR-SFP28)
So I am figuring out what fits best to get the most optimal configuration.
For example, RoCE and iWARP are such a choice where one says that iWARP is faster:
https://www.chelsio.com/wp-content/uploads/resources/iwarp-s2d-updates.pdf
And where the other person says that RoCE v2 is faster:
http://www.mellanox.com/related-docs/whitepapers/WP_RoCE_vs_iWARP.pdf
My questions:
It is advisable to have a witness (FSW) available with a S2D cluster with 3 nodes. This is possible because of a 4th server that is not part of the S2D cluster. So a FileShare witness can be realized on this. Can a 3-node setup with witness (FSW) be realized without a switch? Should the witness FS be part of the iWARP or RoCE subnet? Or is it advised to always use a switch?
Which technique should I use? iWARP or RoCE? What is the fastest?