Thousands of hosts on a gigabit LAN, maybe not

[chuckchurch at gmail.com (Chuck Church)]

Sounds interesting.  I wouldn't do more than a /23 (assuming IPv4) per subnet.  Join them all together with a fast L3 switch.  I'm still trying to visualize what several thousand tiny computers in a single rack might look like.  Other than a cabling nightmare.  1000 RJ-45 switch ports is a good chuck of a rack itself.

Chuck

-----Original Message-----
From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of John Levine
Sent: Friday, May 08, 2015 2:53 PM
To: nanog at nanog.org
Subject: Thousands of hosts on a gigabit LAN, maybe not

Some people I know (yes really) are building a system that will have several thousand little computers in some racks.  Each of the computers runs Linux and has a gigabit ethernet interface.  It occurs to me that it is unlikely that I can buy an ethernet switch with thousands of ports, and even if I could, would I want a Linux system to have 10,000 entries or more in its ARP table.

Most of the traffic will be from one node to another, with considerably less to the outside.  Physical distance shouldn't be a problem since everything's in the same room, maybe the same rack.

What's the rule of thumb for number of hosts per switch, cascaded switches vs. routers, and whatever else one needs to design a dense network like this?  TIA

R's,
John