Thousands of hosts on a gigabit LAN, maybe not

[joe at nethead.com (Joe Hamelin)]

On Fri, May 8, 2015 at 11:53 AM, John Levine <johnl at iecc.com> wrote:

> Some people I know (yes really) are building a system that will have
> several thousand little computers in some racks.  Each of the
> computers runs Linux and has a gigabit ethernet interface.


Though a bit off-topic I ran in to this project at the CascadeIT
conference.  I'm currently in corp IT that is Notes/Windows based so I
haven't had a good place to test it but the concept is very interesting.
They distributed way they monitor would greatly reduce bandwidth overhead.

http://assimproj.org

The Assimilation Project is designed to discover and monitor
infrastructure, services, and dependencies on a network of potentially
unlimited size, without significant growth in centralized resources. The
work of discovery and monitoring is delegated uniformly in tiny pieces to
the various machines in a network-aware topology - minimizing network
overhead and being naturally geographically sensitive.

The two main ideas are:

   - distribute discovery throughout the network, doing most discovery
   locally
   - distribute the monitoring as broadly as possible in a network-aware
   fashion.
   - use autoconfiguration and zero-network-footprint discovery techniques
   to monitor most resources automatically. during the initial installation
   and during ongoing system addition and maintenance.



--
Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474