On old hardware clusters rengolin@systemcall.com.br
Abstract: We tryied to show that one could assemble a cluster with old hardware and achieve a computer power close to some commodity hardware available today at low or none cost considering that we have a fast technology flow that invalidade a good piece of hardware in less than two years. Unfortunately we couldn't.


 
Introduction: Clusters already proved their value by transforming commodity hardware into super computers all over the globe and have already changed the notion of 'powerfull machines' from big vectorial super-hardwares to big lots-of-machines clusters.

All clusters have problems regarding network trafic (bandwidth and latency), problem distribution, parallelization of old codes and algorithms, process distribution and response collecting and most of this problems are being solve by smart workarounds. As this workarounds, in most cases, involves network minimization and parallelization of algorithms we can assume that it'd work on any kind of machine, being it fast or slow, vectorial or scallar, clasic or quantic. Indeed, many kind of clusters, from simple intel to powerful NECs have proven their clusterability.

We have received a bunch of old 486s, from SX-25 to DX4-100 of several flavors like HP, Compaq and some generic ones. The idea when they came to us was to assemble a cluster with them and try to achieve a computer power comparable with a Pentium III 1Ghz to use this kind of machine on schools with no funds for technology and insert them into the digital world to give their students more chance on the real life.


 
Material and Methods: We have received 10 machines: 2 486 sx-25, 1 486 sx-33, 3 486 dx2-66, 2 486 dx2-80, 1 486 dx4-100 and 1 Pentium 100. All of them with at least 4Mb of 40pins RAM sticks and most of them with onboard video and network chip.

All network interfaces we had, including onboard and offboard ISA ones, was 10baseT Ethernet cards and the video cards were all SVGA 512Kb to 1Mb internal memory.

We had to chose an operating system and we picked up Linux as our choice because we had more expertise in Linux than in other *nix operating systems and other OSes than *nix would slow down our already slow machines. We have chosen Conectiva Linux 6.0 as our distro.


 
Problems and Solutions:

We had several problems before installing Linux, that involved hardware, money and time issues. All pieces were old ones, with lots of dust and dirt on them and some with mal-functioning components, what restricted us to no more than five network cards, four motherboards (2 dx2-80, 1 dx4-100 and the pentium) and 40Mb of total memory.

We than, have decided to assemble the best ones first than try to fix the bad hardware later so we could optimize our time because we had already spent a whole month figuring out whose machines were alive and those that weren't.

We could assemble and install the Pentium-100 with 16Mb of RAM running Linux and so we went for the 486 dx4-100, using more 16Mb of RAM for it. Now the problem begin to be hard, our Linux distro kernel was pre-compiled for Pentium machines so the installation was fine but it won't boot from HD. So we would have to take one of two different paths:

  • Install it on the Pentium-100 and re-compile the kernel with 486 support for each machine and than put the HD on the 486 machine and boot. This process would have been far way boring and with so many pitfalls.
  • Choose an older distro, what would bring us new problems about old unsecure versions of software, lack of atualization procedures and loss of function (we wanted to use it on the real world today not 10 years ago!)

    At this time we'd spent more than two months and we've considered more facts:

  • First, assembling the dx4-100 using 16Mb of RAM we'd have only 8Mb for the next machine. At that time 1Mb of 40pins cost the same as 8Mb of EDO and 16Mb od DIMM, what made the cost of the project goes up too much.
  • The time we're spending in assuring the hardware would work every time was too much. Some times old junky hardware fails for no reason and other times it works fine. We could not afford this lack of quality.
  • The cost (money and time) for adding new nodes would be comparable as the cost of buying a more powerful (yet older) machine, like a K62-350 with 16Mb of RAM at that time, that had at least 10 time the computational power of one machine we had.
  • To have 10 times of the computational power of a single 486 we would have to assemble and link more than 10 486 machines, what would raise the cost of our project to a level that we could buy a brand new Pentium III 1Ghz with 64MB of RAM at that time.


    		•  
    Discussion and Conclusion: We really wanted it to be a reality for our junk hardware and that's why we'd spend so many time on this project, and that's why we have tryied so hard for it to work. This is the main reason why I can assure any one that this kind of project wouldn't work.

    The cost of hardware is not a monotonic crescent curve, it's more like an even function, being the lowest cost the pieces that are assemble in mass for the masses (the actual median computer with a cost of aproximated 800-1000 dolars). Hardwer more powerfull have not yet paid it's own investiment and older hardware are not produced any more, so that's rarity.

    For that reasons, the best aproach for schools and for low money associations is to buy no more than 1 year old hardware and assemble a cluster with the most machines you can with your money and know that you'll have to throw it away in a matter of years (2 or 3) and buy more 1 year old hardware at that time.


    		 
    Rengolin This page was created using Vim