Wednesday, December 31, 2014

Nostalgia in Clustering

The close of 2014 made me remember an old clustering project I did around 2004 on a shoestring budget. The project was correlating customers into families, with a sub-task of deduplicating customer records (from typos and other issues). The entire project team was… me.

I gathered up a server with a couple of old hard drives as a mySQL server and PXE boot server, and four other computers PXE-booting into linux with openMOSIX for clustering. I didn’t have budget for cases for the four slaves, so used old cookie sheets to mount them. I used wooden dowels to fix two cookie sheet nodes together so they could sit vertically.

My processing was done in perl. Once OpenMOSIX reported a slave was free, a perl process would spawn and grab a workload from mySQL. OpenMOSIX would migrate the process to the open slave.

Fortunately I was able to complete the project with only four slaves. I had figured out my power supplies could power two slaves. I was working on converting a couple of ATX power supply extension cables into a Y-splitter and only using one power supply per "cookie".

I found some pictures of the nodes from an old presentation: