Buildroot Finally. . . Almost Builds the Root

Buildroot finally built most of the root! At the end of yesterday's work, it would not build the ISO image. So I did a git pull - downloaded the files from our online code storage - today and tried again. Something corrupted the cpupower module - the component I had the make problem with - even more. My fix would not work anymore, so I disabled cpupower and it kept going. Buildroot finished compiling, but the ISO still would not build.

I decided that I am going to set up my Buildroot compilation on a flash drive manually. Hopefully the bootloader will not become compromised again, as I have failed at setting up bootloaders on USB sticks before. Either way, I am going to keep trying things until it works. Minimally, I want the system to work before our part of the robotics team meets on Wednesday; ideally I want it to work today!

Tech Stuff: Creating a Personal Cluster Part 1 - Selecting SBCs and Storage

In my previous post, I do not believe that I explained everything in such a way as to allow one to plan a personal cluster. In this series of post, I will explain everything required to plan, design, construct, setup, and maintain your own cluster based off of single-board-computers. This explanation will be divided into these parts:

1. Selecting SBCs
2. Selecting Storage
3. Designing a Network
4. Power Supply
5. Selecting Management Systems
1. Mass SSH
2. Docker
3. KVM
6. Storage Management Systems
1. GlusterFS
2. LVM
7. Designing an Organizer
8. Using Buildroot to Create System Images
1. Minimizing Attack Surface
9. Setting up Firewalls
1. IPTables
2. HTTP proxies
10. Assembling the Cluster
11. Debugging and Testing the Cluster
12. Deploying Software on the Cluster
13. Adding on to the Cluster

Selecting SBCs

First, figure out how much RAM you need. Do you need 512 MB per node, 1 GB per node, or 2 GB per node? Or do you need a mix? After that has been decided, you should figure out your networking needs. After these have been determined, you should select a board with a good amount of CPU. It is important to note that the clock frequency and core count do not matter as much as the core type. For example, an ARM Cortex-A17 quad core at 1GHz is about equal to a quad Cortex-A7 at 1.5GHz for many tasks. However, Cortex-A7 has higher energy efficiency and is generally cheaper. Quad ARM Cortex-A7s are probably the most efficient available configuration in terms of CPU cost and energy efficiency for ARM SBCs. These include the Raspberry Pi 2, Orange Pi line, and similar. For 1 GB per node, Orange Pi PC is probably your best bet for moderate-network-traffic operations. If you can squeeze down to 512 MB per node, you can get similar performance for even cheaper. Also, you may want to carefully modify the clock-speeds of the SBCs to fit their environments and workloads. For example, bursty tasks should have a lower base clock and higher burst clock (with a lower temp threshold). You may want to test this on an individual board before using these settings on the entire cluster for safety reasons.

Selecting Storage

You must also determine your medium for data storage. I mainly suggest NAS units or USB hard drives. If some of your SBCs have USB 3.0, you can use a USB-to-SATA adapter to attach an SSD for quick access files. But if your tasks are more like archiving, you may want to just plug in inexpensive backup drives to USB 2.0. This will be slower, but will achieve higher storage capacity at a lower cost. The storage can easily become the more expensive than the compute nodes, so you should carefully think over all aspects of your decision. If you are performing mixed tasks, you may want to combine high-speed-low-capacity and low-speed-high-capacity storage mediums to achieve optimal speed and capacity. You should also use the RAM of your compute nodes to cache the storage devices for optimal performance.

#slice2016