Buildroot Build Completed

Apparently I didn't get the bootloader working fully before my previous blog post. After failing at every attempt to add a terminal in the bootup scripts, it eventually occurred to me that it might not actually have booted. So, I ripped out Buildroot and used it to create an initialization ramdisk. Then I tried to use that to boot, and it failed for an unknown reason. At that point I tried using the search command of the bootloader to have the bootloader find the filesystem to run from. It also failed.

Eventually I was tired of this, and just tried setting root=/dev/sdb2 (second partition of second hard drive). Finally, this worked. However, it would do something else if I tried it on a device with more than one hard drive (it would boot from the wrong hard drive), which is why I didn't try it before. When it finished booting in two seconds, two logins popped up. Whenever I typed something in, it would go to a randomly selected login process. To fix this, I deleted the login I had added earlier, and left the login that had been automatically generated by Buildroot.

This caused it to work on my little laptop, so I tried it on Tien's (leader of the robotics club) laptop. To put it simply, it did not work. The entire screen came out flipped and most of the text was scrambled. Also his touchpad and keyboard crashed, so they wouldn't work when we booted back into Windows 10. We borrowed a mouse and keyboard from his brother and used it to access control panel. Resetting the touchpad brought both it and the keyboard back to normal. I will need to reconfigure the bootloader before I try it again on his computer.

This is the last post on this blog. I hope you have enjoyed it. I was going to make the previous post my last, but it seemed unfair to end without saying what happened with Buildroot.

kkbye

Almost There!

YEAAAAAAAAAAAAAH!

I think I got the bootloader working. That was . . . much more complicated than it should have been. Of course, I still am not done with that USB stick. Today, there were so many failed attempts to get the bootloader to function that I never thought I would figure out what would make it work. I first just dumped the image onto the flash drive and tried grub-install (GRUB is the bootloader), but of course that didn't work! Then I tried creating an EFI (boot protocol that newer computers use) partition . . . and ended up with multiple conflicting partition tables. At that point I had no idea what was going on - which one was the real one? Determined to figure this out, I kept issuing random commands until . . . GRUB Menu! YEAH! I still don't know how I did it.

After a little more work I managed to get it to boot the kernel (the core of the operating system). Shortly after my success, I physically broke the flash drive. UGH!!!!! This is not very surprising, considering how things in this process have been going. My little laptop had tilted sideways and bent the connector. In order to get the broken flash drive to work, I had to keep pressing down on the stick to hold it at a precise angle in order to maintain electrical contact. Imagine doing that while frantically trying to delete files because my big laptop was running out of disk space (I had to free 5 GB) for the copy operation. Finally, it copied onto my computer and I was able to set it up on a new working flash drive. Now I just need it to be able to open a terminal on the screen of my little laptop . . . I am almost there! The end is in sight!

Tech Stuff: Creating a Personal Cluster Part 2 - Designing a Network

Most of you will probably make a simple tree network - just attaching them to a network switch, which may in turn be connected to another network switch, etc. This method is extremely simple and is great if you do not have much internal traffic and your cluster is relatively small. Descent 100 Mbps 5-port switches are available online for $10, such as this. You should pay attention to the overall switch capacity though - some switches may not be able to run at top speed. The one which I provided a link to has a switching capacity of 1.6Gbps, so it can run at full speed (if they aren't lying/using unrealistic conditions). You can just connect a few of those to another to accommodate all of your nodes (assuming 16 nodes).

It is also possible for the more adventurous to create an OpenFlow software-defined network. These networks scale efficiently and can handle ludicrous amounts of bandwidth, but are overkill for most small clusters. These may be especially helpful if you want to use your cluster as a testbed for datacenter applications. In order to deploy SDN, you will need to add USB ethernet ports. Or, with enough work, you could theoretically run all of the networking over USB.

#slice2016

Buildroot Finally. . . Almost Builds the Root

Buildroot finally built most of the root! At the end of yesterday's work, it would not build the ISO image. So I did a git pull - downloaded the files from our online code storage - today and tried again. Something corrupted the cpupower module - the component I had the make problem with - even more. My fix would not work anymore, so I disabled cpupower and it kept going. Buildroot finished compiling, but the ISO still would not build.

I decided that I am going to set up my Buildroot compilation on a flash drive manually. Hopefully the bootloader will not become compromised again, as I have failed at setting up bootloaders on USB sticks before. Either way, I am going to keep trying things until it works. Minimally, I want the system to work before our part of the robotics team meets on Wednesday; ideally I want it to work today!

Tech Stuff: Creating a Personal Cluster Part 1 - Selecting SBCs and Storage

In my previous post, I do not believe that I explained everything in such a way as to allow one to plan a personal cluster. In this series of post, I will explain everything required to plan, design, construct, setup, and maintain your own cluster based off of single-board-computers. This explanation will be divided into these parts:

1. Selecting SBCs
2. Selecting Storage
3. Designing a Network
4. Power Supply
5. Selecting Management Systems
1. Mass SSH
2. Docker
3. KVM
6. Storage Management Systems
1. GlusterFS
2. LVM
7. Designing an Organizer
8. Using Buildroot to Create System Images
1. Minimizing Attack Surface
9. Setting up Firewalls
1. IPTables
2. HTTP proxies
10. Assembling the Cluster
11. Debugging and Testing the Cluster
12. Deploying Software on the Cluster
13. Adding on to the Cluster

Selecting SBCs

First, figure out how much RAM you need. Do you need 512 MB per node, 1 GB per node, or 2 GB per node? Or do you need a mix? After that has been decided, you should figure out your networking needs. After these have been determined, you should select a board with a good amount of CPU. It is important to note that the clock frequency and core count do not matter as much as the core type. For example, an ARM Cortex-A17 quad core at 1GHz is about equal to a quad Cortex-A7 at 1.5GHz for many tasks. However, Cortex-A7 has higher energy efficiency and is generally cheaper. Quad ARM Cortex-A7s are probably the most efficient available configuration in terms of CPU cost and energy efficiency for ARM SBCs. These include the Raspberry Pi 2, Orange Pi line, and similar. For 1 GB per node, Orange Pi PC is probably your best bet for moderate-network-traffic operations. If you can squeeze down to 512 MB per node, you can get similar performance for even cheaper. Also, you may want to carefully modify the clock-speeds of the SBCs to fit their environments and workloads. For example, bursty tasks should have a lower base clock and higher burst clock (with a lower temp threshold). You may want to test this on an individual board before using these settings on the entire cluster for safety reasons.

Selecting Storage

You must also determine your medium for data storage. I mainly suggest NAS units or USB hard drives. If some of your SBCs have USB 3.0, you can use a USB-to-SATA adapter to attach an SSD for quick access files. But if your tasks are more like archiving, you may want to just plug in inexpensive backup drives to USB 2.0. This will be slower, but will achieve higher storage capacity at a lower cost. The storage can easily become the more expensive than the compute nodes, so you should carefully think over all aspects of your decision. If you are performing mixed tasks, you may want to combine high-speed-low-capacity and low-speed-high-capacity storage mediums to achieve optimal speed and capacity. You should also use the RAM of your compute nodes to cache the storage devices for optimal performance.

#slice2016

Buildroot Resumes. . .

Buildroot is STILL building the root. It crashed last night because of a bug in a make script in the Linux kernel. While attempting to get the compilation to resume, I first tried re-running. That didn't work. Then I noticed one of the error messages said that it was missing a library, and googled it. The first search turned up only 8 results, none of which had anything to do with my issue. So I tried shortening the search term.

The second search came up with one related search result (all the others were garbage), which said to check the make script. When I opened the script, the compiler option was surprisingly absent. It is strange how nobody previously realized that it wouldn't compile when testing the code. Anyway, I inserted the compiler option and Buildroot resumed.
Still waiting . . .
     hoping . . .
          praying . . .
               crossing my fingers . . .
                     you name it, I'm doing it . . .
I really want to get going on transferring the robotics code to our flash drive so that we are not dependent on using one particular computer.
You never know what will happen in life, as this project has shown,
        so it is always good to have a back-up plan.

Bonus Tech Stuff: Personal Cluster Processing

With SBC's (single board computers) available for cheap, it has become incredibly inexpensive to deploy personal clusters. In this post, I will try to explain some of the necessary choices for constructing a cluster, and what you may want to use it for.

What can I use a Cluster for?

Personal clusters can be used for highly parallel tasks that may take a long time to complete on a personal computer. One that I need badly, and other people also probably need, is compilation of large source trees. With a powerful cluster running distcc, you could theoretically compile the Linux kernel in a minute. For people who do not compile that much, you could use it for simpler tasks such as web serving, http caching, a noSQL server, or high-capacity file storage. Or, you could donate processing time to a BOINC-based crowd processing project such as VirtualLHC@Home.

What Nodes (Computers) can I use?

You may use any device that you want that runs Linux. It is often appealing to set up a cluster with old out-of-use computers that you may have lying around. However, if you are going to be serious about your cluster, the energy consumption of the old computers may outweigh the cost of new nodes in the long run. I suggest using cheap single-board-computers for building a cluster because they are cheap, low-energy and long-lasting. You should select an SBC based on your needs. For general purpose, an Orange Pi PC or Orange Pi One would be your most cost effective. However, if your tasks involve a lot of RAM or are disk intensive, you may want to consider using an ODROID-XU4 with a USB-connected SSD or similar. For some people, a mix of boards could be best.

What Software can I use?

For some people, stock OS images and SSH will suffice. However, I suggest using Docker on Buildroot for more advanced users. This will allow you to select whatever Linux software you like, and will free up as much of the RAM and processing power as possible. Buildroot has support for most popular SBCs, and allows you to generate a minimalist image to run Docker on. Removing unnecessary software will increase processing capacity, decrease attack surface, and significantly reduce the chances of unexpected crashes. Docker will allow you to quickly deploy any Linux userspace software stack you want.

What can I do to Keep this Organized?

If possible, I would suggest building a slide-in mounting system with proper wire routing. However, some people just tend to make everything messy. In order to mitigate this, I have a few simple tips:

1. Label all wires
2. Color code whenever possible
3. Use wire routing whenever possible
4. Avoid floating boards and routers - if one wire slips, the whole thing could come crashing down
5. Whether or not you do the above, you should check that you know where every wire goes before bringing the system online

If you are only using a few nodes, 1-4 may not make sense. But most important is number 5. It may save hours of debugging because a networking switch was unplugged from power or something similar.

#slice2016

Buildroot Won't Build the Root!

Ugh, Buildroot won't build the root. Yesterday, I tried to set up a Buildroot system for robotics during our 9 a.m. - 6 p.m. meeting. The compilation would have taken forever on my laptop, so I tried it on my desktop through SSH. It took me until today, though, to figure out why it was not working. Apparently last time I tried to fix my desktop graphics, I broke a lot more than my graphics drivers. When I tried installing a dependency for compiling Buildroot, I was surprised to find my system in dependency hell.

To get out of this, I attempted to reinstall my operating system. This time I wanted to use Debian Stretch, because it had AMDGPU available (see previous post). However, the Stretch installer didn't work, so I instead installed Debian Jessie. That worked (slow graphics though. . .) and I was able to start the Buildroot compilation process. Hopefully it works this time. I am writing this while I am waiting, so the final outcome is yet to be determined.

Bonus Tech Stuff: Gaming Console Emulators on Linux

There are many open source console emulators available - both for newer (Wii) and older (Atari) consoles. For most older gaming systems, you can use RetroArch. RetroArch is an emulator framework for classic gaming consoles. There are RetroArch emulators for PSX, SNES, GBA, Sega Genesis, Atari 2600, and many other systems. There is a Linux distro called Lakka which acts as an easy-to-use front-end for RetroArch.

Wii and GameCube fans can use Dolphin - a high-speed, highly featured emulator. Wii and GameCube games are not actual system binaries - they are actually in an interpretive bytecode language. Dolphin dynamically compiles this bytecode into system code, allowing it to take advantage of high-performance hardware. The graphics are handled through OpenGL, so you can get better graphics than you would get on the actual console (HD/4K if you get a high-res texture mod). Dolphin also works with 84.4% of tested games, so you can run most games.

#slice2016

My Favorite Linux-Compatible Game: Robocraft

Earlier today, I was playing Robocraft. Fortunately it was not having connection errors, so I was able to finish many rounds. Using my awesome robot and a small glitch in the Unity physics engine, I was able to lift an enemy into the air. A second later the enemy exploded midair - blocks flying in every direction. Because of situations like this, I was able to gain points and help my team to victory against enemies which were much higher leveled.

During my first battle with my "lawnmower" robot (see below), I lost because I kept getting blasted out of the sky before I could deal any damage. However, I did earn a large sum of game money from these battles and, after selling all of my other robots to combine my funds, was able to upgrade significantly. This massive upgrade improved my robot enough to make a big difference in my gaming ability. So, in the next game my robot destroyed several enemy robots which helped my team to victory. The next few games provided me with opportunities to further improve my skills. It feels great to finally become proficient at my favorite video game!

Robocraft is a unity-based Steam game where you build and fight robots. It is currently in development, and is rapidly improving. This game requires design skill in order to create an efficient destroying machine. As you deal damage and scan with radar, your overclock gradually increases. Once the bar is full, you overclock level up. Each overclock level increases attack rate, damage, speed, and material strength.

In Robocraft, there are 8 types of weapons: Laser, Plasma Launcher, Rail Cannon, Nanotech Disruptor, Tesla Blade, Proto Seeker, Aeroflak Cannon, and Lock-on Missile Launcher. The most basic is the laser - which shoots low damage zero gravity projectiles with a relatively high firing rate. Also available relatively early on is the Plasma Launcher - which shoots higher damage projectiles relatively fast. This weapon is not overpowered because of a game mechanic called energy. All weapons consume energy and energy regenerates at a constant rate. Later in the game, aircraft are haunted by the Rail Cannon - a long-range high-accuracy weapon with a low firing rate. A single shot from one of these can blast a plane out of the air. Also important a little bit later is the Nanotech Disruptor - which allows you to heal members of your team. My current favorite - the tesla blade - is the only melee weapon in the game. Proto Seekers and Aeroflak Cannons can auto target and follow enemies, but only if they are detected on radar.

My "Lawnmower"

For my robot, I use a design which classifies as a "lawnmower." It is a plane with a row of five triple tesla blades on the front. If I fly into an enemy, it does fifteen impacts of 47400 damage in quick succession - enough to obliterate an enemy. However, this type of design has significant drawbacks. It is incredibly brittle and can be destroyed in a few shots. When I miss, my target typically destroys me in a few seconds. Also, my robot can be destroyed before it even reaches the target. Being especially vulnerable to powerful weapons, a few successful shots will render it useless or destroy it. I mitigate these risks by using high-level radar jammers and radar equipment to scan for potential targets.

Robocraft is an exciting and challenging game.  If you have never played it, you may want  to consider checking it out over break.

#slice2016

Break has Arrived!

Break has arrived! As soon as I came in the door and dumped my books on the floor, I headed straight to the Wii to start playing Super Smash Bros. Brawl. It is great to be able to take a break and play games without high level thinking. I still find it hilarious that I can often win in a 5-life battle against a max-level computer player without paying attention. It is difficult to recall the last time I have played more than a few rounds of Brawl during the week.

After Brawl, it was on to StarCraft II. When I previously played this game on my big laptop it was incredibly slow because of my Windows setup. So I decided to install it on my Linux system through PlayOnLinux. The install worked perfectly, however when I tried to launch it, the launcher was distorted (see picture). This was not because of PlayOnLinux - it was because my graphics driver was out of date. Updating the graphics driver should be easy once I get around to it. For right now, I just want to chill, not think too much, and enjoy the first day of break. That is why I am ending this blog post, and going to watch TV.

Bonus Tech Stuff: PlayOnLinux - Easily Running Windows Games (and other software) on Linux

PlayOnLinux is a system that simplifies the installation of Windows programs. It uses WINE. PlayOnLinux has configurations for running various games (and of course other stuff) that are Windows-only. It automatically installs the appropriate WINE version and sets up a WINE configuration which has been known to work with that program. For some games, it can even automatically fetch the installer.

PlayOnLinux runs separate WINE instances for each program in order to prevent them from clashing and manage different configuration needs. These are called "Virtual Drives." To make it easier to use, PlayOnLinux has both a command line and graphical interface. It is written in Python. There are Python scripts for each game which configure WINE to work with them.

#slice2016