Very Large Raspberry Pi Cluster - Part IV

So I've had a couple weeks here without much of a progress update. A lot of work has gone on. So here's a quick update before the holiday. First some photos. The first one is of the original prototype 3D printed out with some Pi in it.

This is a very nice design. The Pi just snap in place, there's room in the back to get to the micro USB power plug. But there are three problems:

1. I can only print half a 2U, so things had to be designed in such a way that once side is put into place, slid over, the other side goes in and is slid over and bolted, the first one slide over bolted, then the two remaining Pi on the ends are put in. And it requires two people.

2. It takes 50 hours to print one 2U holding 21 Pi.

3. It requires access to the rear of the rack. Without those rear bolts it wouldn't survive shipping.

So back to the drawing board. And here is the new design.

It's difficult to see what's going on here but there are 23 individually 3D printed pieces; one for each Pi and one for each end cap. They connect together by sliding on a piece of series 10 80/20 1"x.5"x10" long that has had about 3/4" milled off and the ends tapped. These are at the machine shop now so I will show some photos when I get them back.

One thing that's really nice about 3D printing is the RepRap; or the ability to rapidly prototype and revision. 3D printing is slow, but it's faster enough to do this. Here are some of my tests.

The one in the middle is close to the final design. You can see that two of the bolt hols of the Pi have a post and the last one has a threaded insert for a bolt.

Next post I will go into these two parts in detail. They have been designed for strength, speed of 3D printing and ease of assembly.

Very Large Raspberry Pi Cluster - Part III

I spent most of today fighting with Fusion 360 so I don't have much progress to report besides a serious complaint about Fusion 360. A new updated was released. If I didn't upgrade I couldn't save. If I upgraded I lost my changes. I had no choice but to upgrade, loose my changes and redo them. Then when I loaded up my models there were errors with unresolved imported components. The error states that if I save my model in this state I will loose my model. Personally I find this unacceptable and while I like Fusion 360, I like the disruption to the CAD world, many of the features, the price is reasonable and the fact that it runs on Mac and Windows, I also find it extremely buggy and a toy. I may be switching back to SolidWorks. Hopefully I get all of this resolved in the next few days and can report on the progress.

Update: After restarting Fusion 360 five times, and reloading the model after closing it with the error saying it will be destroyed, I finally got all the linked models and everything to load. So it was just a lot of ignoring errors and closing and reopening things and much ado about nothing, but extremely worrisome that just an update being pushed could cause such concern.

Very Large Raspberry Pi Cluster - Part II

Last week I posted about the very very large Raspberry Pi cluster that I'm building for Code One. Read that post here.

I realized after that post went live that I didn't give any backstory as to what we are actually building. When building something like this it's easy to document the entire process, it just takes 2-3 times as long to build. I figured I'd document the process in a fast and loose to blogs, YouTube and Twitter. So, before we go any further with the status of current progress let's backup.

Last year Stephen Chin and Gerald Venzl came to me with an idea. The conversation went something like this:

Stephen: "Gerald has an idea"
Gerald: "Let's build a HUGE biggest Raspberry Pi cluster for 2019 CodeOne. Like 1000 Raspberry Pis. The developers will love it. We will call it OSCAR!"
Me: "1000 is a lot, but 1024 is a better number"
Gerald, Stephen: "😃"

Back in the lab a few months later Jasper Potts and I did some math about how to pull off a cluster like this and we produced this rendering.

There will be 49 2U racks containing 21 Raspberry Pis (last weeks post was about this).
There will be 22 network switches.
There will be 18 USB power supplies.
There will be one server.
There will be 8 fans. This thing is going to get hot.
There will be 5 six foot server racks.
It will consume 120 amps of power.
It will require a fork lift to move.
It will network boot because can you imagine flashing 1024 SD cards?
In actuality, the server running the entire thing may be more powerful than all those Pis.
All of this barely fits in 5 racks.
This will travel to other Oracle events around the world next year.

WARNING: Actual numbers may vary, but the 1024 is the goal.

Very Large Raspberry Pi Cluster - Part I

I am building a very large Raspberry Pi Cluster and I'm going to document the process here. My goal is to post once a week until it's complete and shows up at Oracle Code One in San Francisco  September 2019.  Thus far an unimaginable amount of work has been put into this project to determine if it's even viable. We think it's viable so I'm going to start talking about it. There may be some mistakes. No, there will be some mistakes. Hopefully I won't make too many of them.

For the first taste, here is a rendering of a 2U rack that will hold 20 Raspberry Pi Model 3B+. I've printed out about 50 versions of this and this packs the most in while still allowing heat dissipation and still being serviceable in a standard server rack.

Disclaimer: The Raspberry Pi model is one I found on the Fusion 360 store thing. The rest is rendered by my own mad modeling skills.

Now that I've modeled this up and printed it I'm going back to the drawing board. It takes 14 hours to print the orange part which holds half of the Pi rack using an Ultimaker S5 with a .08mm nozzle 10% infield and .2 layer height. The Pi snaps in nicely but there are too many issues and this part needs to be nearly perfect.

Fusion 360 Autosave

Why this isn't the default I have absolutely no idea, but the first thing you should do when learning Fusion 360 is to turn on this option:

How to Turn on Automatic Save in Fusion 360

May the 4th Be With You

New Chapter

It has been nearly a year since I last posted to this blog. It isn't that I haven't wanted to or that I've been sitting around watching TV. I've been busy. Real busy. I now work for the Oracle Developer Marketing team as a Groundbreaker building experiences for developers. It's an amazing opportunity that I had to jump into 1000%. Here's one experience that I built last year and blogged about but not on this blog. Go check it out https://blogs.oracle.com/developers/building-the-oracle-code-one-2018-escape-rooms. Not only do I get to draw on my 20+ years of Software Engineering/Architecture experience, I get to use everything I've ever learned about being an inventor, tinkerer professional photographer, maker, prop builder, and videographer. I use 3D printers, CNC, camera's, Arduino, Raspberry Pis and a whole lot more on a daily basis. So follow me on Twitter https://twitter.com/chrisbensen and YouTube https://www.youtube.com/channel/UCLf9yhC2uo7TxNIwiJqIDcw as a lot more content should be coming! Like the world's largest Raspberry Pi cluster.

Buying Stuff on Amazon

There are two websites that I think are important when buying stuff on Amazon and I thought I'd share with everyone.

Amazon prices change. Sometimes you add something to your cart because you don't want to forget it but you don't need it right now. However if it were a really good price or something then it might change things. Paste the URL into Camel Camel Camel and it'll give you a graph of the price over time. That way you know you're paying way too much too.

https://www.camelcamelcamel.com/

Often times we base our purchase of a product on reviews, but often times the reviews are garbage. Review Meta changes that. They analyze the reviews in a way that none of us have time to do and provide a breakdown as well as their own star rating based on the reviews.

https://reviewmeta.com/

Call an AWS Lambda Function from another Lambda Function

AWS Lambda is I think one of the best additions to AWS, however it does have some issues becuase the architecture has changed over the years and one such problem the default VPC that it runs in. Default configuration runs great, but if you want better security, then not so much. But I digress. One of the most compelling features of Lambda is that you don't have to manager the VM or Docker Container. However because of that you might want to break up the responsibility between multiple Lambda functions, and in this case one Lambda will most certainly want to call another Lambda. So here's how you do that:

const aws = require('aws-sdk');
const lambda = new aws.Lambda({region: 'us-west-1'});

lambda.invoke({
      FunctionName: 'LambdaFunctionName',
      Payload: JSON.stringify(event, null, 2)
   },
   function(error, data) {
     if (error) {
        context.done('error', error);
     }
     if (data.Payload){
        context.succeed(data.Payload)
     }
   });


Note: I really appreciate Lambda. But to be honest they are difficult to use. There is a time and a place to use them and it should be in every AWS developer's bag of tricks. The real trick I alluded to above is the VPC, but that's for a proper DevOps person to discuss the nuances of that. Also know that while in small uses Lambda is cost effective there is a cross over point where Lambda becomes more expensive than EC2 or Elastic Container Services (Docker).

Docker ENV vs ARG

There are two types of variables in a Dockerfile, ENVs and ARGs. YOu define them in the Dockerfile like this:

ARG some_argument

This requires that you run:

$docker build --build-arg some_argument=something

Then there's ones with a default value.

ARG some_argument="default value"

You don't have to specify it during the docker build, but you can if you want.

ENV some_value=something
ENV some_other_value=${some_argument}

You can override these when you do docker run:

$docker run -e "some_value=others_stuff" ...

Sync a Shared Google Calendar with Calendar in iOS or macOS

I spent a long time trying to get a shared Google Calendar to show up in the Calendar in iOS or macOS and finally ran across this link containing the settings of which calendars are shared:

https://www.google.com/calendar/syncselect

You must go directly to this link as there doesn't appear to be any way of getting there otherwise, and you must check or uncheck the calendars to share and then save the settings. Good luck!

vlog 03 - Building a Death Star Ornament

Here is a Death Star Christmas ornament build with STl files provided on thingiverse here. A little bit of 3D printing, remixing an existing item on thingiverse and some electronics. Enjoy!

vlog 02