Tuesday, August 9, 2016

#DeltaGate Review: A+++++ Would Experience Again

I was lucky enough to be included in the #DeltaGate beta test yesterday, so I thought I should write a review. It’s my way to thank Delta for a truly memorable day!

What is #DeltaGate?

Based on the available evidence, #DeltaGate appears to be Delta’s version of Netflix’s Chaos Monkey, except it starts actual fires instead of terminating VM instances. Usually it is reserved for the most elite of travellers, but in an unprecedented move, Delta gave us mere mortals a glimpse of true luxury.

Delta opened up the beta test at 2:30am on August 8th, 2016 with a bang. Normally they would only set one generator on fire, but in order to handle the load of a wide scale beta, they opted to set all of the generators on fire. I am grateful that Delta truly cares about me, the #DeltaGate beta participant.

Lighting a datacenter on fire normally wouldn’t be a problem, as Delta surely has redundancy across multiple sites. I suspect that Delta wanted so badly for #DeltaGate to be a success that they invented time travel, and travelled back in time to decide against any datacenter redundancy. Thank you Delta, your efforts have not gone unnoticed.

Tarmac Adventure #1

Since I was a small child, I’ve dreamed of spending weeks at a time on the tarmac. Delta dropped the ball, however, and only gave me three hours the first go around :( According to our captain, it was too hot in Las Vegas in August for us to take off, and he was waiting for the wind to be just right so we wouldn’t risk losing an engine.

Apparently the risk of losing an engine was too great, so we returned to the gate at the 3 hour mark at which point a new maintenance issue was discovered. What was this maintenance issue? How long would it take to fix? Nobody knew! This is the beauty of #DeltaGate: everything goes horribly wrong, and everybody just runs around with their hair on fire! (Figurative hair, however, as I don’t think #DeltaGate sets actual people on fire, only equipment.)

Tarmac Adventure #2

At 4:00pm, we boarded the 9:40am flight for another adventure on the tarmac. This time I was certain Delta would let us stay on the tarmac for longer than 3 hours, and they certainly made a valiant effort. This time #DeltaGate struck a baggage tractor on its way to our plane.

deltagate-tractor.jpg

Despite Delta’s best efforts to set everything on fire, we departed after only one more hour on the tarmac. In order to keep everybody happy, Delta decided to leave several Comfort+ seats open, despite passengers queued up on the upgrade list. Thank you, Delta.

upgrade-list.jpg


After a sufficiently bumpy flight, #DeltaGate had one last trick up its sleeve, and we were privileged to a gate delay upon arriving in Atlanta. I’m certain you could have made it worse, Delta, and for that I am disappointed, but you truly made your best effort to set everything on fire with #DeltaGate. I finally arrived at my destination only eight hours late, in one piece, and not on fire.

Sunday, September 27, 2015

Lightwad Top Hat

Introducing the Lightwad Top Hat

Burning Man has this way of causing otherwise reasonable people to spend their entire summer and thousands of dollars preparing for a week of suffering in the desert.

People go to Buring Man for all sorts of reasons. I, for one, enjoy playing the role of teetotalling, fully-clothed cyclist, lost in the desert on opposite day. But mostly, Burning Man is an excuse to build cool toys.

This year I built a shiny gold top hat covered with 480 addressable LEDs. It's driven by a Teensy 3.1 (96 MHz ARM Cortex-M4), which is wired up to a compass, gyro/accelerometer, GPS receiver, and a 2.4 GHz nRF24L01+ transceiver. I also built an Arduino-based wireless NES controller to operate the hat.

This thing was a real pain in the ass to build, but v2.0 is in the works (which I'm going to open source). I'm removing as much of the frustration as possible so you too can get asked to turn down your lights at Burning Man (true story).

Interestingly, not everybody who goes to Burning Man draws inspiration from the sun. There exists a special breed of burners known only as darkwads. The veritable ninjas of the playa, they stealthily remove all lighting from their person and bike. This enables them to dart around unseen once the sun goes down, only to reappear in front of moving art cars and bikes. I salute you, darkwads, for it takes a true hero to brave the playa at night with but a faded glow stick to light your way.

Despite my lifelong dream of being a darkwad, I lack the sheer dedication required to reach their level. Much to my chagrin, I was given the playa name Lightwad after rescuing a village of refugee LEDs and teaching them to love. In lieu of some web 2.0 cloud-enabled gluten-free short-on-vowels name, I'm calling this thing the Lightwad Top Hat.

By The Numbers

This project got a little out of hand, in the best way possible. It started out relatively modest, but I have this feature where I get really excited (drunk) and brainstorm (beerstorm?) more engineering work for myself.

  • Parts: ~$350-400 (excluding the battery pack)
  • Time: ~200 hours
  • Solder Joints: 364
  • LEDs: 481
  • Lead Fumes: Change places!

LED Matrix and Hat Construction

The core of the hat is a 48x10 matrix of WS2812B addressable LEDs (popularized by Adafruit as NeoPixels). I'm a big fan of these things because they're exceptionally bright, reasonably easy to work with, and can do well above 30 fps in a matrix this size.

I soldered the matrix together one column at a time, testing after attaching each column, and applying a layer of duct tape to the backside as I went. I attached power and data connectors to the beginning of the matrix, and a second power connector to the end. A strip of this length requires power at multiple points to maintain even brightness and color.

After soldering and testing the matrix, I applied hot glue to the ends of the silicone sleeves.

The main cylinder of the hat is made out of boxboard that I first covered in a layer or two of duct tape. I attached the matrix to the boxboard with double-sided tape, and applied some more hot glue for good measure.

I was in the mood to lose a little fingertip skin at this point, so I broke out the gorilla tape and wildly stuck it to everything in sight. The result was a cylinder of LEDs with some wires sticking out the top.

I then fashioned a cardboard divider to sit about halfway up the cylinder and serve as the top of the head (who said head?) compartment.

Before I started on the hat, I had ordered a couple shiny gold cloaks for the burn. It seemed too good to be true at $12/each from China, but I'm an eternal optimist. Sadly they weren't the luxuriously flowing wizard cloaks I had envisioned, but the fabric was still nice and shiny. To make the top and brim of the hat, I glued some gold fabric to cardboard, and willed it into shape.

The hat wasn't exactly comfortable yet, so I used a bicycle helmet padding kit to fit it to my head, and added a chin strap so I wouldn't lose it out on the playa.

Hat Electronics

Shiny gold top hats are cool, but shiny gold top hats that light up are even better.

Parts list:

  • Teensy 3.1
  • nRF24L01+ transceiver
  • u-blox NEO-6M GPS breakout board w/ U.FL antenna
  • HMC5883L compass breakout board
  • MPU6050 gyroscope / accelerometer breakout board
  • LM2596 based buck converter
  • 8x AA battery box
  • Industrial strength velcro

The hat electronics went through a number of revisions, eventually reaching the current design. The core of the hat is the excellent Teensy 3.1, which offers a small form factor, Arduino compatibility, lots of I/O pins, and a 96 MHz (when overclocked) ARM Cortex-M4. It has an nRF24L01+ transceiver to receive input from the NES controller, a GPS receiver (along with a compass and accelerometer) to guide you safely back to camp, and an LM2596 based buck converter to support various power sources if needed.

My initial plan was to use alkaline AA batteries to power the hat, and in that configuration, the electronics shared an 8x AA battery holder with 5x AA batteries.

This design had a couple flaws.

  • The buck converter I used in this iteration was generating a ton of heat, and while it had a very wide operating temperature range, everything in the enclosure was getting pretty toasty.

  • I neglected to RTFM on the WS2812Bs, and was incorrectly assuming that there wouldn't be much power draw from an "off" LED. As it turns out, the idle draw is roughly 6mW. Coupled with the rest of the electronics, the total idle draw is around 4.5 watts. With even a small number of LEDs turned on, the load caused a significant voltage drop, making alkaline AAs unsustainable.

I didn't like the idea of having a bunch of lithium batteries resting on my scalp, and additional alkaline weight would have been too heavy to wear. I opted then to use an external power source and a better buck converter.

I cleared out the rest of the 8x AA battery holder to give the components some breathing room (and make room for the new buck converter). It turns out that digging out a jumbled mess of hot glue and narrow gauge wire is a giant pain in the ass, so I velcroed the components in place in case I needed to fix a solder joint at the burn.

I assembled the hat electronics about as you'd expect -- wired everything up and tested it incrementally. I attached the enclosure (8x AA battery holder) to the hat divider with velcro, and added connectors for battery in, LED matrix data out, and LED matrix power out.

To power the hat, I used a 195 watt-hour lithium-ion battery bank, and kept it charged with a 10 watt ~18v solar panel.

Wireless NES Controller

What fun is a hat if you can't control it with a wireless NES controller?

Parts list:

  • USB NES controller
  • 2x AAA battery holder
  • 5.1 volt boost converter (like you'd use to make a USB charger)
  • SPST rocker switch
  • Arduino Nano
  • nRF24L01+ transceiver
  • 1x WS2812B LED
  • Vibration motor

I started with a cheap USB NES controller and retrofitted it with an Arduino Nano. It talks to the hat using an nRF24L01+ transceiver, and provides feedback to the user using a single WS2812B LED coupled with a cell phone vibration motor.

For DEF CON, I built an NES controller that can hijack Logitech wireless mice, and I quickly fell in love with using NES controllers as a platform for wireless input. The nostalgia combined with a sufficiently roomy case and simple PCB makes these things pretty fun to work with.

I had to make some modifications to the controller to fit all of the goodies inside. This required removing some plastic to provide access to the USB port, mounting the rocker switch, making room for the voltage converter and batteries, and mounting the LED.

The PCB also required some love. The largest thing in the controller is the 2x AAA battery holder, and I needed to modify the PCB to make room. Conveniently, the section of the PCB above the A/B/start/select buttons doesn't contain anything we care about.

As I approached the Ballmer peak, I tediously soldered everything together and hot glued that shit down.

The controller software is really simple, and just transmits a single byte each time a button is depressed. I included some basic rate-limiting logic to conserve battery, and further input rate-limiting is implemented on the receive end.

Lightwad Top Hat in Action

These functionality examples barely scratch the surface, and the real magic is going to come from community exploration of the software side of the hat.

Procedural Randomized Rainbow Tail

NES Controller Driven Rainbow Tail

Compass Demo

Version 2.0 (making it suck less)

This hat is pretty fucking cool, but it's certainly rough around the edges. I've been iterating on the design, and will be open sourcing version 2.0 as soon as it is ready.

If you're one of those maker-wizard types, you should probably start building one right now, but if you're a mere mortal like the rest of us, you'll probably hate yourself less if you hold off until v2.0.

If you want to stay in the loop, send me an email (gmail) at marc.newlin (or find me on the twitter) and you'll know the moment it's released.

Safety Third!

Monday, June 1, 2015

Write beautiful code

Writing code is much more than an engineering discipline; it is a true art form. Each line of code you write reflects your understanding of the language, your ability to communicate ideas to other engineers, and the lessons lessons from previous projects.


There is no "correct" code style, just as there is no "correct" solution to any given engineering task. The internet is littered with style guides written in absolutes, and these can be useful, but often forget that the context of a project is as important as the code itself. Mocking up a proof of concept for a hobby project is very different than implementing a base class to be used by your entire team.


When you write code, your first priority is to satisfy functional and performance requirements. This is universally requisite, but as engineers, we often forget the importance of ensuring that our code be easy to understand by other programmers (including your future self). Remember that time you looked at a project from six months ago and had no idea how it worked? It doesn't have to be that way.


Let's dispel some myths about writing beautiful code.


Myth #1 - Verbose comments make you a bad programmer


Many engineers profess that comments are unnecessary, and that a good engineer should be able to read code completely devoid of comments. This sounds cool, and conjures up an image of a badass motherfucking sorcerer-hacker wizard-god reading the matrix on 16 goddamn monitors at once. The thing is, as mere mortals, we are better at processing natural language than code. Certainly we need to be able to read and understand code quickly, but it is usually easier to parse a few sentences in your native language than to truly understand the code in an unfamiliar function.


Myth #2 - White space is bad


A common school of thought is that less code is better, and less white space is better. Surely if we remove all blank lines and keep our code as tiny as possible, everybody wins, right? I mean, classic Greek and Latin was written in all caps and without spaces, so we should apply that to our code, right?


Let's consider the following function, with and without comments and line breaks.


No comments, no line breaks


std::vector<std::complex<double>> plcp_data::encode(Rate rate, std::vector<unsigned char> payload, unsigned short service)
{
   RateParams rate_params = RateParams(rate);
   int num_symbols = std::ceil(double((16 + 8 * (payload.size() + 4) + 6)) / double(rate_params.dbps));
   int num_data_bits = num_symbols * rate_params.dbps;
   int num_data_bytes = num_data_bits / 8;
   int num_pad_bits = num_data_bits - (16 + 8 * (payload.size() + 4) + 6    std::vector<unsigned char> data(num_data_bytes+1, 0);
  memcpy(&data[0], &service, 2);
   memcpy(&data[2], payload.data(), payload.size());
  boost::crc_32_type crc;
   crc.process_bytes(&data[0], 2 + payload.size());
   unsigned int calculated_crc = crc.checksum();
   memcpy(&data[2 + payload.size()], &calculated_crc, 4);
  std::vector<unsigned char> scrambled(num_data_bytes+1, 0);
   int state = 93, feedback = 0;
   for(int x = 0; x < num_data_bytes; x++)
   {
      feedback = (!!(state & 64)) ^ (!!(state & 8));
      scrambled[x] = feedback ^ data[x];
      state = ((state << 1) & 0x7E) | feedback;
   }
   data.swap(scrambled);
std::vector<unsigned char> data_encoded(num_data_bits * 2, 0);
   viterbi v;
   v.conv_encode(&data[0], data_encoded.data(), num_data_bits-6);
   std::vector<unsigned char> data_punctured = puncturer::puncture(data_encoded, rate_params);
  interleaver inter;
   std::vector<unsigned char> data_interleaved = inter.interleave(data_punctured);
   std::vector<std::complex<double>> data_modulated = modulator::modulate(data_interleaved, rate);
  return data_modulated;
}


Verbose comments, liberal line breaks


// Encode the data portion of a PLCP frame given a rate, payload, and service field
std::vector<std::complex<double>> plcp_data::encode(Rate rate, std::vector<unsigned char> payload, unsigned short service)
{
   // Get the parameters associated with the given rate
   RateParams rate_params = RateParams(rate);

   // Calculate the number of symbols
   int num_symbols = std::ceil(
           double((16 /* service */ + 8 * (payload.size() + 4 /* CRC */) + 6 /* tail */)) /
           double(rate_params.dbps));

   // Calculate the number of data bits/bytes (including padding bits)
   int num_data_bits = num_symbols * rate_params.dbps;
   int num_data_bytes = num_data_bits / 8;

   // Calculate the number of padding bits (past the 6 tail bits)
   int num_pad_bits = num_data_bits - (16 /* service */ + 8 * (payload.size() + 4 /* CRC */) + 6 /* tail */);

   // Concatenate the service and payload
   std::vector<unsigned char> data(num_data_bytes+1, 0);
   memcpy(&data[0], &service, 2);
   memcpy(&data[2], payload.data(), payload.size());

   // Calcualate and append the CRC
   boost::crc_32_type crc;
   crc.process_bytes(&data[0], 2 + payload.size());
   unsigned int calculated_crc = crc.checksum();
   memcpy(&data[2 + payload.size()], &calculated_crc, 4);

   // Scramble (whiten) the data
   std::vector<unsigned char> scrambled(num_data_bytes+1, 0);
   int state = 93, feedback = 0;
   for(int x = 0; x < num_data_bytes; x++)
   {
      feedback = (!!(state & 64)) ^ (!!(state & 8));
      scrambled[x] = feedback ^ data[x];
      state = ((state << 1) & 0x7E) | feedback;
   }
   data.swap(scrambled);

   // Convolutionally encode the data
   std::vector<unsigned char> data_encoded(num_data_bits * 2, 0);
   viterbi v;
   v.conv_encode(&data[0], data_encoded.data(), num_data_bits-6);

   // Puncture the data
   std::vector<unsigned char> data_punctured = puncturer::puncture(data_encoded, rate_params);

   // Interleave the data
   interleaver inter;
   std::vector<unsigned char> data_interleaved = inter.interleave(data_punctured);

   // Modulate the data
   std::vector<std::complex<double>> data_modulated = modulator::modulate(data_interleaved, rate);
   
return data_modulated;
}

A good engineer can parse either function and figure out what's going on, and that makes sense, it's kinda what we do. The thing is, it takes at least an order of magnitude longer to parse and understand the uncommented version.

Programmers often argue that comments are redundant, but as projects grow in size and complexity, comments are like the notes you use to cram before a big exam. In a profession that values efficiency, it makes sense to comment early and often.

Difficult to decipher code might look cool, and obfuscated programming competitions reveal some pretty fucking rad language features. Unfortunately, we live in the real world, and in the real world we need to cater to every engineer who might look at your code in the future. Don't be a dick. Comment your code.
Myth #3 - Naming conventions matter


This is really a half-myth. In the ever raging battle between PascalCase, camelCase, underscore_case, I_LOVE_LAMP_CASE and the like, there will never be a winner. It is a pointless war. Everybody has a preference on naming conventions, and some languages overwhelmingly prefer one style over the other. The only thing that matters is consistency.


  • If you are working on an existing project, for the love of science, follow the conventions already in place.
  • If you are starting a new project, figure out what naming conventions you want to use, and stick with them.
  • If you are writing a project in a language with a style guide, it is good to follow the naming conventions defined in it. It might not matter in the scope of your project, but six months from now, your company is going to hire a C# veteran. This veteran is going to use camelCase and fuck up the underscore_case you lobbied so hard for. You are going to get sad, and it is better to be happy. Be happy.

Myth #4 - Using third party libraries is bad

Engineers like building things and can be hesitant to use others' code. I love the sense of accomplishment of implementing a complex algorithm in an efficient way. It feels great, and I think, "golly gee, I must be a good engineer, I implemented this complex mathsomethingorotherfunction(...) in only two days! Go me! I can do hard things!"

The thing is, most of the time, had I Googled around a bit, I would have found an MIT/BSD licensed library to do exactly what I needed and more.  If somebody else has already written the code you need, and it satisfies the licensing requirements of your current project, use it. As much fun as it is to have complete ownership over your code, you could have spent those two days building something new and original instead.

Get to the point already...

I could spend all day ranting about what not to do, but you would probably just get bored and go back to Reddit or Hacker News or Slashdot or whatever it is that you do at work, so it is about time to get to the point. If you take away one thing from this blog post, it is this:

Write your code so a freshman computer science student can read it. Keep it simple and don't over-complicate things.

The enlightened programmer will leave his or her ego at the door, write as little, well commented code as possible, and be more productive for it.

Write beautiful code.