Friday, September 13, 2013

DARPA Spectrum Challenge Preliminary Competition: what worked, and what didn’t.

“The best way to have a good idea is to have lots of ideas, and throw away the bad ones.” - Linus Pauling

DARPA Spectrum Challenge

The DARPA Spectrum Challenge is a competition to develop software defined radios that can successfully function in a contested spectrum. There are two tournament types, each addressing a different problem. The competitive tournament pits two teams against each other in a race to transfer their data the fastest, with both teams operating in the same band. In the cooperative tournament, three teams occupying the same band attempt to transfer their data as quickly as they can without interfering with their peers.

The preliminary competition took place over the last two days at a DARPA facility in Arlington, VA, where the top 18 teams were invited to compete. This was the first chance the teams had to compare their radio designs, and will be followed up by a final competition in March of 2014. I placed 10th during the qualifying rounds earlier this year (team “wasabi”), so I had the opportunity to attend the preliminary competition and battle it out against some very ingenious radio designs. The majority of the teams (14) were from universities, and the remaining four were made up of individuals. I had the distinction of working solo on this project, and went into it without any prior software defined radio experience.

I used nearly the same strategies for both the cooperative and competitive tournaments. This was primarily due to time constraints, but also due to my lack of understanding of what types of strategies would work well. I performed very well on the cooperative side, but was easily defeated by wideband interference in the competitive matches. Having an intelligent and adaptable high level strategy will certainly be key to winning the final competition, but I believe that the success or failure of teams will be much more dependent on successful execution of their ideas. I quickly learned that implementing a cognitive radio that performs as you envision is an incredibly difficult programming task.

Soccer, or radio competition?

For both the competitive and cooperative tournaments, the goal was to transfer as much data as possible in as little time as possible. In my mind, all teams were going to transfer their entire 15000 packets, and the matches would be a race to the finish. This turned out not to be the case, as many teams opted for a wideband, high transmit power approach with a 100% duty cycle. When these teams went head to head, the matches were won or lost in the first few milliseconds. The winning team would get a packet or two across before their opponent started transmitting, at which point nobody could successfully transfer any data. There were many matches that ended up zero to zero and had to proceed to a tiebreaker against the house bot.

My radio design

I opted for a closed loop approach that worked well in the cooperative rounds, but proved to be too optimistic to do well in the competitive rounds. I used a channelized system with three independently modulated channels, each with adaptive forward error correction. The TX node transmitted packets in a half-duplex bursting configuration, and the RX node would then reply with information on received packets and channel quality information. Based on that feedback, the FEC and modulation was updated for optimal throughput.

What didn’t work

There were three major flaws in my design that hurt my performance in the competitive matches. The initial channel configuration was too optimistic, I had no open loop fallback, and in the event that the ACK packets weren’t received, I waited far too long to resume transmitting.

Each channel started out using QPSK and very little FEC, where I should have started with BPSK and very high FEC. When no ACK packets are received, the TX node was designed to continue transmitting packets without changing the channel configuration, but I should have stepped up the FEC and stepped down the modulation. Finally, if the handshake didn’t complete, the RX node would automatically retry several times, which proved to be devastating in the competitive matches. Sufficient interference would block the ACK requests from reaching the RX node, and the TX node would spend a lot of time turned off (waiting for data from the RX node), providing my opponent with an open channel.

What worked

My behavior in the cooperative tournament was similar, with a few exceptions. I disabled all interference generation, turned down the antenna gain, increased the wait between bursts, and sent less ACK packets where possible. This worked well, and my software contributed to the two highest cooperative match scores.

When it worked, my closed loop design was very efficient at getting data across. It was interesting for me to watch my radio design go up against those from other teams, because it either worked very well or failed completely due to interference.

Future design ideas

I learned more about spectrum sharing strategies in the last two days than I did in the last six months of research and development. Watching the 17 other teams’ radios and discussing strategy with them provided invaluable insight into what works and what doesn’t.

I think the winner of the final round of competition will be a team that successfully implements a combination of spectrum sensing, feedback, and adaptation. This is not an easy task, but there are some brilliant minds working on it, and I expect to see huge improvement in all teams between now and March. There is a tradeoff between design complexity and robustness, and whoever finds that balance should do very well.

Unsung heros

All of the challenge teams have put in very long hours working on this project, but this is only possible thanks to the hard work of teams both at DARPA, and the ORBIT lab at Rutgers University. It’s refreshing to see that they’re working just as hard on this project as I am.