UAVs and FOSS — AR.Drone

This is the first in a series of articles about the state of FOSS software in terms of UAVs. We start with the place where I have the most experience, the Parrot AR.Drone.

On the client side, Parrot has released a complete SDK for controlling the AR.Drone. It includes support for the control protocol, the navigation protocol, and the video streams (which is handled by ffmpeg).

However, many people choose to make a clean implementation based on the SDK docs. This includes my own UAV::Pilot, and also NodeCopter, an implementation for Node.js. Both of these are under a BSD-style license. The parts of UAV::Pilot that interface to ffmpeg are LGPL.

Control is done over WiFi. It will create its own ad-hoc network by default. With some trickery, you can connect it to an open access point, though it doesn’t seem to support WPA authentication.

The on-board electronics for the AR.Drone has a Linux system with Busybox, which you can telnet into. This seems to be mostly useful for debugging the commands sent over the network, or configuring it to connect to an open access point. Hacking it to control external devices via USB seems like it should be possible, though I haven’t been able to find anybody who actually accomplished this. Just a lot of people talking about it. C’est la vie.

Version 2.0 of the AR.Drone includes front and bottom cameras, which can stream 720p video. They’re on par with cheap webcam quality.

As I’ve complained about before the AR.Drone’s SDK doc is rather poorly documented. The control protocol is mostly fine, but there are a lot of missing details about the nav data and video protocols. The information you need is spread out on the dev fourms, and it just shouldn’t be that way. I’m also dealing with the multiconfig options, which requires getting the ACK bit off the nav data stream, which seems all backwards to me even if it was well-documented. The config should have been handled over a TCP stream, especially when multiconfig was implemented.

It is relatively cheap, though, at $300. A basic ArduPilot setup will be around $700 if you include the telemetry module and a cheap 6-channel radio. That setup wouldn’t even have a camera.

All things said, I think the AR.Drone is a good way to get started considering what’s out there right now. But it could definitely be improved.

Musings on Hackable UAVs

Programming the AR.Drone has been a mostly fun challenge, and occasionally a frustrating challenge. The nav data is particularly under-documented, and UAV::Pilot still suffers from a few video parsing issues, probably because the documentation doesn’t fully explain the PaVE headers. But I pushed through them, figured it all out, and now there’s a release that I would consider close to feature-complete.

I never intended to stop with just the AR.Drone. It was a cheap way to get started–cost about $300 rather than $500-700 for a some other types–but it’s ultimately a toy. I don’t have a problem with big-boy toys; in fact, I own quite a few of them. But it’s a bit limited.

More “serious” UAV platforms, such as Ardupilot, often use some kind of mission planner software that allows you to put in a path of GPS coordinates and do something at the waypoints (like take pictures). With a GPS attachment and the right software, this is technically possible with the AR.Drone, but not out of the box.

The AR.Drone also has decent but somewhat limited hacking potential. If you’re willing to void your warranty, you can set on-board scripts to connect to AP-mode WiFi, add cheaper high-capacity batteries, or use the USB port to run attached devices. But it’s ultimately a closed platform with all the limitations that implies.

I find some of the other autopilots out there just as frustrating, for a somewhat opposite reason. They allow you to do anything, but their starting point is more sophisticated and need some work to dial them back down. The Ardupilot hardware, for instance, needs a compass, GPS, barometer, and some other assorted electronics. Not all this stuff is necessary for all uses. If you’re flying lower than 20-50 feet or so, the barometer isn’t much use and is too inexact. An ultrasonic range finder would be better for that case.

It’s all FOSS, so I’m sure you can get it all to work one way or another, but it isn’t designed for it.

By way of analogy, for years before the iPad, Microsoft had tried pushing tablets by taking their desktop OS and scaling it down to a tablet. They were largely ignored. What Apple (and later, Google) showed was that the correct strategy was to take a smartphone OS and scale it up.

That’s similar to the strategy I’d like to try with UAVs. If you want a toy UAV like the AR.Drone, you should be able to put it together cheaply without a GPS or barometer or anything. But if you want to get more serious, you should be able to add all that stuff without much trouble.

The platform should allow you to alter every aspect of the design, allowing the frame to be fully 3D-printable. Some printable UAV designs are already out there, such as the PL1Q Vampire, though often under a non-commercial license, which doesn’t meet widely accepted definitions of FOSS.

My goal is to build a UAV platform with the following requirements:

  • Complete design is under FOSS-compatible licenses, allowing modification for personal or commercial use. Licenses like BSD, Lesser GPL, or CC-BY[-SA] (but not CC-NC).
    • Software will probably be BSD, with CC-BY for hardware
  • On-board systems controlled by cheap and easily available computing platforms, such as Raspberry Pi, Arduino, and/or ATtiny.
  • Frame is fully 3D printable
  • Wide choice in attaching hardware; GPS, barometer, cameras, nerf guns, etc.
  • Wide choice in control methods; WiFi, XBee, cell phone towers, etc.
  • Complete documentation on control, navigation data, and accessories

I respect the work that’s already gone into projects like Ardupilot, but they don’t seem well suited for toy UAVs. And there’s nothing particularly wrong with toy UAVs; they are bound to have some rather un-toy-like uses, and even if not, there’s nothing wrong with a few toys. This looks like an open niche among the FOSS autopilots, which I intend to fill.

UAV::Pilot v0.5 Released, Now Supports Real Time Video

UAV::Pilot, a Perl library for controlling the Parrot AR.Drone, has released version 0.5.

This was the big one. With the ffmpeg library installed, you can decode the h.264 video stream coming right off the drone, in real time. This is where UAV::Pilot starts to get really interesting, opening up applications like augmented reality games and object recognition.

Demo video:

I’m going to be taking a short break from working on UAV::Pilot for a while. After lazing about, I’ll start work on the next release, which will mostly be cleaning up the API and bugs.

Also, I’ll be giving a presentation on UAV::Pilot in September for the Madison Perlmongers Group, which I plan on recording.

UAV::Pilot v0.4 Released, Now Supports Video

UAV::Pilot, a Perl library for controlling the Parrot AR.Drone, has released version 0.4.

The big change for this update is streaming video to a file. Display in real-time is not yet
implemented–that’s coming in 0.5.

There are also some API changes in how event loops are initialized to make the naming more consistent. There will probably be another API change in the video handling in 0.5. Once I do a complete release, API changes will have a proper deprecation plan, but for right now I think it’s fine to change things willy-nilly.

UAV::Pilot v0.3 Released

Version 0.3 of UAV::Pilot has been released on CPAN. The major change in this version is an improved event loop based on AnyEvent.

AnyEvent is a very nice framework overall, but unfortunately doesn’t have quite the right syntax for UAV::Pilot’s purposes. Consider this plain-English description of a UAV flight:

Takeoff, wait 5 seconds, then pitch forward for 2 seconds, then pitch backwards
for 2 seconds, then land

In AnyEvent’s standard interface, things would work out something like this:

my $timer; $timer = AnyEvent->timer(
    after => 5,
    cb => sub {
        $uav->pitch( -1.0 );
        my $timer2; $timer2 = AnyEvent->timer(
            after => 2, 
            cb => sub {
                $uav->pitch( 1.0 );
                my $timer3; $timer3 = AnyEvent->timer(
                    after => 2,
                    cb => sub {
                        $uav->land;
                        $cv->send;
                    },
                );
                $timer2;
            },
        );
        $timer;
    },
);

$uav->takeoff;
$cv->recv;

All that indentation gets hairy. So I created an interface on top of AnyEvent, UAV::Pilot::EasyEvent, which is based on the Node.js event interface used by NodeCopter. Here’s how you’d write the same problem using EasyEvent:

my $cv = AnyEvent->condvar;
my $event = UAV::Pilot::EasyEvent->new({
    condvar => $cv,
});

$event
    ->add_timer(
        duration => 2000,
        duration_units => $event->UNITS_MILLISECOND,
        cb => sub { $uav->pitch( -1.0 ) },
    )->add_timer(
        duration => 2000,
        duration_units => $event->UNITS_MILLISECOND,
        cb => sub { $uav->pitch( 1.0 ) },
    )->add_timer(
        duration => 2000,
        duration_units => $event->UNITS_MILLISECOND,
        cb => sub { 
            $uav->land;
            $cv->send;
        },
    );

$event->activate_events;
$uav->takeoff;
$cv->recv;

Much better.

As an added bonus, having a good event interface means we can also support controlling UAVs through SDL joysticks. See bin/uav_joysticks in the distribution for details.

Announcing: UAV::Pilot v0.2

UAV::Pilot version 0.2 is now up on CPAN. The big change in this one is initilizing the navigation stream and parsing the data. I also whipped up this retro-videogame looking graphical output:

nav_output_sdl

The usability of the nav data is a bit limited at this point, because we need to grab the stream continously in a loop for display while also processing the flight commands. This is where an event system like AnyEvent will come in handy. You can try programming it around AnyEvent (or whatever event system you like) with what’s there right now.

The major work for the v0.3 release will be getting an event loop integrated into the library.

AR.Drone Nav Data

There are parts of the Parrot AR.Drone that are rather underdocumented, and one of those is getting the navigation data.  Once you activate the nav data, it doesn’t send the data directly to you. Instead, it sends it to the 224.1.1.1 muticast address.

Which is great and all. That means you can have one person controlling on their laptop and a bunch of other people watching the nav data on their own systems. But it’d be nice if their docs told you this.

There’s also a way to get it to unicast to your address, but they certainly don’t lay that out.

Anyway, the good news is, I can receive the AR.Drone’s nav data now, which is a big milestone for the 0.2 release.

Announcing: UAV::Pilot v0.1

UAV::Pilot is a Perl library for controlling UAVs.  It currently works with the Parrot AR.Drone, with plans to expand to others in the future.

Demo video

The current library supports basic commands, such as takeoff, pitch, roll, yaw, vert speed, and land.  All the preprogrammed flight animations are also in place. Navigation data and video are not yet supported–see the ROADMAP file for future plans.

Github repository: https://github.com/frezik/UAV-Pilot

CPAN: https://metacpan.org/module/TMURRAY/UAV-Pilot-0.1/lib/UAV/Pilot.pm