Low Latency FPV Streaming with the Raspberry Pi

One of the biggest challenges in running my local quadcopter racing group has been overlapping video channels. The manual for the transmitters list a few dozen channels, but in practice, only four or five can be used at the same time without interference.

The transmitters being used right now are analog. Digital video streams could make much more efficient use of the spectrum, but this can introduce latency. Of late, I’ve been noticing a few posts around /r/raspberry_pi about how to do an FPV stream with an RPi, and I’ve been doing some experiments along these lines, so I thought it was a good time to share my progress.

It’s tempting to jump right into HD resolutions. Forget about it; it’s too many pixels. Fortunately, since we’re comparing to analog FPV gear, we don’t need that many pixels to be competitive. The Fatshark Dominator V3s are only 800×600, and that’s a $600 set of goggles.

You’ll want to disable wireless power management. This will tend to take the wireless interface up and down a lot, introducing a delay each time. It’s not saving you that much power; consider that an RPi takes maybe 5 watts, while on a 250-sized quadcopter, the motors can easily take 100 watts or more each. So shut that off by adding this anywhere in /etc/network/interfaces:

And reboot. That should take care of that. Check the output of iwconfig to be sure. You should see a line that says “Power Management:off”.

You’ll want to install GStreamer 1.0 with the rpicamsrc plugin. This lets you take the images directly off the RPi camera module, without having to use shell pipes to have raspivid to go into GStreamer, which would introduce extra lag.

With GStreamer and its myriads of plugins installed, you can start this up on the machine that will show the video:

This will listen on UDP port 5000, waiting for an RTSP h.264 stream to come in, and then automatically display it by whatever means works for your system.

Now start this on the RPi:

Modify that last line to have the IP address of the machine that’s set to display the stream. This starts grabbing 640×480 frames off the camera with h.264 encoding, wraps them up in the RTSP protocol, and sends them out.

On a wireless network with decent signal and OK ping times (80ms average over 100 pings), I measured about 100ms of video lag. I measured that by displaying a stop watch on my screen, and then pointing the camera at that and taking a screenshot:

rpi_cam_latency

This was using a RPi Model 2, decoding on a fairly modest AMD A8-6410 laptop.

I’d like to tweak that down to the 50-75ms range. If you’re willing to drop some security, you could probably bring down the lag a bit by using an open WiFi network.

I’ll be putting together some estimates of bandwidth usage in another post, but suffice it to say, a 640×480@30fps stream comes in under 2Mbps with decent quality. There will be some overhead on that for things like frame and protocol headers, but that suggests a 54Mbps wireless connection will take over 10 people no problem, and that’s on just one WiFi channel.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Tagged , , , . Bookmark the permalink.

9 Responses to Low Latency FPV Streaming with the Raspberry Pi

  1. Olav Strehl says:

    Hi, thanks for point the possibility to use the Pi with gstreamer for video streaming out. I just tried it myself and the first results looks interesting.

    I just ordered 2 CSL 300Mbit USB WLAN adapters to perform some outdoors tests. Have you done any further research how the latency can be reduced further?

    Best regards from Germany

    Olav

  2. Jose Antonio says:

    In which version of gstreamer are you working? I’m currently using gstreamer1.0 version 1.4.4 and I don’t know if that will work there

  3. Mr.Holub says:

    Hello,

    Thank you for the article, it contains very valuable information!
    I’ve set up streaming as described in this article and it works perfectly (I use self-made RC rogallo with raspberry pi on it).
    Only issue is the signal loss during the flight depending on position etc….
    I use USB wifi adapters on both sides with detachable antennas.
    What kind of antennas/Wifi adapters do you use for your drone ?

    Thanks in advance for any info,

    Mr.Holub

  4. Chad says:

    I would look into Ubiquiti wireless gear. I have and am getting ready to test a very similar setup using bulletM5 NAnoStation NSM5 12-24vdc to power output adjustable up to 1watt come either 2.4ghz or 5.8ghz I have done ground based testing with stock antennas and travel about 1 mile from the house with no issue. Keep in mind this is ground testing so Im anxious to get one in the air to see, they claim 50k(@30miles) point to point and I think 100mbps throughput so for video and telemetry should be great and stripped of cases.

  5. Kristian taylor says:

    any more progress on this?

    • Timm Murray says:

      Afraid not. I took some stabs at it–multicasting, playing with bitrate and framerate, couple others–but nothing held. It either increased framerate or stayed about the same.

      Short of something more drastic, it looks like ~100ms is about the limit on the Rpi.

  6. Thanks for this write-up, Timm. I was looking into using a NanoPi Neo AIR, using the in-built h.264 hardware, as a Digital video transmitter until realizing it only has a 2.4Ghz transmitter.

    Then the latency question reared it’s head!

  7. David says:

    Thanks for this post, this got my Pi flying and streaming. I just wish there were more user friendly gstreamer clients for Android and/or Windows that supported side by side video for VR

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">