Low Latency FPV Streaming with the Raspberry Pi

One of the biggest challenges in running my local quadcopter racing group has been overlapping video channels. The manual for the transmitters list a few dozen channels, but in practice, only four or five can be used at the same time without interference.

The transmitters being used right now are analog. Digital video streams could make much more efficient use of the spectrum, but this can introduce latency. Of late, I’ve been noticing a few posts around /r/raspberry_pi about how to do an FPV stream with an RPi, and I’ve been doing some experiments along these lines, so I thought it was a good time to share my progress.

It’s tempting to jump right into HD resolutions. Forget about it; it’s too many pixels. Fortunately, since we’re comparing to analog FPV gear, we don’t need that many pixels to be competitive. The Fatshark Dominator V3s are only 800×600, and that’s a $600 set of goggles.

You’ll want to disable wireless power management. This will tend to take the wireless interface up and down a lot, introducing a delay each time. It’s not saving you that much power; consider that an RPi takes maybe 5 watts, while on a 250-sized quadcopter, the motors can easily take 100 watts or more each. So shut that off by adding this anywhere in /etc/network/interfaces:

And reboot. That should take care of that. Check the output of iwconfig to be sure. You should see a line that says “Power Management:off”.

You’ll want to install GStreamer 1.0 with the rpicamsrc plugin. This lets you take the images directly off the RPi camera module, without having to use shell pipes to have raspivid to go into GStreamer, which would introduce extra lag.

With GStreamer and its myriads of plugins installed, you can start this up on the machine that will show the video:

This will listen on UDP port 5000, waiting for an RTSP h.264 stream to come in, and then automatically display it by whatever means works for your system.

Now start this on the RPi:

Modify that last line to have the IP address of the machine that’s set to display the stream. This starts grabbing 640×480 frames off the camera with h.264 encoding, wraps them up in the RTSP protocol, and sends them out.

On a wireless network with decent signal and OK ping times (80ms average over 100 pings), I measured about 100ms of video lag. I measured that by displaying a stop watch on my screen, and then pointing the camera at that and taking a screenshot:

rpi_cam_latency

This was using a RPi Model 2, decoding on a fairly modest AMD A8-6410 laptop.

I’d like to tweak that down to the 50-75ms range. If you’re willing to drop some security, you could probably bring down the lag a bit by using an open WiFi network.

I’ll be putting together some estimates of bandwidth usage in another post, but suffice it to say, a 640×480@30fps stream comes in under 2Mbps with decent quality. There will be some overhead on that for things like frame and protocol headers, but that suggests a 54Mbps wireless connection will take over 10 people no problem, and that’s on just one WiFi channel.

Keep WiFi Alive on Raspberry Pi

So solving my connection problems to a Pi by turning off power savings didn’t work. I’m now using a crude but effective method where the Pi pings the gateway every 2 minutes.

Start by adding this script to /etc/keepalive.sh:

This script works in an infinite loop, sending a single ping every 120 seconds. Call chmod 755 /etc/keepalive.sh to make it executable. Next, call it from /etc/rc.local:

Which will run the script on the next reboot. Call /etc/keepalive.sh & in a shell to start it running, and you’re done.

So far, this seems to make ssh connections start immediately every time. I’d like to figure out what the actual problem is, but this will do.

Fix for Unresponsive Raspberry Pi?

After letting a Raspberry Pi server sit for a while, I would often try to login over ssh and get a “Read from socket failed: Connection reset by peer” message. Pinging it would also fail, until about 20 seconds later. Then everything was fine.

(If this happens to you, the first thing you should check is your power adapter. If you grabbed an old cellphone charger, it might not be good enough. I like using the CanaKit 1A chargers.)

I suspect this is due to the power saving mode on USB (it’s a USB WiFi adapter, and I believe the Pi’s built-in ethernet is also handled over USB). Sure enough:

power/control . . . The default for all devices is “auto”, which means that they may be subject to automatic power management, depending on their drivers.

I set this to /etc/udev/rules.d/50-usb_power_save.rules:

Which will stop the kernel from signalling the device to go into powersaving mode. If power is a concern, then you might want to add parameters to filter the specific USB vender ID for your USB device. Or just leave it on auto mode and put up with the delay.

Not 100% sure this is the real problem, but I'm hopeful.

Update: nope, that didn't do it.