Callback Functions Between C and Perl

Everything you ever want to do with xs is documented somewhere in perlxs, perlguts, perlapi, perlxstypemap, and perlcall. Figuring out where it’s documented, and how it relates to everything else, is the hard part.

In my case, the part I wanted was in perlcall, in the section “Strategies for Storing Callback Context Information”. What I wanted to do was to glue together the callbacks for the Perl interface in Games::Chipmunk to the Chipmunk 2D physics library.

From Perl, we want to be able to say things like this:

The C version of cpBodySetVelocityUpdateFunc() does not know what a Perl sub is. It sees an SV, which happens to contain a bunch of things that the Perl interpreter can execute as a Perl sub. What we need to do is hand off a C function to the callback, and then use that to grab our SV and use call_sv() to call it.

If C supported closures, this would be easy. C does not support closures.

Some C libraries with callbacks have a parameter that will pass whatever data you give it directly into the callback later on. Chipmunk has this on a few functions, but not consistently.

One thing the Chipmunk libraries do give us is a pseduo-Object Oriented interface, where we pass the associated datastructure in as the first parameter on every function (Object Oriented languages move this parameter to the left of the function call). This gives us something we can grab on to for getting the SV we need that stores our Perl sub.

The “object” is a pointer, and pointers are just numbers. Numbers can be looked up in a hash. So we’ll make a bunch of global hashes, one for each set of callbacks, and use the address as the lookup key and the SV as the value.

The examples below use Perl’s context macros, which means those global hashes are still thread-safe. You can read about them in perlxs, under the section “Safely Storing Static Data in XS”. Their use won’t be detailed here.

Using cpBodySetVelocityUpdateFunc as an example, we start with the xs declaration:

We’re taking an SV* and assuming it holds a reference to the sub. We store it in the HV* bodyVelocityFuncs, which is initialized elsewhere. Using (char*)&body (C is a true bastion of type safety), we convert the address of the cpBody* into a char pointer, which the hash can use as a key. Lastly, we call the actual cpBodySetVelocityUpdateFunc() in the C library, and pass it our own C function as a callback.

That C function looks like this:

This goes at the top of your xs file, before any PACKAGE declarations, up with #include "EXTERN.h" and such. The perlxs doc doesn’t show this part in its examples very well, but you need to put the dTHX; call here to declare the context for a bunch of Perl macros that come later. Otherwise, you’ll get a bunch of cryptic compiler errors and spend a few hours scratching your head, until you finally come across the section in perlguts entitled “How multiple interpreters and concurrency are supported”. Like I said, everything is documented, you just won’t know where.

Going past the Perl macros, we get to hv_fetch(). This function should have gotten the same cpBody* as we got earlier, so we once again torture the type system and pretend it’s a pointer to a char array for the sake of a hash lookup key. The person who wrote this part of the interface was a two-star C programmer, so we better check that we actually got the SV we wanted by checking perl_func == (SV**) NULL.

Now we’ll need to grow the stack so we can push the subref’s arguments there. In this case, the callback received four arguments, and we want to pass all four to the subref. In this particular case, the last two arguments are easy. They’re double precision floats, which we can pass directly into a Perl SV that contains a number.

The first two are the tricky ones. They’re pointers to complex structs. In my typemap, I made cpBody* into a T_PTROBJ. This means it converts the ‘*’ into ‘Ptr’, collapses any whitespace, and uses the resulting name as the Perl class. You can make an SV contain a pointer to this C object and then carry it around in Perl. The cpVect* comes out the same way (since we take a pointer to the gravity struct).

Finally, we can call call_sv() to call the actual subref.

The good news is that Games::Chipmunk v0.3 has most of the callbacks implemented. There’s some in cpSpatialIndex that aren’t there because they don’t follow the same conventions. Still, this should be good enough to write real physics systems for games in Perl.

Welcome, Readers

The Wumpus Cave is‘s site of the day! Here’s a few links to old entries that y’all might be interested in:

Happy reading!

Perl Advocacy Fail

Guy comes by Perlmonks wondering why his Perl program is so slow to start on a Raspberry Pi. Muses that Perl may be inappropriate for small platforms like this, and that perhaps the program should be rewritten in C. Monks get salty at the thought.

So great, now that guy probably won’t be back.

Now, the program in question was I/O bound, particularly on the Astro::Sunrise module. The initial thought of many Monks was that rewriting in C would not help, but that’s not obvious. Loading the modules involved here is a big task that would be built into a single binary for a C program, plus maybe some shared libraries that will likely be loaded up anyway at boot time.

Even so, there are better ways of helping here. The program uses threads and Switch, which are both probably unnecessary. Using threads in particular is a big performance suck.

I also double-checked, and the default perl on Raspbian actually is compiled with threads. I’m sure that’s because the base Debian distro has to be compatible with any Perl script you throw at, but that’s a big, unnecessary performance suck for a little Rpi. I’ll have to check, but Hiveberry might have a more sensible compile of perl. It’s a more up-to-date 5.20, as well (Raspbian comes with 5.14). That could make for a nice performance boost.

Coding for 80 characters per line — it’s not just for old farts anymore

In a discussion on /r/coding, people once again debated the merits of the old 80 character per line rule. My usual argument is that we want to put several code windows next to each other, so yes, we do want to limit things to 80 characters. The author of the linked piece mentions this, but I don’t think he makes a persuasive counterargument.

I might have changed my mind as 4K monitors become standard, but then someone in the discussion linked this:

This suggests that you should limit things to 50-100 characters per line for typographical reasons. Now, they’re mostly talking about prose writing there rather than code, but lacking studies otherwise, setting the limit to 80-100 seems sensible no matter how big monitors get.

GStreamer1 and Device::WebIO::RaspberryPi

Previous versions of Device::WebIO::RaspberryPi grabbed still images from the camera by calling out to raspistill. Given the limitations of the Rpi, this meant it had to load a program off the SD card into main memory and execute.

Meanwhile, the GStreamer framework has a plugin to read from the Rpi camera on its own. Problem was, the existing GStreamer module on CPAN was compiled against the deprecated 0.10 API, and rpicamsrc wouldn’t work against it.

I ended up asking around about 1.0 API bindings on the gtk2-perl list, and they were very patient in walking me through how to create them using Glib::Object::Introspection. Creating the bindings themselves was easy; hard part was figuring out all the magic it did behind the scenes to link to the C libraries and build Perl classes out of them.

After getting all that worked out, I released GStreamer1 on CPAN (the version number in the module name follows convention from Gtk2). Short on its heels, Device::WebIO::RaspberryPi 0.006 was released, which uses GStreamer1 to grab camera data for still images.

This greatly improves the wait time for grabbing an image in Device::WebIO::RaspberryPi. It also neatly solves a problem I’ve been struggling with since building the WumpusRover, which is that it was hard to reliably get images off the Rpi camera via Perl. With better Gst bindings, I think this is finally nailed down.

Considering the Security of SSL Client Certs Versus HTTP Basic Auth

People often overlook this option, but SSL allows clients to have their own certificates for authentication. It’s similar to SSH key authentication, except because it’s SSL, it’s mind-numbingly complicated to setup. For optimal results, you’ll want to have one client cert for each desktop, laptop, tablet, etc. that you want to connect to the site.

Tablets and smartphones are particularly tricky, because they can be stolen so easily. If you have a client cert loaded, my Galaxy S5 forces you to use at least a pin code for unlocking the phone. Sensible, but also more awkward to use.

Given that I was working on an HTTPS site, I wondered how the security of a long, random password (using basic HTTP auth) would be compared to SSL.

Basic auth is transfered in plaintext. The HTTP protocol does support digest encryption for plaintext connections, but that’s unnecessary for SSL.

On the server side, basic auth passwords can be stored in encrypted form. Apache’s default htpasswd uses either MD5 or crypt(), neither of which is adequate.

What about the security of the authentication handshake? Consider that SSL initiates connections with public key crypto, but for performance reasons, it uses that connection to transmit a newly-created, random block cipher key. The server and client negotiate for the specific block cipher, but it’s probably going to be AES128, or maybe something else of around 128 bits.

Therefore, transmitting a password with 128 bits of entropy will be just as secure as AES128. That is, if the password were stronger than this, then an attacker would have an easier time attacking the block cipher rather than the password.

So what do you need to get to 128 bits of password entropy? It’s a function of how many characters are allowed in the password, and its total length. Since we’re talking about characters that can be typed on a keyboard (whichever kind is standard in your country–US for me), we aren’t using the complete space of an 8-bit byte. So we need to get out some math:

Where H is the bits of entropy, L is the password length, and N is the number of characters that you are allowing in your password.

Here’s 90 characters that can be typed out on a US-standard keyboard:

Run that through the formula, and you find that a 20 character password will get you about 130 bits of entropy–more than AES128. If you’re considering against AES256, then 40 characters will go to 260 bits.

Given that, I wonder if it’s even worth it to use SSL client auth over HTTPS. Apache’s password storage needs modernizing, but that can be handled with server modules.

Device::WebIO Family Version 0.002

A little while ago, I made a tentative release of Device::WebIO. That was a basic version so I didn’t talk much about it. I’ve now finished up what I wanted for v0.002, which gets things to where I want.

Device::WebIO provides a unified API for controlling all the wonderful System on a Chip hardware that’s been coming out of late. If you wanted to control one of these devices on Perl previously, you would have used HiPi for Raspberry Pi, Firmata for Arduino, or Device::PCDuino for PCDuino. Each of these has their own API. With Device::WebIO, you use a single interface with a driver for your specific device.

The project borrows from WebIOPi, a Python project which is specific to the Raspberry Pi. What I really liked about it was the REST interface, which lets you control the pins over a web browser. But I also wanted to branch out from just the Raspberry Pi, and hey, I also prefer Perl.

So I grabbed the REST interface and implemented in Device::WebIO::Dancer. It’s mostly compatible with WebIOPi’s interface, but I made a few changes where it tended to be too specific to the Raspberry Pi. For instance, Analog-to-Digital Converters need a voltref and a bit resolution. On the Rpi (or rather, expander boards for the Rpi, since it doesn’t have an ADC pin on its own), the voltref and resolution values are the same across all pins, but that’s not true on the PCDuino. The original interface didn’t specify such a pin, so I added a pin value to the call.

All such changes are documented in the pod for the REST API.