What would you like to see from an ebook on programming the Raspberry Pi in Perl?

I guess the title says it all.

We’re well past the time when Perl should have an ebook about programming on the Raspberry Pi, so let’s fix that. I’d like to get a feel for what people will want to get out of this.

It will probably be based around Steve Bertrand’s series of RPi::* modules. We’ll expand from there into web projects (Device::WebIO::Dancer), capturing from the camera (GStreamer1), and interfacing with Arduino (Device::Firmata).

For i2c/SPI examples, we’ll show how to interface directly with some low-cost sensors. Tentatively, these will be an accelerometer for SPI (ADXL345) and a temperature sensor for i2c (MPL115A2). Both are available with breakout boards from Adafruit and Sparkfun.

Then there will be real project examples, such as a garage door opener and a temperature logger.

Here’s my chapter outline so far:

  • Introduction
  • RPi Models, Basic Setup, and Tools/Hardware You Need
  • GPIO [blink an LED, read from a switch, pullups/pulldowns]
  • SPI [ADXL345 accelerometer?]
  • i2c [MPL115A2 temperature sensor?]
  • Camera [GStreamer1]
  • Serial [GPS]
  • Expanding with Firmata
  • Interfacing from the Web [Device::WebIO]
  • PWM and Analog Input
  • Asynchronous code
  • Build Project: Garage Door Opener
  • Build Project: Temperature Logger

And there’s even a start on a cover:

ProgrammingThe Raspberry Pi

Nginx direct cachefile hosting, or I found a hammer, get me a nail

Let’s say you had an API that served JSON. Some of the responses don’t change very often. Perhaps something like user data:

It’s an obvious case where caching could help. Perhaps you stick the data in memcached and write it out directly from your app.

This means you’re still hitting application code. Wouldn’t it be nice if you could have nginx write the cached data back to the client as if it were a static file? This is possible using a ramdisk and nginx’s try_files.

Start with this little Mojolicious app:

This provides two paths to the same JSON. The first one, /ramdisk/*, will write the JSON to a path we specify under our nginx root. This has a deliberate sleep 5 call, which simulates the first request being very slow. The second, /direct/* is for benchmarking. It dumps some pre-encoded JSON back to the client, which gives us an upper limit on how fast we could go if we pulled that data out of memcached or something.

(If you use this code for anything, do note the security warning. The code as written here could allow an attacker to overwrite arbitrary files. You need to ensure the place you’re writing is underneath the subdirectory you expect. I didn’t want to clutter up the example too much with details, so this is left as an exercise to the reader.)

Save it as mojo.pl in a directory like this:

The html dir will be the place where nginx serves its static files. Create html/ramdisk and then mount a ramdisk there:

This will give you a 10MB ramdisk writable by all users. When the mojo app above is called with /ramdisk/foo, it will write the JSON to this ramdisk and return it.

Now for the nginx config. Using try_files, we first check if the URI is directly available. If so, nginx will return it verbatim. If not, we have it proxy to our mojo app.

Start this up and call http://localhost:8001/ramdisk/foo. If the file hadn’t been created yet, then that sleep from earlier will force it to take about 5 seconds to return a response. Once the file is created, the response should be nearly instant.

How “instant”? Very instant. Here’s the result from ab of calling this 100,000 times, with 100 concurrent requests (all on localhost):

And the results from calling the mojo app with /direct/foo:

We took 88ms down to just 5ms. This is on an Intel Core i7-6500 @ 2.5GHz.

If you’re wondering, I didn’t see any benefit to using sendfile or tcp_nopush in nginx. This may be because I’m doing everything over localhost.

What I like even more is that you don’t need any special tools to manipulate the cache. Unix provides everything you need. Want to see the contents? cat [file]. Want to clear a cache file? rm [file]. Want to set a local override? EDITOR-OF-CHOICE [file].

Now to go find a use for this.

Slicing 3D Objects for a Laser Cutter with OpenSCAD

Recently, my makerspace got a new laser cutter that can fit 4×2 foot plywood sheets. This means I can redo our arcade’s control panel with a design in OpenSCAD. Here’s the mockup:


The design is based on the old X-Men 6-player cabinet, though the final build got cut down to 4-players for space reasons. The red areas you see on the bottom are keepout sections for the structure underneath. There’s also a cutout in back, which is for a built-in keyboard. It’s 3/8 inch thick, and intended to be made from two layers of 1/8 inch plywood plus 1/8 inch acrylic on top.

Now, how do you go from an OpenSCAD file to a laser cutter?

In theory, you can have OpenSCAD generate an STL file, and then import that into a 3D printer slicer. A slicer’s job is to take a 3D model, break it into 2D slices, and then generate the printer head’s path for each slice. So you might think that you can set the slicer to have a layer height of 1/8 inch, and then have it output some kind of vector file format for each slice. Slic3r does indeed have an SVG output, which gives you hope.

Your hope will soon be dashed as you run into impedance mismatches. First, Slic3r wants the nozzle diameter to be larger than the layer height. We’re not going to be dealing with a physical nozzle here, but we have to set it to a ridiculous size (like 10mm) just to make Slic3r happy. Once we get past that, we get our SVG output in a single file with multiple layers. Slic3r’s SVG output is meant for resin 3D printers, and isn’t going to work well for laser cutters. What we want is one file per layer with simple paths, not filled-in.

It turns out we can hack our own slicer with OpenSCAD’s projection() command, combined with importing an STL. It works like this:

The translate() brings the object’s Z coord down by -0.1mm. The projection() command then creates a 2D slice of whatever intersects at Z axis zero. Note that you need the cut = true here. Without it, OpenSCAD creates the projection as a kind of shadow of the entire object, rather than whatever is on the Z axis.

That gets us a vector image like this:


If we loaded the code above into OpenSCAD, compiled it, and exported to a vector file like SVG or DXF, we would get the slice of the 0.1 layer. Then we’d have to do -0.1 - 3.175 (3.175mm is about 1/8 inch), compile and export, and then again for the third layer. This is too much manual effort, even for just the 3 layers I need for the control panel. I would never want to do it for something like a terrain piece.

Fortunately, OpenSCAD has a command line interface for compiling and exporting models. What we can do, then, is programmatically create the OpenSCAD projection code, run the command line, and get each layer in turn.

Which is exactly what Laser Slicer does. Here’s how to run it:

We put in a start height of 0.1 (you might get weird results if you try to start with zero), then go up 3.175mm each step, ending whenever we go above 7mm. It then generates 3 dxf files, one for each layer. It’s probably slower per layer than 3D printer slicers, but since the layers are thicker, it doesn’t need to generate as many of them.

All code on Github now.

Games::Chipmunk now at v0.5, can do useful things

Games::Chipmunk, the Perl bindings for the Chipmunk 2D graphics library, are now at version 0.5.

There are no changes to the bindings themselves in this version, but there is an example Pachniko program:


You click the mouse to drop a ball in that spot, and away it goes. Things seem to be at the point of doing real games with it.

SQL::Functional Cookbook–Updates

Updates are easy, too.

UPDATE takes a table to update, followed by SET and WHERE. In SET, we’re using the op function, which is actually an alias for match. Calling it op is more readable here, since we’re not matching anything. Note that the data there is still handled as placeholders.

The WHERE part works just like it does in SELECT.

SQL::Functional Cookbook: Inserts

We can build an insert statement easily, too.

INTO takes a table, which it feeds to INSERT. The INSERT function likewise takes a list of table columns (in an array ref), followed by VALUES, which itself takes an arrayref of the data you’re going to put into the columns. All the scalars in that arrayref are passed back in @sql_params and filled in as placeholders.

Inserting with a subselect is supported, which we will cover later.

SQL::Functional Cookbook: ANDs and ORs

It seems like it should be easy, but I was always disappointed with how other solutions handle arbitrarily nested ANDs and ORs. Most SQL creation libraries seem to start by adding support for a list of AND statements. At some point, the authors realize they need ORs, so they slap that in. Much later, they realize they need to mix ANDs and ORs, and then all sorts of convolutions get written.

With SQL::Functional‘s approach, nesting ANDs and ORs together is as natural as straight SQL. First, we’ll back up a few steps and demonstrate the ANDs:

Notice that unlike SQL, the AND is a prefix rather than infix. This might take some getting used to, but it does mean you can pass it an arbitrary number of statements:

In the final SQL, all of these will be joined together with AND. The OR statement works the same way:

If we need to mix the two together, we just do that:

Thus, the nesting falls naturally out of the system, just like it should be.

SQL::Functional Cookbook: Simple Select

SQL::Functional now has reasonably complete converge of common SQL statements. There’s always going to be something else to add, but the main thing it needs now is a cookbook of standard cases. Along those lines, I thought I’d start up a series of blog posts that can be compiled into a complete cookbook.

We’ll start with a basic SELECT statement.

One of SQL::Functional‘s strong points is in making easy jobs easy. Its other strong point, making hard jobs possible, will be covered later.

Let's break down what’s actually going on. The SELECT function takes a scalar for all the fields, so we have to pass it as an arrayref. FROM takes a list of tables, which we might do for joins, but we'll just pass it one for now. WHERE takes a list of clauses.

Ideally, we could say something like:

But that would require deep Perl voodoo (e.g. source code filters), so we content ourselves with the match function to build the same thing. It also helps us here by having the matched data passed as a placeholder.

In the end, $sql will contain:

With @sql_params containing the data for all the placeholders. We can run this through DBI like any other statement:


Callback Functions Between C and Perl

Everything you ever want to do with xs is documented somewhere in perlxs, perlguts, perlapi, perlxstypemap, and perlcall. Figuring out where it’s documented, and how it relates to everything else, is the hard part.

In my case, the part I wanted was in perlcall, in the section “Strategies for Storing Callback Context Information”. What I wanted to do was to glue together the callbacks for the Perl interface in Games::Chipmunk to the Chipmunk 2D physics library.

From Perl, we want to be able to say things like this:

The C version of cpBodySetVelocityUpdateFunc() does not know what a Perl sub is. It sees an SV, which happens to contain a bunch of things that the Perl interpreter can execute as a Perl sub. What we need to do is hand off a C function to the callback, and then use that to grab our SV and use call_sv() to call it.

If C supported closures, this would be easy. C does not support closures.

Some C libraries with callbacks have a parameter that will pass whatever data you give it directly into the callback later on. Chipmunk has this on a few functions, but not consistently.

One thing the Chipmunk libraries do give us is a pseduo-Object Oriented interface, where we pass the associated datastructure in as the first parameter on every function (Object Oriented languages move this parameter to the left of the function call). This gives us something we can grab on to for getting the SV we need that stores our Perl sub.

The “object” is a pointer, and pointers are just numbers. Numbers can be looked up in a hash. So we’ll make a bunch of global hashes, one for each set of callbacks, and use the address as the lookup key and the SV as the value.

The examples below use Perl’s context macros, which means those global hashes are still thread-safe. You can read about them in perlxs, under the section “Safely Storing Static Data in XS”. Their use won’t be detailed here.

Using cpBodySetVelocityUpdateFunc as an example, we start with the xs declaration:

We’re taking an SV* and assuming it holds a reference to the sub. We store it in the HV* bodyVelocityFuncs, which is initialized elsewhere. Using (char*)&body (C is a true bastion of type safety), we convert the address of the cpBody* into a char pointer, which the hash can use as a key. Lastly, we call the actual cpBodySetVelocityUpdateFunc() in the C library, and pass it our own C function as a callback.

That C function looks like this:

This goes at the top of your xs file, before any PACKAGE declarations, up with #include "EXTERN.h" and such. The perlxs doc doesn’t show this part in its examples very well, but you need to put the dTHX; call here to declare the context for a bunch of Perl macros that come later. Otherwise, you’ll get a bunch of cryptic compiler errors and spend a few hours scratching your head, until you finally come across the section in perlguts entitled “How multiple interpreters and concurrency are supported”. Like I said, everything is documented, you just won’t know where.

Going past the Perl macros, we get to hv_fetch(). This function should have gotten the same cpBody* as we got earlier, so we once again torture the type system and pretend it’s a pointer to a char array for the sake of a hash lookup key. The person who wrote this part of the interface was a two-star C programmer, so we better check that we actually got the SV we wanted by checking perl_func == (SV**) NULL.

Now we’ll need to grow the stack so we can push the subref’s arguments there. In this case, the callback received four arguments, and we want to pass all four to the subref. In this particular case, the last two arguments are easy. They’re double precision floats, which we can pass directly into a Perl SV that contains a number.

The first two are the tricky ones. They’re pointers to complex structs. In my typemap, I made cpBody* into a T_PTROBJ. This means it converts the ‘*’ into ‘Ptr’, collapses any whitespace, and uses the resulting name as the Perl class. You can make an SV contain a pointer to this C object and then carry it around in Perl. The cpVect* comes out the same way (since we take a pointer to the gravity struct).

Finally, we can call call_sv() to call the actual subref.

The good news is that Games::Chipmunk v0.3 has most of the callbacks implemented. There’s some in cpSpatialIndex that aren’t there because they don’t follow the same conventions. Still, this should be good enough to write real physics systems for games in Perl.

Adventures in Code Generation — Graphics::GVG

Vector graphic games, like Battlezone or Asteroids, are old favorites of mine, and I’ve been wanting to make a game with that same style. Partially, that’s because I’m not that artistic, and it’s easy to make the style look cool. Just make everything come together at hard angles and let it go.

I considered SVG for the job, and leaned towards it for a while just for the sake of not falling into Not Invented Here. The problem is that SVG is incredibly complicated, especially for rendering. In a game where I’d likely be writing my own rendering of the SVG standard, I just didn’t want to do it. What’s more, for any kind of complicated effect, I’d probably have to use CSS, which is a second really complicated standard to implement.

So I went off to do it myself. When I inevitably end up having to reimplement some feature of SVG, I’ll just live with that.

Anyway, this led me to make Graphics::GVG. It uses a simple scripting language (parsed by Marpa::R2) to define how to draw your vector art:

The drawing commands inside the glow { ... } block will be rendered with a glow effect. Exactly what that means is up to the renderer.

There’s an included OpenGL renderer, which is where the real fun starts. The script above would be parsed into an Abstract Syntax Tree (AST), and then renderers compile that into the system of their choice. In the OpenGL case, it compiles the AST into a Perl package, which has a draw() method that does a series of OpenGL functions.

For example, a simple rectangle GVG script:

Gets turned into the Perl code below by the OpenGL renderer:

Which isn’t going to win any formatting awards, but it’s not meant to be edited by humans, anyway.

The above gets returned as a string, so you can compile it right into the running program using eval(STRING) (see Dynamic Code Loading for why you shouldn’t be afraid of this sort of thing). Alternatively, you could save it as a .pm file and load it up that way.

Either way, you get yourself an object from that package with new(), and then call draw() on it for each frame.

The generated code could be improved, for certain. For performance, it’ll probably move to vertex buffers. There should also be a way to make predictable package names rather than the UUID. If the overhead of calling all those OpenGL functions ends up being an issue, it could be compiled to a C function that can be called from XS.

In the future, there will be other renderers, which I hope can combine to output one set of package code. Meaning you would call $obj->draw_opengl for the OpenGL renderer, or $obj->init_chipmunk to setup the geometry for the Chipmunk2D physics library.