Hacker | Engineer

I love Ruby. As yet another Java convert, I enjoy Ruby’s expressiveness and simplicity, not to mention Rails. Lately I’ve been playing more with developing domain-specific languages in Ruby, and as Matz pointed out in his RubyConf keynote a couple of weeks ago, Ruby is great for building DSLs. As I start out on my next project, though, I’m beginning to understand why Lisp people go so crazy about macros. Basically, it’s all about the metaprogramming.

I see DSLs as 1st-order metaprogramming. You’re writing code that writes code, but you stop there. Rails does an amazing job of this, most notably in ActiveRecord relations (has_many, belongs_to, etc). As it turns out, many programming problems can be solved more elegantly with metaprogramming, and you generally only need one order of it. Instead of expressing the solution in basic language constructs, you go one level up and use a DSL. Not all problems, however, lend themselves to 1st-order metaprogramming. This is where macros come in: they’re nth-order metaprogramming.

Let’s say I want to design CPUs in FPGAs. I’m doing this for a class next semester, and the language we get to use, VHDL, is terrible at abstraction. So, I’m going to design a DSL for defining processors. Instead of working from the ground up, I’m going to start at the top and work down. Here’s what I’d like to write to build a simple little CPU.

Now, how am I going to get an FPGA from this? First off, I need to generate something more low-level that looks more like VHDL. Then, I’m going to need to translate that into actual VHDL so I can use an FPGA design tool like Quartus to get the end result. Now, how would I go about writing this? Well, I already started a bit, and the first step was a Ruby DSL for generating VHDL. But wait, I’ve got another DSL on top of that for the computer! Now what? Unfortunately, the backend for Ruby DSLs can get a bit messy, so trying to layer them is going to be a significant effort. Let’s take a step back here, though, and look at what I’m doing. What I really want here is 2nd-order metaprogramming. It can be done with two layers of 1st-order metaprogramming, but is there a better way?

Enter Lisp. Now, I’m not very experienced with Lisp yet, but that’s going to change now that I’ve recognized the significance. In a Lisp dialect, I could define my computer quite similarly to the Ruby above. Then, I use macros to decompose the “data” into something else closer to my goal. I can keep doing this until I’ve created a “data” structure that contains the code I want to execute, then I execute it! Layering the metaprogramming becomes a simple process of keeping track of the macros transforming the code, but they’re completely independent and loosely coupled. No more nasty DSL backend. No more struggling with code generation and parsing (How would you handle the “x+y” part of the above in a code-generating DSL?). This project is going to happen, but it’s going to happen in Clojure.

Lisp natively supports nth-order metaprogramming. In fact, from what I can tell, that’s pretty much the only thing it has going for it. The code/data duality of Lisp is somewhat akin to the wave/particle duality we encounter in physics. You can just accept it and move on, but to really understand things you must embrace the duality. This can be a big leap for a procedural programmer who tends toward languages like C, but I see it as the next step in my growth as a software developer. With nth-order metaprogramming, you can define the problem in code and work forward from there toward the solution, using as many layers of metaprogramming as you need. Often this is only one layer (thus Ruby’s awesomeness), but the flexibility to add another layer of metaprogramming when you need it can be well worth the learning curve required to pick up a language like Clojure.

I ran into a great article on Hacker News this week about passionate developers. I certainly consider myself an alpha geek, like him, but I still think I can find a job somewhere where I can be passionate about creating software and still love my job. Jonas, on the other hand, feels that a true alpha geek will never be happy with his (or her) day job, and that’s why we code on our off hours. Corporate culture is critical to keeping alpha geeks happy and productive. Alpha geeks, to use his term, are difficult to manage for a variety of reasons, most notably our disinterest in money. Keeping an army of them, much less one, happy and productive is a tough management task that most larger companies aren’t willing to undertake. That’s why I love small companies.

The passionate developer: I do like my profession, I don’t like my job

A friend of me recently asked why I use Textpattern for all three of my blogs. Most people don’t even host their own blogs these days, and the rest either use Wordpress or something they rolled from scratch. So why do I use an off-the-shelf blog engine that’s not Wordpress? Flexibility, speed, and security.

Textpattern isn’t really a blogging engine. It’s a CMS that can do blogs easily. For my personal site, I was migrating from hand-written HTML, and I wanted some flexibility to build a site that wasn’t just a blog. As it turned out, I couldn’t even keep the blog updated, much less the rest of the site, but I got addicted to the flexibility. Yes, you can make “real websites” with Wordpress, but there’s a point at which you’re working within the confines of your CMS. Textpattern’s Page/Form + Article model is amazingly flexible, and theming the whole thing with CSS is a cinch. If I want to tweak something, Textpattern’s tag-based markup language often makes it trivial. It would be hard for me to give up the flexibility of Textpattern, especially now that I’ve grown so used to it.

Have you ever noticed how Wordpress blogs always go down when they get Slasdotted or Dugg? From what I’ve heard (and seen), Wordpress is a mess inside. It’s more talky to the database than a poorly-written Rails view! It’s slow, monolithic, and…just gross. Ironically, my Textpattern blog was dreadfully slow for a few months until I figured out the comment system was hitting a defunct spam-checking site to weed out bad comments. Turning that off brought things back to blazing speed. Someday I might write something important, and I’d like for as many people to be able to read it as possible. Speed matters.

Script kiddies seem to love Wordpress. It seems like there’s not a month that goes by without another major Wordpress exploit. There are tools out there to completely automate the process of compromising a Wordpress blog and gaining full access to the underlying filesystem. Yikes! Yes, they’re usually pretty good about patching, but until very recently upgrading Wordpress was a bitch. Not only do I trust the Textpattern developers more, but so few people use it compared to Wordpress’s userbase that it’s not nearly as appealing a target. I like to sleep sound knowing my websites aren’t compromised.

Many of the reasons I chose Textpattern have become irrelevant or resolved, but now that I’ve gotten good at it I’m not about to switch. I like Textile, the flexibility is wonderful, and it Just Works. What more could you want from a CMS?