Hacker|Engineer

Object-oriented programming (OOP) is practically gospel in some parts of the programming world today. At its most basic, it means attaching functionality to particular pieces of data. For example, if you have a bunch of information about a person, you might want to group that together and attach some person-specific functionality to it, say, the ability to send a letter to the person. Different pieces of data can represent different kinds of things, so it makes sense to couple functionality with “type.” Ruby is a great object-oriented language (my favorite), and embodies OOP quite well in my opinion. Everything in the Ruby universe is an Object, and things are instances of particular Classes (which are themselves Objects). Every object in the Ruby world has a particular set of operations that you can perform on it, and it usually contains some kind of data. Encapsulation, Polymorphism, and other Good Things arise out of the object-oriented paradigm.

On the other end of the spectrum, we have Lisp. In a Lisp, everything is data, including code. At its most basic, Lisp supports very few kinds of data, with lists being the only way to group data. Clojure (my favorite Lisp) supports a few more types, such as maps and sets, but thanks to the Sequences API, they appear fairly homogenous. Instead of coupling data with functionality with an object system, Lisp treats everything the same: many functions can act on just about any chunk of data. This way of thinking completely separates functionality and data, in stark contrast toOOP. In the Lisp way of doing things, you build general-purpose functions that you compose together and then operate on homogenous data with. You lose Encapsulation, but Polymorphism isn’t even a concept since types have mostly disappeared.

In the process of writing Computer.Build, I’ve used both paradigms to varying degrees of success. In the Ruby implementation, I build an abstract syntax tree (AST) up using a block-based internal DSL. I then ask this tree to generate itself into VHDL, and it recursively calls generate() on each node. Different nodes have different types, allowing them to generate different VHDL. In Clojure, on the other hand, you create the AST directly using a list literal, so you’ve got this giant homogenous data structure. Without an object system, how do you generate different code based on what kinds of nodes you encounter in the AST? With a dispatch function, of course! Clojure (and many other Lisps) supports a concept of “multimethods,” which are sets of functions that are dispatched to based on another function. This allows me to write a dispatch function that analyzes a particular node in the AST and decides what kind of type it is (generally by getting the unqualified name of the first element). The result of the dispatch function is then used to choose one of many different VHDL-generating functions to run against the particular node, some of which recurse through this multimethod again.

Since I started in Java, I still tend to think in an OOP style. This means the Ruby implementation of Computer.Build should be considerably easier for me, but it turns out I’m gravitating more toward the Clojure. It’s just so…clean! The syntax is terse, the structure is pure, and the multimethod-based dispatch seems so much clearer than fancy inheritance dances. As indoctrinated with the OOP style as I am, I’m starting to feel like a data-oriented style (a la Lisp) is better for some applications. When you’re trying to represent real-world objects, then yes, a type system that allows you to represent classes of things like people and trees and whatnot makes sense. Once you get into the abstract, esoteric world of software, though, the OO metaphor isn’t quite as simple. Heavy OO users tend to end up with huge, complex hierarchies of types that can be pretty confusing. If you switch to a data-oriented approach, though, you can focus on exactly what you need to get done. I can represent the AST for Computer.Build as a homogenous list, and then act on each node based on some arbitrary analysis of it. I don’t have to worry about what type things are or where they inherit from; I just dispatch and go.

I’m a firm believer that different problems require different tools to solve, and programming style is no different. At work, I’m encountering asynchronous, evented I/O for the first time, and it’s a great tool for some applications. Some problems are going to lend themselves to an OOP style, while others are going to fit a more data-oriented approach. With Computer.Build, I’m leaning toward favoring the data-oriented approach.

For a long time now I’ve had a habit of railing about “incompetence.” People who were “incompetent” made no sense to me. How could you consistently underperform at a task without improving? Last week, I realized I’ve been thinking about it entirely the wrong way. It’s not incompetence per se that upsets me, it’s carelessness. People who don’t care about things are completely inscrutable to me. How do you live if you don’t care? What gets you up in the morning if things don’t matter to you? Passion is what drives me day in and day out. Doesn’t it drive everybody?

I don’t think I’m alone in the software world when I say that. Most of the great developers I know are driven by passion, and when I asked on OnStartups Answers “What drives you?” a lot of people came back with passion. It seems to be a common theme, from Getting Real to education, that people who are passionate are more successful. My confusion of carelessness with incompetence wasn’t completely wrong: people who are passionate improve over time. People who aren’t can be comfortable staying at the same level of ability without significant improvement. Over time the two groups diverge, with the passionate people getting better and better.

I think software engineering runs the risk of losing the passionate people. What I love most about the Ruby community is its consistent theme of passion and experimentation. We have great characters like _why (or hadhim) and fun frameworks like Sinatra, where the web server tells you that “Sinatra has left the stage (applause).” Ruby people are clearly passionate about what they do, and that makes me feel like I’ve found my people. Passion drives me, and passionate people make sense to me. No wonder Ruby has so quickly become my favorite language to code in.

It’s hard to get passionate about something you’re no good at. Every time I’ve gone into a new field and tried to learn it, there’s a period I call bootstrapping. I know so little about the topic that I struggle even to figure out what I should be learning, much less actually learning it. It’s often not very fun, and I’m rarely that passionate about it. Getting over that hump, though, is so worth it. When you’re just starting out, you may not be passionate, but you’re not going to get there without caring.

I’m in the process of re-starting the RPI Entrepreneurship Club over the next several months. Creating an organization is difficult, but I’ve come to understand there are two (and probably more) very different ways of going about it. One kind of leader focuses on building the infrastructure for the organization to function well, while the other focuses primarily on the people. Of course, both are necessary for a strong organization, but the different emphases can have a significant impact on the organization’s growth.

When you start with tools, you get a less intimate organization. When people interact through tools, they have different feelings about the other individuals in their organization. The other people are simply text on the screen, as rendered by the tool. Tools allow administrators to enforce policy and lubricate processes, but they don’t inherently encourage interpersonal interaction. An organization’s people are often its greatest asset, and they need to be brought together as much as possible.

A leader who focuses on people first is taking a risk. When you focus on the people, you face a couple of dangers. First, you can misstep and create a negative culture. Second, you can focus too little on the other concerns, such as tools, and end up with a bunch of people getting nothing done. When played correctly, though, a people focus can create a tight-knit, efficient team that accomplishes great things with ease. Once the people are working together well, the tools fall into place. A strong team with good people is the basis of a strong organization, and I believe that a focus on people from the start creates the best kind of organization.

Leadership is hard, and I’m nowhere near good at it. After seven years of trying, I’ve finally gotten to the point where I don’t screw it all up. I think I have an idea at this point of what the right thing is, and I’m striving this semester to work toward effective teams. I believe that the people are far more important to a team than its tools, and focusing too much on tools early on will hurt the team.

Getting it right is easy. What you do when things go wrong shows real talent.

I was in lab last week with my two lab partners working on a Computer Hardware Design project. They’re two of the top students in my department, seniors, and they even hold down part-time consulting gigs during school. By virtually every measure available, these guys are amazingly good.

Yet they can’t debug. We hit a number of snags during our lab, and they just started freaking out. Was the oscilloscope bad? It must be, let’s run calibration. Try the circuit on another ‘scope. Different results. Curse some more at the equipment. Rip out half the circuit and replace it with a pulse generator (which caused its own family of problems). Blame the “bad” equipment some more. Eventually, we went home with no results.

These guys are used to getting it right the first time, and that cockiness hindered them when things went wrong. They’re fast, sure, when they already know how to do it and nothing goes wrong, but how often does that happen? Frankly, if I were a hiring manager, I would be hard pressed to hire either one. From what I’ve seen, things rarely go according to plan.

How you handle things when they don’t go according to plan is when real talent emerges. Maintaining a calm manner, having patience, and methodically isolating the problem are critical to being a good developer of anything, especially software.

I’ve often been puzzled by why some hot new technologies or services take off, and others that are similar (or even better) fizzle. Two that come to mind are Arduino and Dropbox. I’ve been working with PIC microcontrollers for years, and I get my chips for free via Microchip’s sample program. Why, then, would anybody pay $30 each? It’s the same functionality, just packaged differently. And there’s the key. Dropbox? I’ve had secure, remote storage for a long time. But now I have a dead-simple way of doing it, and guess what? I use Dropbox far more. Neither of these technologies are new, but both have taken their markets by storm nonetheless.

As a geek, I tend to miss what makes products appeal to others. For this situation, though, I think I’ve cracked it. Many purchasing decisions, especially for free things, are no longer about price. They’re about barriers. The easier it is for me to start using the tool or service, the more likely I am to buy it. Price factors in, but it’s secondary. Both Arduino and Dropbox are pathetically easy to start using, so why not? Even with all of my PIC expertise, I’d rather just pull out and Arduino and get started. With more and more things clamoring for our attention in this century, anything that reduces the time I have to spend on the task is a godsend.

The little things, then, are what differentiate the great products from the truly exceptional. Companies who figure out the key little things to make their product truly easy to get started with and use will often win in market others say is impossible to enter. Apple has demonstrated an amazing ability to hone in on these differences and exploit them to enter markets like MP3 players and cell phones and completely dominate within a few short years. Their products are often technically inferior, but they’re so much easier for the average user that customers continue to stand in line to buy iPhones even today.

If you’re a consumer-facing business, these little things could make or break your entire venture. With “free” web software, it’s even more critical since there’s no price to try and compete on. I certainly don’t yet have the ability to identify these differences before the fact, but anybody who does is a huge asset to whatever company he/she works for.

Most engineers have physical limitations. A civil engineer has gravity, tension, compression, etc. An electrical engineer has Ohm’s law, Maxwell’s laws, and other laws of physics. From what I’ve learned, Chemical Engineering is almost all about limitations, since much of what you’re doing is dealing with the elements nature has given you. Software isn’t like that. Software Engineers don’t really have physical limits beyond compute power and memory, both of which are growing exponentially.

Without the limits of other engineering fields, Software Engineering is a class of its own. Our limits are mostly our own abilities, so the discipline spends much of its energy dealing with the limitations of human programmers. There’s always the dream that someday computers will program themselves based on natural-language instructions, but that reality is still far away. Some human limitations are easy to correct: syntax errors, obvious bugs, and the like. The more insidious problem, though, is technical debt.

As a piece of software builds up over time, small mistakes accumulate. Bugs are introduced that can’t be easily reproduced. Architectural decisions make seemingly arbitrary requests nearly impossible, and the features that do get written end up hacking around those decisions. This is no big deal in the moment, but as time goes on, more and more of these little things accumulate. If ignored, they build up at such a rate that they can sink an otherwise healthy business. Thus, we software engineers must be constantly vigilant for the accumulation of technical debt, and management needs to be made aware of the issue.

I call it “technical debt” because this mass of mistakes ends up behaving much like financial debt. Like a teenager swiping a credit card without a second thought, some programmers never realize they’re creating it. After it has been created, it constantly takes time away from new features by forcing programmers to go back and correct or work around it. This diversion of resources can eat a big chunk of developer time if left ignored, much like “just paying the bills” is a struggle for people. Finally, paying technical debt off can be a huge challenge, because you must divert resources away from moving forward into what seems like a black hole. I see so many parallels between financial debt and technical debt that I would like to see a line item on financial statements for the technical portion of a software company’s debt.

Treating technical debt like financial debt gives business people a solid handle on how it works and what to do with it. If the engineering team can just provide reasonable estimates (and I think most programmers can do that), the executives of software companies will have a much better understanding of how to allocate resources. Why does it take you twice as long to get anything built this year compared to five years ago? Because we built up so much technical debt that we’re spending half our time dealing with it. Communication between business and engineering in the software world is notoriously bad, and introducing the concept of technical debt could go a long way towards facilitating that communication.

During our weekly team meeting yesterday, it occurred to me that many development teams could benefit from rotating developers through specific roles on a regular schedule. Bugs need to be fixed, but making one team member shoulder more of the bugfix burden is not healthy. Likewise, building new features instead of re-factoring old ones can be great fun, but everybody needs to share in the ups as well as the downs.

I’ve worked at places where the bug report list is so long nobody thinks we can ever get through it all. As such, it ends up being this intimidating mass of yuck that we all deliberately ignore, and bugs go unfixed. Since no member of the engineering team was specifically responsible, nobody did anything. I think a far more effective system would be to put the responsibility for fixing bugs on a single individual, but that’s not going to last very long. Therefore, I propose rotating bugfix duty every week, but every week somebody is responsible for only one thing: fixing bugs. With a system like that in place, I think you could get to zero major bugs fairly quickly, and then start tackling somewhat more fun user requests and such that come in through the ticketing system.

Other developers may disagree, but I find building new features by far the most fun and exciting part of software development. If I had my way, I’d just spend all my time building new features instead of updating old ones or ensuring consistency in the interface. When you’re adding features, you’re creating something from nothing. That’s cool. Every developer should get his or her chance in the sun, a time to dig in and build some cool stuff. Rather than sneaking features in between fixes and infrastructure work, feature-building time should be concentrated and rotated between developers, much like bug fixing. I’m not sure what the interval should be, but probably longer than a week. Of course, some features require far more work than one developer can do in a couple of weeks, but there are generally plenty of small features in the queue to work on.

User advocacy is critical to a well-functioning software development team. No matter who is doing it, somebody needs to be thinking like a user, talking to users, and generally immersing him or herself in how the users use the software and what their needs are. Too often this work is pushed off to non-technical, even management people, without giving developers a chance to connect directly with users. Sure, many of us are a bit rough around the edges (some are downright antisocial), but I believe communicating with users can be a growth experience for every developer. Rather than pushing user advocacy off to somebody else, let a developer do it for a week!

Behind everything is the infrastructure work: refactoring, optimizing performance, building out architecture, and maintaining software all fall under this umbrella. I think there is a wide variety of developer attitudes toward infrastructure, but much like the other three areas it should be rotated through the team. Some developers call themselves “backend” or “frontend” engineers, but really everybody should have a picture of how the whole system is put together. Without a common vision (including the infrastructure), how can you move forward efficiently?

Feature creation, bug fixing, user advocacy, and infrastructure work are all critical pieces of a functioning software development team. By asking developers to concentrate on one specific area at a time and rotating all developers through the focus areas, I believe a team can be better integrated. In the end, a well-integrated, efficient team can get the job done cheaper, better, and faster.