Thursday, July 1, 2010

Hacking is not a Dirty Word

I used to think hacking was bad. It was something you did when you didn’t have a plan, when you didn’t know what you were doing; it’s what amateurs do, noodling around the code without clear direction or intent. Or it was something you did, quick and dirty, just to get something out the door. Hacking is the best, surefire way to waste time and create mountains of unmaintainable code you’ll have to rewrite anyway. I've always struggled with this since I feel so drawn to hacking culture - DIY, playing with things just to see how they work, ignoring the instruction manual, reveling in the aesthetic of a job well done, making what you can within limited constraints - and many of the brightest minds in software development speak quite passionately about the merits of hacking.

Thinking about this for a while, I've come to the conclusion that there’s nothing wrong with hacking. Quite the contrary, hacking could be one of the most positive things a team can do. In fact, many of the agile processes and techniques that have worked for me in the past seem to be designed to cultivate teams of hackers.

It seems I had fallen for some of the myths about hackers.

Myth: Hackers don’t have a plan


Hackers do have a plan but they acknowledge that the plan will change and then plan with this in mind. This means creating plans only to the level of detail to which you are currently comfortable making plans. For example we all know that making a detailed plan at the start of a project, down to the code level, for all but the tiniest programs, is silly since the plan is mere conjecture of a positive outcome. Rather than writing a fantasy on paper, hackers lay plans based on the current knowledge. These plans then expose areas for further exploration (hacking!), the resultant knowledge of which can be turned into more detailed plans.

In XP these detailed plans are created just in time during the iteration planning game, sometimes even later than that. That's not to say that there wasn't a plan before the iteration began. Not so. A great degree of planning, communication, and coordination was required to be able to plan so adeptly, so quickly.


Myth: Hackers don’t know what they’re doing


In the literal sense, a lot of times hackers don’t know what they are doing. That is why they are hacking. The purpose is to figure out how something works or the best approach for something. And that’s the key. At a meta-level, hackers know exactly what they are doing - they are searching for information, for knowledge, for a means to achieve an end.



Action research is a relatively new-to-software-engineering research method that seemed to take agile researchers by storm at XP2010. The basic premise is that a researcher works closely with a consenting organization to observe practices in software engineering. Rather than acting as a passive observer, as is traditionally done, the researcher injects practices directly into the team and observes before and after reactions. For example, in recent work by Tor Erlend Fægri, having observed issues with team collaboration he suggested several techniques to help solve the problem. Some suggestions worked, some didn’t. Fægri was able to take the time to understand what happened and why because he was dedicated to understanding almost as a participant, watching the events unfold. This take on research makes me think of Jane Goodall’s work with chimpanzees in which she directly interacted with wild primates in order to better observe their behavior.



When I first heard about action research it sounded a lot to me like hacking - trying things out to understand how something works.


Myth: Hackers don’t create value


Yes, the code that comes out of a hard core hacking session might be a big pile of spaghetti, but consider the purpose of this work in context. Another perspective is that hacking may incur technical debt which sooner or later will need to be paid off. But it’s important to keep in mind that not all technical debt is bad. When you are hacking, you are either in learning mode or in finish mode. Learning mode is never wasted effort. Finish mode, where you are just trying to get something finished so it can be used, can leave scars on your code, but the immediate value provided to customers can be enormous. When you have a desperate and immediate need, anything that works today is better than something perfect that works next week.

The same goes for architecture. Of course, many of the architecture researchers use words and phrases like "experiments" or "trade-off analysis" to describe this same act of quickly creating something for the sake of knowledge. That's really just hacking too.

Sometimes the artifacts you produce are throwaway. Sometimes the working solution is good enough and you’ll never have to worry about it again. Sometimes you’ll have to pay off the technical debt. But hacking for the right reasons is never wasted effort.


Agile Development Helps Cultivate Good Hackers


Agile software development, be it controlled through time boxes or flow, is all about ensuring that work doesn’t teeter out of control a day at a time. The focus on providing customers with value creates an environment that encourages action and creation. The agile manifesto gently nudges hacking behaviors. Frequent releases of working software, welcoming changing requirements, focus on simplicity and continuous attention to technical excellence, helping the customer understand what they need and then doing your best to make it for them. Even the idea of continuous process improvement is about hacking your team to make yourselves better. Hacking is the very lifeblood of agile software development.

Hacking isn't a dirty word. Hacking could be the most important thing you do all day.