Monday, June 25, 2012

Scarce Resources: The Hidden Architectural Driver

All things equal, the difference between success and failure on a project will be determined by how well you understand and deal with the scarce resources.
If a design, particularly a team design, is to have any conceptual integrity, one should name the scarce resource explicitly, track it publicly, control it firmly. – Fred Brooks, "Inches, Ounces, Bits, Dollars – The Budgeted Resource," The Design of Design
Scarce resources are hard to come by and in high demand. They are the ones that, if they were to suddenly become unavailable, could threaten your ability to deliver promised functionality and quality on time and on budget. Of course, this could describe a lot of things on a lot of different projects. Calendar time, engineering hours, system memory, domain knowledge, number of calls to a web service, CPU time, money, time in a day, bandwidth, database queries – depending on the project, any one of these things could become scarce enough that you need to pay extra attention to how you deal with it. Nearly anything could become scarce.

To make matters worse, scarce resources are moving targets and could change at any time. Different design decisions can make resources hard to come by or even free resources from scarcity, making things that were once scarce, plentiful. Architectural integrity demands constant vigilance from all team members.

Though scarcity is rarely mentioned explicitly, scarce resources drive design decisions in the architecture of a system, as much as functional requirements, quality attributes, and technical and business constraints. Generally there are only 1 or 2 scarce resources on a project at any given time and you'll nearly always want to strictly budget and track the most limiting, most scarce resources you know about.

Planning for Scarcity

To deal with scarce resources you must first identify them. The easiest way to uncover scarcity is with a little risk management. My personal preference is to make identifying scarcity a part of the team's regular continuous risk management routine. Combining the other architectural drivers with your team's threshold of success makes it easy to identify and prioritize risks that may influence the architecture. Looking for themes among the highest priority risks will uncover possible points of resource scarcity. Pay close attention to the technical and business constraints as well as quality attribute scenarios that seem especially difficult. These are areas where resources are likely to be scarce.

Most of the times your gut is a good indicator, but translating your gut feelings into something you can talk about as a team is tough.

Since resource scarcity evolves, it's not so important that you identify everything perfectly. It's much worse for you to do nothing than it is to measure and control the wrong things. Usually you'll need to identify some kind of a proxy that allows you to see and measure the scarce resource. And since scarcity tends to move around, it's ok (actually it's a great thing) if you refine your tracking and control methods as the project evolves. I've found that dealing with scarce resources is more like horseshoes and hand grenades than it is brain surgery – as long as you're focusing on the right general area it's better to do something than abstain from action out of worries about a lack of precision.

Looking for tension in quality attributes is another source good source for locating scarcity. Tension within the needs of a system is an indication that design decisions are experiencing influence from conflicting properties. Relieving this tension may not be possible, but understanding why it exists and managing scarcity around it is essential for satisfying all properties involved.

Resource scarcity should be included in teams' architecture discussions alongside functional requirements, quality attributes, technical constraints, and business constraints. How you choose to budget a scarce resource will influence how you design a system. Missing a scarce resource could create conflicts and prevent you from achieving specific qualities in your architecture.

Examples of Scarcity

Often times it's easy to get a feeling for where a scarce resource will probably be, but figuring out how to name, track, and control the resource explicitly can be much more challenging. In some projects, the scarce resource can be pretty obvious. In "Ten Pounds in a Five-Pound Sack" from The Mythical Man Month, Brooks talks about how storage (memory) was scarce in the design of the OS/360 and some of the lessons he learned when trying to budget that resource.

From my own experience creating Velocity search applications with Vivisimo, CPU is nearly always a scarce resource. When it comes to quickly crawling and indexing vast amounts of data (on the order terabytes of data, millions of individual documents), CPU cores are always in high demand and seemingly short supply. Identifying processing capability as a scarce resource gives us the upper hand in designing the system. We know to ask pointed questions about how often data changes, how much lag time is acceptable until changes are reflected in the search index, and how long it takes to crawl a complete data set. Carefully scheduling crawls is one strategy we use to manage this scarce resource. We also spend time profiling crawl efficiency, especially how much time is necessary to prepare documents for indexing. Shaving just 50 milliseconds off the time required to crawl and index a document can shave over 2 days off the total crawl time for a data set of 4,000,000 documents. Knowing that CPU time is scarce, we know that we need to spend more time thinking about the design of the system with respect to this scarcity.

Another great example can be found in The Xbox360 Uncloaked" a case study by Dean Takahashi. Scarce resources for the Xbox360 project included things like heat, the size of a silicon wafer (a proxy for a number of factors including performance, future earnings potential, and improved graphics quality), game developers' mindshare, calendar time, and physical size of the box. Juggling these scarce resources was essential to the project's success.

Practical Scarcity Management?

While I understand that scarcity is an important and often overlooked architectural driver, I don't have any definitive concrete guidance for identifying and managing it other than the nuggets of advice based on my own experiences and the few examples that I have shared. If these ideas resonate with you and you have more examples, please share them in the comments to help flesh this idea out further.