Making Software Better, Step 2: Get Serious About Internal Quality

Posted by Robert Merrill on March 11, 2013 under 100 Words, Agile Methods, Waterfall (SDLC) | Read the First Comment

One frustrating evening years ago, when I was a professional scientist and a self-taught programmer, I blurted, “Programming takes longer when I’m in a hurry than it does when I’m taking my time!”

Graph of Unit Cost against Quality; unit cost decreases with increasing quality for typical business software, and then cost rises sharply for the level of quality required for life-critical software. Due to a lack of internal quality, most organizations have higher software costs than necessary.I had stumbled upon what is sometimes called the Law of Software Quality.

Whoever said “Fast, Cheap, or Good—pick two!” started a dangerous lie. Unless it’s extremely simple software, to be used by its creator, once, and then deleted from the computer, improving quality makes the software less costly, not more.

OK, this isn’t true if you’re writing software where people could die if there’s a single bug (mistake) in it. That level of quality costs extra.

But for typical business software, there’s a sweet spot of quality, and most organizations aren’t there. Unless you have intentional quality measures in place, you can probably double your productivity by doing a little extra work of the proper sort at the right steps in the process.

So, why are effective quality practices not universal?

  • To uninformed outsiders, everything but typing looks like Extra Work. Budgets with tasks (as opposed to features) as line items invite cutting quality measures to save time or money (but it doesn’t work in the end). Feature-based budgets are better because they drive conversations about feature value and scope when “cutting to fit.” But if your organization has separate groups that do quality-related tasks, task-based budgets are hard to avoid. Many big shops are organized this way.
  • The most cost-effective quality measure, formal Inspection, isn’t as fun as programming. Lesser forms of requirements, design, and code review, while not as anal-retentive as Inspection, are less effective and still not as fun as programming. “The only problem we have around here is not enough meetings,” said no programmer, ever. Startups begin with just programmers (and maybe UI designers) and it’s easy to establish an inefficient code-and-fix culture and lay down a foundation of low-quality code that makes it hard for the business to pivot or scale up.

The first “quality measure” most firms introduce is Testing, in order to keep External Quality at an acceptable level. (External Quality is essentially, “does it work?”) The problem is that testing can only detect mistakes, not prevent or fix them, and then it can only detect certain kinds of mistake. Software can have high External Quality but still be made of Magic Spaghetti—poor Internal Quality.

Improving Internal Quality is where the payoff really happens. The reason why the quality curve looks the way it does is that tracking down mistakes in software is the most time-consuming form of Extra Work there is. It’s also hard to predict. Quality measures other than manual post-coding testing pay off because they detect mistakes relatively soon after they’re made, when they’re still easy to find.

Although I believe it’s easier for most teams to create high-quality software using an Agile method (I’m assuming this means automated testing, one of the Load-Bearing Walls of Agile), quality isn’t the main reason I prefer Agile. Regardless of methodology, introducing the right quality measures is a sure win.

Waterfall shops tend to use more formal requirements, design and code reviews, while Agile shops tend to use refactoring, supported by automated testing. Coding standards, especially naming conventions, are a good idea anywhere, provided they don’t trigger a religious war. Most languages have one or more de facto standards. Pick one and use it. Pair programming is the Agile form of code review. It catches a lot of mistakes but drives cost accountants crazy, unless they believe the quality graph. The most common review mistakes are reviewing too late and reviewing too much at once.

Another term for poor Internal Quality is Technical Debt. Like financial debt, it bears “interest,” in the form of extra time spent changing, enhancing, repairing, and upgrading a system that’s full of Magic Spaghetti and therefore hard to understand and change without making another mistake.

So whenever I have to help a firm Make Software Better, I first help them get a grip on their commitments. That frees up some time for them to introduce some quality measures. Those take time to gain traction. The longer the cycle time (time from gleam in the eye to feature in use), the longer it takes for the low-quality work in progress to clear the development pipeline and—gasp!—enter production, where it will continue to drive higher-than-necessary costs until remediated.

Business Analysis Triage: Breadth or Depth

Posted by Robert Merrill on September 18, 2012 under Agile BA, Business Analysis and Requirements, Project Set-Up, Waterfall (SDLC) | Be the First to Comment

Syed Raman asked the IIBA LinkedIn group,

“Which way should i cover? Vertically ? Or horizontally? I have to meet my deadline..!!!” — most of the BAs face this part …Need some expert’s opinion.

If you can’t negotiate an extension, you have to perform triage. “What will cause the people downstream the most problems if left undone?” My starting answers, not knowing anything about the context, are:

  1. Requirements with a high architectural impact, and
  2. Requirements that are essential, but implied

If I don’t provide adequate guidance about #1, they are more likely to make a bad architectural decision, and those can be time-consuming and costly to fix later.

#2 might need some clarification. Suppose the stated requirements call for the system to provide information about X. But there are no stated requirements for putting information about X into the system. But without information about X, the primary requirements can’t be met. I should be asking, “Where does my system get information about X?“ That might uncover a whole interface to another system, or a set of input screens, and a team that will use them, and a whole new stakeholder! A gap this big could totally blow a schedule that’s probably already too “aggressive.”

That’s what I do. And when I get some time, I go upstream and find out where the schedule came from that didn’t allow me the time to do my job in the first place. That’s how I got involved with estimation and negotiation, a vital BA responsibility, IMO.

Robert presenting at HTHH Pecha Kucha on Thursday 11/18

Posted by Robert Merrill on November 14, 2010 under Agile Methods, Waterfall (SDLC) | Be the First to Comment

I’m speaking at the High-Tech Happy Hour’s Pecha Kucha this Thursday the 18th sometime between 3:00 PM and 4:30 PM.

A Tale of Two Processes” shows what happens to a hypothetical twelve-feature, one-year project when run following first the waterfall SDLC and second an agile method, assuming that the estimate/commitment is too low by one-third. This is a highly visual presentation of an article I published on the Wisconsin Technology Network back in February 2009.

What makes a Pecha Kucha fun for presenters and attendees alike is the run-and-gun format—20 slides, 20 seconds each (they’re on timed advance), no exceptions. Pecha Kucha enforces Franklin Delano Roosevelt’s advice to public speakers, “Be sincere, be brief, be seated.”

Thank you, WI BADD

Posted by Robert Merrill on October 6, 2010 under Agile Methods, Estimation, Project Set-Up, Waterfall (SDLC) | Be the First to Comment

Thank You to the three Wisconsin IIBA chapters who hosted the fourth annual WI BADD™ on Tuesday, 10/5/2010, and the firms that sponsored it. I met some new people and learned some new things, and I know that was a lot of work to put on.

Thanks also to the enthusiastic audience for my talk, “Room To Breathe: The Role of the BA in Shaping Early Project Expectations.” If you would like me to speak with your software sponsors or development group about the project target, estimation, and commitment problem, please contact me and we’ll set something up.

Several of you requested additional material on the estimation problem. Here are a few things:

Thanks again, everybody at WI BADD™, for a great day. I hope to see you again on Tuesday, 10/11/11.

Deniability and Software–Ouch!

Posted by Robert Merrill on April 26, 2010 under Software-Intensive Businesses, Waterfall (SDLC) | Be the First to Comment

I’ve long told my customers that big waterfall software specs are more like insurance policies than blueprints, especially when I hear the phrase “sign-off” more than once a week. They are part of Covey the Younger’s Trust Tax.

But in Deniability, Seth Godin puts it better that I ever could. “At some point, that effort [anticipatory CYA] becomes so great you never actually ship anything, which of course is the very best protection against failure.”


Waterfall, RUP, and Agile: Which is Right for You?

Posted by Robert Merrill on December 17, 2009 under Agile Methods, Waterfall (SDLC) | Be the First to Comment

Serhiy Kharytonov published a fine summary of software methodologies for non-technical leaders at Executive Brief. He makes several excellent observations, but he also perpetuates a destructive myth. I’ve worked with all three of Waterfall, RUP, and Agile, and here’s my take.

Excellent observations

Then, stay agile in the approach to the process itself, constantly looking back, re-evaluating and revising the development process until it fits your current circumstances most successfully.

If you learn nothing else from Serhiy and I, learn this. Change from a culture of blame-fixing to a culture of continuous improvement, with no political unmentionables, and you will get a lot more value for your software-development money, and everything else besides.

Then, if you want to learn one more lesson the easy way rather than from a painful and expensive experience, Read more of this article »