A Note on Programmer Productivity

Posted by Robert Merrill on November 22, 2008 under Estimation | Read the First Comment

In Recession: A Quiet Opportunity for your Software-Intensive Business, I suggested that the productivity gains from improving your team far outweigh the cost savings possible in a buyer’s market for talent.

Just how variable is programmer productivity, anyhow?

Person-to-person and same-person variability

If you give a group of people the same programming project, how long does it take them? How much does an individual’s productivity vary from task to task?

As Sherwood (2006) says clearly, “I have found it surprisingly hard to find good academic literature that measures productivity.” She quotes the following ranges:

• 28:1 best to worst (Sherwood 2006a, citing Sackman et al. 1968)
• 22:1 to 8:1 best to worst for two problems, depending on how you treat the outliers (Sherwood 2006a, citing Curtis 1981)
• 3:1 to 2:1 best to median for two problems (Sherwood 2006a, based on Curtis 1981 data)
• 4:1 to 2:1 best to median (Sherwood 2007, citing DeMarco and Lister 1985. The latter also found that productivity was correlated with the organization that programmers came from, even though they worked independently during the study).
• 2:1 best to worst (Sherwood 2006b, citing Ko et al. 1996)

Spolsky (2006) quotes another range:

10:1 to 5:1 best to worst (Spolsky 2000, based on several years of student project data from a computer science professor at Yale)

Project to project variability

My own project metrics over a period of about five years show:

• 20:1 best to worst project productivity (largely the same people, in different teams, using different technologies to implement various web application projects. Some were all-custom, and others were customization of commercial or open source software).
• 3:1 difference between productivity on two projects completed by the same developer using the same technology

The COCOMO II estimation model (Chulani et al. 1997) has adjustment factors for total project effort based on personnel attributes by role:

• 2.25:1 best to worst analyst
• 1.85:1 best to worst programmer
• 1.5:1 most to least familiarity with the application domain
• 1.5:1 most to least familiarity with the technology in use

Interpretation

There is a lot of variability. No wonder estimation is difficult. Here are some principles about productivity from the above.

• Productivity probably varies within a given organization by at least 2:1
• Individual productivity varies from task to task, probably by an equal amount of 2:1
• A team is more than the sum of its parts. Analysts have the highest leverage; analyst capability swings productivity by over a factor of two, even though they directly contribute maybe one-fourth the effort (my rule of thumb).
• Project productivity, based on individual capability, team capability, familiarity with the domain and the technology, and the technology in use, easily varies by 10:1.

References

Chulani, S., B. Clark, and B. Boehm 1997: Calibration results of COCOMO II.1997. 22nd Annual Software Engineering Workshop, NASA Goddard Space Flight Center.

Curtis, B 1981: Substantiating programmer variability. Proceedings of the IEEE, 69, 846. ISSN 0018-9219.

DeMarco, Tom and Tim Lister 1985: Programmer performance and the effects of the workplace. International Conference on Software Engineering, 268-272. ISBN 0-8186-0620-7.

Ko, A., Myers, B. A, Coblenz, M. J. and H. H Aung 2006: An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Transactions on Software Engineering, 32, 971-987.

McConnell, Steve 2006: Software Estimation: Demystifying the Black Art. Microsoft Press, 308 pp. ISBN 0-7356-0535-0.

Sackman, H., W. J. Erickson and E. E. Grant 1968: Exploratory experimental studies comparing online and offline programming performance. Communications of the ACM, 11, 3-11. ISSN 0001-0782.

Sherwood, Kaitlin Duck 2006a: Don’t get stuck, a review of Sackman and Curtis. http://www.webfoot.com/blog/2006/12/07/programmer-productivity/.

Sherwood, Kaitlin Duck 2006b: Review of Ko’s “An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks.” http://www.webfoot.com/blog/2006/12/15/programmer-productivity-part-2/.

Sherwood, Kaitlin Duck 2007: Review of “DeMarco and Lister’s Programmer performance and the effect of the workplace.” http://www.webfoot.com/blog/2007/02/16/demarco-and-lister/.

Spolsky, Joel 2000: Hitting the High Notes. http://www.joelonsoftware.com/articles/HighNotes.html