Scaling Agile Software Development for Larger Projects 

"Small projects can succeed through sheer force of will and a bit of luck.  Medium and large projects require a more systematic approach" (McConnell, 1998, p. 36).  

Most of the Agile Software Development methods are designed for small team sizes, for example the original XP team had 8 people, but what if we're talking about 30 developers (let alone customers); what process should we use?   

Categorising projects and methods 

Alistair Cockburn (2002) categorises projects by the Criticality of the product and the Number of People, and he recommends picking a method based on this categorisation. He measures Criticality on this scale: 

C=Loss of Comfort, e.g. order a pizza but get lasagne. 
D=Loss of Discretionary Money, but system errors can be fixed by a manual system 
E=Loss of Essential Money. System errors cause bankruptcy. 
L=Loss of Life 

In one project I was almost involved in, the categorisation would have varied over time. The initial functionality was a proof of concept involving change of address functionality, which was C (Comfort), or at worst D (Discretionary Money). Subsequent functionality was unlikely to kill the customer, i.e. if the system made a mistake it would still count as Discretionary Money. The team was expected to grow from 6 (ish) to 30 (ish). So in terms of Cockburn's  categorisation system the project would have gone from a C6 to a D30. 

[By the way Steve McConnell (1998), quoted above, categorises a medium sized project as 3-25 people for 3-18 months with a code base of 20-250 K source lines of code.]

XP 

"Size clearly matters. You probably couldn't run an XP project with a hundred programmers. Nor fifty. Nor twenty, probably. Ten is definitely doable" (Beck, 2000, p. 157). 

Cockburn (2002) says XP is appropriate for C4 to E14 projects. This is not to say that XP can't be used for bigger projects (it is already), just that it has to be tweaked to work for them: 

XP, as written, has been demonstrated on projects with up to 12 programmers and four onsite customers. It may have trouble with larger teams due to its reliance on tacit knowledge. It is difficult to build extensive tacit knowledge without good osmotic communication, and it is hard to do with more people than conveniently fit in a room. A larger project team trying XP will have to adjust the teaming structures interfaces, and use of documentation to accommodate the greater coordination needs of the larger group and the thinner communication lines. (p. 169). 

This has been demonstrated on at least one large XP project run by ThoughtWorks.  In his article XP On A Large Project – A Developer’s View Amr Elssamadisy described a big XP team that abandoned some of the XP practices, and introduced more conventional practices to cope with the size.   

Crystal Orange 

Cockburn's (1998, 2002) claim Crystal Clear is a D6 method, Crystal Yellow is D20 and Crystal Orange is D40 - all are similar, it is just for bigger projects he introduces more formality. The Crystal family of methods is considered light-but-sufficient so I thought I'd compare Cockburn's Clear (D6) to his Orange (D40) to see what we might add to our mix for larger projects. 

The process definition (what Cockburn calls the "policy standards") is identical in the two methods, except the duration of an increment is slightly longer in Crystal Orange. 

Crystal Orange adds a number of specialist roles. 

The Crystal methods require more documentation than XP. However, Crystal Orange adds only two extra work products Crystal Clear - status reports and inter-team specs - and replaces the lightweight requirements specification with a more conventional requirements document. 

So a bigger Crystal team uses more specialised roles, and produces more deliverables.  

DSDM 

The DSDM manual (DSDM, 1997) sets a limit of 6 teams of 6 people (including customers), and says not to use it for safety critical systems, so DSDM is probably a C4 to E36 method. DSDM is heavier than the other methods in that there are many prescribed processes and work products. 

Scrum 

Many people use Scrum (Schwaber & Beedle, 2002) in conjunction with XP, claiming this makes XP more scalable. The Scrum strategy for scaling is to have more Scrum teams - one team for each "application" and one team for the shared resources (sometimes called the Architecture team). Scrum of Scrums meetings provide inter-team communication. In fact Scrum claims to be infinitely scalable, but the examples from the book only mention 3 parallel application teams, and presumably a shared resource team, so in my opinion, Scrum is proven as a C5 to D32 method. 

Conclusion

Basically a larger team needs more specialist roles and more documentation, even if they're trying to be Agile.  DSDM already has those roles and deliverables as standard.   Crystal and XP add formality to the their default processes to cope with larger projects.  This also fits my experience of running large Agile projects.  

Scrum differs by offering a light process for scaling up.  I haven't tried it so I can't comment on whether it would work.  

References

Beck, K. (2000).  Extreme Programming Explained: Embrace Change.  Reading, Massachusetts: Addison-Wesley. 

Beck, K, & Fowler, M. (2001).  Planning Extreme Programming.  Boston: Addison-Wesley. 

Cockburn, A. (2002).  Agile Software Development.  Boston: Addison-Wesley.  

The DSDM Consortium. (1997).  Dynamic System Development Method [3^rd ed.].  Tesseract Publishing.  

Elssamadisy, A. (??). XP On A Large Project – A Developer’s View.  ThoughtWorks.  [Also on-line at http://www.xpuniverse.com/2001/pdfs/EP202.pdf] 

McConnell, S. (1998). Software Project Survival Guide: How to be sure your first important project isn't your last.  USA: Microsoft Press.

Schwaber, K., & Beddle, M. (2002).  Agile Software Development with Scrum.  NJ: Prentice Hall.