How well does Agile scale?

[Adam White-2]

I'm wondering how well agile scales as the teams get larger. Does
anyone have any experiences they can share about using scrum on a team
of 35-40 developers and testers for an enterprise level product?

[Phlip]

Adam White wrote:

> I'm wondering how well agile scales as the teams get larger. Does
> anyone have any experiences they can share about using scrum on a team
> of 35-40 developers and testers

Turn the question around, and ask how you got up to 40 engineers? I think a team
should start small, and should add developers until its velocity levels out.

 > for an enterprise level product?

In-house development is super easy! You just keep collating feature requests,
doing them, and deploying them daily. You control all the users' hardware &
software, and you can pop the hood on the server if something goes wrong. What
am I missing?

If an enterprise is so big it has more than one kind of end-user population (say
manufacturers vs administrators), then maybe you should split the teams up based
on the stream of feature requests...

--
   Phlip

[Matthew-103]

--- In agile-testing@..., "Adam White" <adam_white99@...>
wrote:
>
> I'm wondering how well agile scales as the teams get larger. Does
> anyone have any experiences they can share about using scrum on a team
> of 35-40 developers and testers for an enterprise level product?
>

How are those teams currently organized?  What are they working on?

McKinsey and Company have, if I recall correctly, thousands of
employees doing knowledge work in an agile fashion.  You can read
about then in "Liberation Management", by Tom Peters.  So I suspect
certain types of work can be sructured to be accomplished in an agile
fashion.

So tell us how the teams are currently organized, and what they are
working on, and maybe we can help.  Organizing the work of a theoretic
al scrum team of 40 people with no context is, as I see it, a chase
after the wind.

--heusser
blog: xndev.blogspot.com

[chrs_mcmhn]

--- In agile-testing@..., "Adam White" <adam_white99@...>
wrote:
>
> I'm wondering how well agile scales as the teams get larger. Does
> anyone have any experiences they can share about using scrum on a team
> of 35-40 developers and testers for an enterprise level product?

I was on a project like this at Thoughtworks.

First we divided the work roughly into 'front end' and 'back end'
teams.  We had Architects designing the APIs for communication between
the projects (which were always changing of course).

Stories for each iteration for each team could range fairly widely in
terms of point size.  Each large team would reformulate itself
iteration-to-iteration, with larger subgroups handling larger stories
and smaller subgroups handling smaller stories or more specialized
stories.  

There was a single Project Manager (who was awesome) to handle overall
burndown charts, project scope, story management, budget, personnel,
etc.  

We also had a single Build Guy managing code repositories and test
environments and such.

Also keep in mind that Thoughtworks had been refining their agile
process(es) for more than a decade at that point, so calling what we
did "Scrum" would not really be accurate.  It was a sophisticated
process created over a long time by very skilled practitioners.  

[Lisa Crispin-2]

I haven't worked on a project bigger than 30 developers, but I've visited
and talked to large development organizations who did this successfully. I
hope some of the teams I know about (whose members are on this list) will
share their experiences - you know who you are!

Generally the teams I've talked to divide into small teams of 10 members or
fewer, with all the roles they need on each team and maybe some supporting
teams of specialists who share time among different teams. They do scrum of
scrums on a daily basis so that everyone stays on the same page. CI and
builds can be a challenge if you have a huge code base and builds take a
long time. Sometimes each team builds their own stuff throughout the day or
the iteration and there is one big build a night or every few days - I don't
think this is great and I'd like also like to know how other teams deal with
this.

I think it's a topic that can use a lot more thinking and study - hope I
have the opportunity to do more of this myself.
-- Lisa

On Tue, Dec 9, 2008 at 7:26 PM, Adam White <adam_white99@...> wrote:

>   I'm wondering how well agile scales as the teams get larger. Does
> anyone have any experiences they can share about using scrum on a team
> of 35-40 developers and testers for an enterprise level product?
>
>  
>



--
Lisa Crispin
Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers
and Agile Teams_ (Addison-Wesley 2009)
http://www.agiletester.ca
http://lisa.crispin.home.att.net
http://lisacrispin.blogspot.com

[Adam White-2]

We got to this team size by being successful and having to keep up
with market demand. 5 years ago we were only 4 developers - now we
have 40. We hired as demand for our product and our ability to add
features grew.

I'm glad you made the second point about the enterprise because I've
never thought of enterprise software as being even close to internal.
Our market isn't the internal enterprise it's the external
enterprises - Fortune 500 that type of thing. There is one type of
end user - the one who pays. There are lots of different roles the
paying customer has within their own organization

Adam

--- In agile-testing@..., Phlip <phlip2005@...> wrote:
>
> Adam White wrote:
>
> > I'm wondering how well agile scales as the teams get larger. Does
> > anyone have any experiences they can share about using scrum on a
team
> > of 35-40 developers and testers
>
> Turn the question around, and ask how you got up to 40 engineers? I
think a team
> should start small, and should add developers until its velocity
levels out.
>
>  > for an enterprise level product?
>
> In-house development is super easy! You just keep collating feature
requests,
> doing them, and deploying them daily. You control all the users'
hardware &
> software, and you can pop the hood on the server if something goes
wrong. What
> am I missing?
>
> If an enterprise is so big it has more than one kind of end-user
population (say
> manufacturers vs administrators), then maybe you should split the
teams up based
> on the stream of feature requests...
>
> --
>    Phlip
>

[Adam White-2]

Matt,

I'm not interested in how the work gets organized - i'm interested in
knowing if anyone has been successful with a team of 40 developers
organized into scrum teams.

I'm more interested in how other people have made agile/scrum work
with this size group (or larger) in the context of external
enterprise software development than I am in how my company is
organizing it our context.

Adam

--- In agile-testing@..., "heusserm" <matt.heusser@...>
wrote:
>
> --- In agile-testing@..., "Adam White" <adam_white99@>
> wrote:
> >
> > I'm wondering how well agile scales as the teams get larger. Does
> > anyone have any experiences they can share about using scrum on a
team
> > of 35-40 developers and testers for an enterprise level product?
> >
>
> How are those teams currently organized?  What are they working on?
>
> McKinsey and Company have, if I recall correctly, thousands of
> employees doing knowledge work in an agile fashion.  You can read
> about then in "Liberation Management", by Tom Peters.  So I suspect
> certain types of work can be sructured to be accomplished in an
agile
> fashion.
>
> So tell us how the teams are currently organized, and what they are
> working on, and maybe we can help.  Organizing the work of a
theoretic
> al scrum team of 40 people with no context is, as I see it, a chase
> after the wind.
>
> --heusser
> blog: xndev.blogspot.com
>

[Adam White-2]

Lisa,

Would you know of anyone at those companies who might know someone
that I could talk to about their experiences and how they use scrum?

Maybe we can facilitate this through the use of linked in?
http://www.linkedin.com/in/adamwhite

Adam

--- In agile-testing@..., "Lisa Crispin"
<lisa.crispin@...> wrote:
>
> I haven't worked on a project bigger than 30 developers, but I've
visited
> and talked to large development organizations who did this
successfully. I
> hope some of the teams I know about (whose members are on this
list) will
> share their experiences - you know who you are!
>
> Generally the teams I've talked to divide into small teams of 10
members or
> fewer, with all the roles they need on each team and maybe some
supporting
> teams of specialists who share time among different teams. They do
scrum of
> scrums on a daily basis so that everyone stays on the same page. CI
and
> builds can be a challenge if you have a huge code base and builds
take a
> long time. Sometimes each team builds their own stuff throughout
the day or
> the iteration and there is one big build a night or every few days -
 I don't
> think this is great and I'd like also like to know how other teams
deal with
> this.
>
> I think it's a topic that can use a lot more thinking and study -
hope I
> have the opportunity to do more of this myself.
> -- Lisa
>
> On Tue, Dec 9, 2008 at 7:26 PM, Adam White <adam_white99@...> wrote:
>
> >   I'm wondering how well agile scales as the teams get larger.
Does
> > anyone have any experiences they can share about using scrum on a
team

> > of 35-40 developers and testers for an enterprise level product?
> >
> >  
> >
>
>
>
> --
> Lisa Crispin
> Co-author with Janet Gregory, _Agile Testing: A Practical Guide for

Testers
> and Agile Teams_ (Addison-Wesley 2009)
> http://www.agiletester.ca
> http://lisa.crispin.home.att.net
> http://lisacrispin.blogspot.com
>

[Phlip]

Adam White wrote:

> We got to this team size by being successful and having to keep up
> with market demand. 5 years ago we were only 4 developers - now we
> have 40. We hired as demand for our product and our ability to add
> features grew.

Which small sub-team has the best turnaround time on features?

> I'm glad you made the second point about the enterprise because I've
> never thought of enterprise software as being even close to internal.

Oh, I get it. You cater to external enterprises. That's why it's hard!

--
   Phlip

[Dietmar Strasser]

Hi Adam,

 

We have ~70 people working on 4 products with a size of ~ 5 MLOC. We
moved from a more traditional development process towards an agile
development model over the last 2 years. My experience is that Agile is
a journey, not a destination. I talked about this at the StarEast
conference this year
(http://www.sqe.com/ConferenceArchive/StarEast2008/SessionsWednesday.htm
l#W8). In this presentation I also talked about the challenge we had
with a distributed environment (I think in a big company you will always
have to deal with that), where we had to find an approach to keep the
communication costs low, but also utilize the existing resources in a
best way.

 

Regarding success: We just released another major version for 3 of the 4
products, which is the first real version where we applied a lot of the
agile practices.  There are a lot of very positive things that happened,
some things need to get improved (got identified during a
retrospective), but overall I think it worked very well and agile is
definitely the way to go forward (we are ready for the next bumps on the
road ;=)).

 

If you are interested in more information or want to share some
experiences, please do not hesitate to contact me.

 

Best regards

Didi

 

Dietmar Strasser

Director QA

Lifecycle Quality Management

Borland

Freistaedter Strasse 400

Austria, A-4040 Linz

[phone] - 4014 Direct

[phone] - 4022 Fax

www.borland.com <http://www.borland.com/>

dietmar.strasser@... <mailto:dietmar.strasser@...>

 

Borland - The Open ALM Company

Open to your processes, tools and platforms.  Empowering you to deliver
software on your terms.

 

This email, and any attachments thereto, is intended only for use by the
addressee(s) named herein and may contain legally privileged and/or
confidential information.  If you are not the intended recipient of this
email, you are hereby notified that any dissemination, distribution or
copying of this email, and any attachments thereto, is strictly
prohibited.  If you have received this e-mail in error, please
immediately and permanently delete the original and any copy of any
email and any printout thereof.

 

From: agile-testing@...
[mailto:agile-testing@...] On Behalf Of Adam White
Sent: Mittwoch, 10. Dezember 2008 19:15
To: agile-testing@...
Subject: [agile-testing] Re: How well does Agile scale?

 

Matt,

I'm not interested in how the work gets organized - i'm interested in
knowing if anyone has been successful with a team of 40 developers
organized into scrum teams.

I'm more interested in how other people have made agile/scrum work
with this size group (or larger) in the context of external
enterprise software development than I am in how my company is
organizing it our context.

Adam

--- In agile-testing@...
<mailto:agile-testing%40yahoogroups.com> , "heusserm" <matt.heusser@...>

wrote:
>
> --- In agile-testing@...
<mailto:agile-testing%40yahoogroups.com> , "Adam White" <adam_white99@>
> wrote:
> >
> > I'm wondering how well agile scales as the teams get larger. Does
> > anyone have any experiences they can share about using scrum on a
team
> > of 35-40 developers and testers for an enterprise level product?
> >
>
> How are those teams currently organized? What are they working on?
>
> McKinsey and Company have, if I recall correctly, thousands of
> employees doing knowledge work in an agile fashion. You can read
> about then in "Liberation Management", by Tom Peters. So I suspect
> certain types of work can be sructured to be accomplished in an
agile
> fashion.
>
> So tell us how the teams are currently organized, and what they are
> working on, and maybe we can help. Organizing the work of a
theoretic
> al scrum team of 40 people with no context is, as I see it, a chase
> after the wind.
>
> --heusser
> blog: xndev.blogspot.com
>

 

[rafaels_ulti]

Hello Adam,

Our company has over 70 developers and over 50 testers and we started
doing Scrum 3.5 years ago.  We went through a lot of pain to figure
out how to make it work in a large scale scenario.

In our case we crossed a major threshold by re-organizing the whole
development organization over a year and half ago.  Basically we were
introduced to Lean and that provided a set of tools that allow us to
identify several issues.  Probably the most useful tool is Value
Stream Mapping.  Looking at our products and processes, we change our
organization and our teams.

Most of our Product Owners and their managers move to a new
department that determines the stuff we need to build and in what
order.  The POs still seat close to the teams and work with them all
the time, but they are part of an organization now that is only focus
on the business side.  On the development side we focus on building
teams to match the business organization created by this new product
strategy department.  They divide everything by business lines, so we
create teams to work on those business lines.  In essence, we end up
with a bunch of feature teams.

We have 27 Scrum teams.  Of those 27 teams, we have what we have 21
customer facing teams.  They are changing the product our customers
use.  Because we are so large and our customers can be large
organizations as well, we have a group of teams who are mostly
infrastructure (build, environment management, etc.) and specialty
testing teams (performance, security and product acceptance).  They
do not change the product but either test the product or support the
other teams work.  We have a few framework type teams.

Now that most of the decisions regarding business value are outside
of product development then we can concentrate on building strong
teams.  We now focus on practices, standards and technology.

As part of the re-org we brought Jeff Sutherland to teach everyone in
our top management (Senior VPs, VPs and Directors) Scrum and
introduce them to how other companies handle their Enterprise Agile
implementation.

On the development side, we re-organize our Scrum meetings and added
one meeting for top management.

Team members go to Daily Scrum
Scrum Master goes to Business Line Scrum of Scrums – Keep Strategy
management aware of what the team is working on every day.
All Scrum Masters go to All Dev Scrum of Scrums – Scrum Masters bring
impediments they can not resolve every day to this meeting.  The
meeting is run by a Director who is responsible for working on
removing those impediments as soon as possible.

Development Management has their own Daily Scrum where we reviewed
those impediments and other issues and try to resolve them as soon as
possible.

These are all short meetings to bring everyone in sync every morning.

Once a week the management of Dev and Strategy get together to review
how we are doing on our release and discuss any priority changes or
major unresolved impediments (MetaScrum).

At the team level, we implemented some changes as well:

Get teams to think more about Business Value by putting an emphasis
on business value delivered (tell me when the story is done, not when
the task is completed).  Burn down story points, not hourly tasks.

Change Retrospective to stop keeping lists of start-stop-continue and
focus on team impediments.  Scrum Master keeps list of impediments in
priority order and team focuses on removing one impediment at the
time.  In essence, we do a more Lean retrospective.
Empower teams to manage their talent.  Teams decide who joins their
team during recruiting process and they can move people out of the
team (go to the bench) if the person becomes an impediment and are
not helping the team.  We evaluate teams, not individuals.

One of the recent things we do to constantly evaluate teams is to use
Henrik Kniberg's Scrum checklist which we added the following section:

High Performing team behaviors:
Team writes test code and product code together
Team runs all their tests as often as possible (at least once every
day)
Team keeps tests passing all the time and stops working when a test
fails
Team runs their unit tests before check-in their code
When a defect is found, the team creates a test case first to verify
defect and then fixes the issue

I can keep going, but it will be an even longer posting.  We also
have a few new changes coming up now like an Enterprise Definition of
Done (EDoD).  If you want to contact me directly, we can schedule a
call and go over some of your specific questions.  I just did an hour
and half call with a company last week to share some of the changes
we implemented.

Best Regards,

Rafael

--- In agile-testing@..., "Adam White" <adam_white99@...>
wrote:

>
> Lisa,
>
> Would you know of anyone at those companies who might know someone
> that I could talk to about their experiences and how they use scrum?
>
> Maybe we can facilitate this through the use of linked in?
> http://www.linkedin.com/in/adamwhite
>
> Adam
>
> --- In agile-testing@..., "Lisa Crispin"
> <lisa.crispin@> wrote:
> >
> > I haven't worked on a project bigger than 30 developers, but I've
> visited
> > and talked to large development organizations who did this
> successfully. I
> > hope some of the teams I know about (whose members are on this
> list) will
> > share their experiences - you know who you are!
> >
> > Generally the teams I've talked to divide into small teams of 10
> members or
> > fewer, with all the roles they need on each team and maybe some
> supporting
> > teams of specialists who share time among different teams. They

do
> scrum of
> > scrums on a daily basis so that everyone stays on the same page.
CI
> and
> > builds can be a challenge if you have a huge code base and builds
> take a
> > long time. Sometimes each team builds their own stuff throughout
> the day or
> > the iteration and there is one big build a night or every few
days -
>  I don't
> > think this is great and I'd like also like to know how other
teams

> deal with
> > this.
> >
> > I think it's a topic that can use a lot more thinking and study -
> hope I
> > have the opportunity to do more of this myself.
> > -- Lisa
> >
> > On Tue, Dec 9, 2008 at 7:26 PM, Adam White <adam_white99@> wrote:
> >
> > >   I'm wondering how well agile scales as the teams get larger.
> Does
> > > anyone have any experiences they can share about using scrum on

a

> team
> > > of 35-40 developers and testers for an enterprise level product?
> > >
> > >  
> > >
> >
> >
> >
> > --
> > Lisa Crispin
> > Co-author with Janet Gregory, _Agile Testing: A Practical Guide

for
> Testers
> > and Agile Teams_ (Addison-Wesley 2009)
> > http://www.agiletester.ca
> > http://lisa.crispin.home.att.net
> > http://lisacrispin.blogspot.com
> >
>

[Lisa Crispin-2]

Super post, thanks, Rafael! It's so cool to see in a nutshell how you have
evolved your software development and keep addressing obstacles and
overcoming them. I like Henrik's checklist, it could be a quick way to
answer the question "is my team agile?"
-- Lisa

On Thu, Dec 11, 2008 at 9:42 AM, rafaels_ulti <
rafael_santos@...> wrote:

>   Hello Adam,
>
> Our company has over 70 developers and over 50 testers and we started
> doing Scrum 3.5 years ago. We went through a lot of pain to figure
> out how to make it work in a large scale scenario.
>
> In our case we crossed a major threshold by re-organizing the whole
> development organization over a year and half ago. Basically we were
> introduced to Lean and that provided a set of tools that allow us to
> identify several issues. Probably the most useful tool is Value
> Stream Mapping. Looking at our products and processes, we change our
> organization and our teams.
>
> Most of our Product Owners and their managers move to a new
> department that determines the stuff we need to build and in what
> order. The POs still seat close to the teams and work with them all
> the time, but they are part of an organization now that is only focus
> on the business side. On the development side we focus on building
> teams to match the business organization created by this new product
> strategy department. They divide everything by business lines, so we
> create teams to work on those business lines. In essence, we end up
> with a bunch of feature teams.
>
> We have 27 Scrum teams. Of those 27 teams, we have what we have 21
> customer facing teams. They are changing the product our customers
> use. Because we are so large and our customers can be large
> organizations as well, we have a group of teams who are mostly
> infrastructure (build, environment management, etc.) and specialty
> testing teams (performance, security and product acceptance). They
> do not change the product but either test the product or support the
> other teams work. We have a few framework type teams.
>
> Now that most of the decisions regarding business value are outside
> of product development then we can concentrate on building strong
> teams. We now focus on practices, standards and technology.
>
> As part of the re-org we brought Jeff Sutherland to teach everyone in
> our top management (Senior VPs, VPs and Directors) Scrum and
> introduce them to how other companies handle their Enterprise Agile
> implementation.
>
> On the development side, we re-organize our Scrum meetings and added
> one meeting for top management.
>
> Team members go to Daily Scrum
> Scrum Master goes to Business Line Scrum of Scrums – Keep Strategy
> management aware of what the team is working on every day.
> All Scrum Masters go to All Dev Scrum of Scrums – Scrum Masters bring
> impediments they can not resolve every day to this meeting. The
> meeting is run by a Director who is responsible for working on
> removing those impediments as soon as possible.
>
> Development Management has their own Daily Scrum where we reviewed
> those impediments and other issues and try to resolve them as soon as
> possible.
>
> These are all short meetings to bring everyone in sync every morning.
>
> Once a week the management of Dev and Strategy get together to review
> how we are doing on our release and discuss any priority changes or
> major unresolved impediments (MetaScrum).
>
> At the team level, we implemented some changes as well:
>
> Get teams to think more about Business Value by putting an emphasis
> on business value delivered (tell me when the story is done, not when
> the task is completed). Burn down story points, not hourly tasks.
>
> Change Retrospective to stop keeping lists of start-stop-continue and
> focus on team impediments. Scrum Master keeps list of impediments in
> priority order and team focuses on removing one impediment at the
> time. In essence, we do a more Lean retrospective.
> Empower teams to manage their talent. Teams decide who joins their
> team during recruiting process and they can move people out of the
> team (go to the bench) if the person becomes an impediment and are
> not helping the team. We evaluate teams, not individuals.
>
> One of the recent things we do to constantly evaluate teams is to use
> Henrik Kniberg's Scrum checklist which we added the following section:
>
> High Performing team behaviors:
> Team writes test code and product code together
> Team runs all their tests as often as possible (at least once every
> day)
> Team keeps tests passing all the time and stops working when a test
> fails
> Team runs their unit tests before check-in their code
> When a defect is found, the team creates a test case first to verify
> defect and then fixes the issue
>
> I can keep going, but it will be an even longer posting. We also
> have a few new changes coming up now like an Enterprise Definition of
> Done (EDoD). If you want to contact me directly, we can schedule a
> call and go over some of your specific questions. I just did an hour
> and half call with a company last week to share some of the changes
> we implemented.
>
> Best Regards,
>
> Rafael
>
> --- In agile-testing@... <agile-testing%40yahoogroups.com>,
> "Adam White" <adam_white99@...>
> wrote:
> >
> > Lisa,
> >
> > Would you know of anyone at those companies who might know someone
> > that I could talk to about their experiences and how they use scrum?
> >
> > Maybe we can facilitate this through the use of linked in?
> > http://www.linkedin.com/in/adamwhite
> >
> > Adam
> >
> > --- In agile-testing@... <agile-testing%40yahoogroups.com>,
> "Lisa Crispin"
> > <lisa.crispin@> wrote:
> > >
> > > I haven't worked on a project bigger than 30 developers, but I've
> > visited
> > > and talked to large development organizations who did this
> > successfully. I
> > > hope some of the teams I know about (whose members are on this
> > list) will
> > > share their experiences - you know who you are!
> > >
> > > Generally the teams I've talked to divide into small teams of 10
> > members or
> > > fewer, with all the roles they need on each team and maybe some
> > supporting
> > > teams of specialists who share time among different teams. They
> do
> > scrum of
> > > scrums on a daily basis so that everyone stays on the same page.
> CI
> > and
> > > builds can be a challenge if you have a huge code base and builds
> > take a
> > > long time. Sometimes each team builds their own stuff throughout
> > the day or
> > > the iteration and there is one big build a night or every few
> days -
> > I don't
> > > think this is great and I'd like also like to know how other
> teams
> > deal with
> > > this.
> > >
> > > I think it's a topic that can use a lot more thinking and study -
> > hope I
> > > have the opportunity to do more of this myself.
> > > -- Lisa
> > >
> > > On Tue, Dec 9, 2008 at 7:26 PM, Adam White <adam_white99@> wrote:
> > >
> > > > I'm wondering how well agile scales as the teams get larger.
> > Does
> > > > anyone have any experiences they can share about using scrum on
> a
> > team
> > > > of 35-40 developers and testers for an enterprise level product?
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Lisa Crispin
> > > Co-author with Janet Gregory, _Agile Testing: A Practical Guide
> for
> > Testers
> > > and Agile Teams_ (Addison-Wesley 2009)
> > > http://www.agiletester.ca
> > > http://lisa.crispin.home.att.net
> > > http://lisacrispin.blogspot.com
> > >
> >
>
>  
>

--
Lisa Crispin
Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers
and Agile Teams_ (Addison-Wesley 2009)
http://www.agiletester.ca
http://lisa.crispin.home.att.net
http://lisacrispin.blogspot.com

[Steven Gordon-2]

Rafael,

Very nice!  Perfect approach for product lines.

However, I got the impression that Adam had a single "monolithic" product.

Adam,

I order to do the equivalent, product management will need to find a
way to subdivide their product vertically (perhaps by groups of
related features) and then allocate teams along those subproducts.

Many organizations with a single seemingly indivisible product fall
into the trap of allocating teams horizontally (i.e., along the lines
of the architectural components) .  This would seem to be a more
efficient use of "resources", but the specialization of teams creates
impediments to getting features completely done in single iterations
and thereby inhibits agility.

It is critical to find a way to subdivide into smaller teams.  Once
team sizes become too big for everybody to directly communicate with
everybody else, lines of communication will naturally devolve into a
scale free network ( http://en.wikipedia.org/wiki/Scale-free_network
).  the result is that some people will naturally become communication
hubs and others will mostly communicate through those people, creating
potential of miscommunication and misunderstanding.  In other words, a
large team will naturally divide into smaller subteams, but not
necessarily along functional lines.

This is why it is best to preallocate subteams along functional lines
(and that those functional lines be vertical rather than horizontal).

Steven Gordon

On Thu, Dec 11, 2008 at 9:42 AM, rafaels_ulti
<rafael_santos@...> wrote:

> Hello Adam,
>
> Our company has over 70 developers and over 50 testers and we started
> doing Scrum 3.5 years ago. We went through a lot of pain to figure
> out how to make it work in a large scale scenario.
>
> In our case we crossed a major threshold by re-organizing the whole
> development organization over a year and half ago. Basically we were
> introduced to Lean and that provided a set of tools that allow us to
> identify several issues. Probably the most useful tool is Value
> Stream Mapping. Looking at our products and processes, we change our
> organization and our teams.
>
> Most of our Product Owners and their managers move to a new
> department that determines the stuff we need to build and in what
> order. The POs still seat close to the teams and work with them all
> the time, but they are part of an organization now that is only focus
> on the business side. On the development side we focus on building
> teams to match the business organization created by this new product
> strategy department. They divide everything by business lines, so we
> create teams to work on those business lines. In essence, we end up
> with a bunch of feature teams.
>
> We have 27 Scrum teams. Of those 27 teams, we have what we have 21
> customer facing teams. They are changing the product our customers
> use. Because we are so large and our customers can be large
> organizations as well, we have a group of teams who are mostly
> infrastructure (build, environment management, etc.) and specialty
> testing teams (performance, security and product acceptance). They
> do not change the product but either test the product or support the
> other teams work. We have a few framework type teams.
>
> Now that most of the decisions regarding business value are outside
> of product development then we can concentrate on building strong
> teams. We now focus on practices, standards and technology.
>
> As part of the re-org we brought Jeff Sutherland to teach everyone in
> our top management (Senior VPs, VPs and Directors) Scrum and
> introduce them to how other companies handle their Enterprise Agile
> implementation.
>
> On the development side, we re-organize our Scrum meetings and added
> one meeting for top management.
>
> Team members go to Daily Scrum
> Scrum Master goes to Business Line Scrum of Scrums – Keep Strategy
> management aware of what the team is working on every day.
> All Scrum Masters go to All Dev Scrum of Scrums – Scrum Masters bring
> impediments they can not resolve every day to this meeting. The
> meeting is run by a Director who is responsible for working on
> removing those impediments as soon as possible.
>
> Development Management has their own Daily Scrum where we reviewed
> those impediments and other issues and try to resolve them as soon as
> possible.
>
> These are all short meetings to bring everyone in sync every morning.
>
> Once a week the management of Dev and Strategy get together to review
> how we are doing on our release and discuss any priority changes or
> major unresolved impediments (MetaScrum).
>
> At the team level, we implemented some changes as well:
>
> Get teams to think more about Business Value by putting an emphasis
> on business value delivered (tell me when the story is done, not when
> the task is completed). Burn down story points, not hourly tasks.
>
> Change Retrospective to stop keeping lists of start-stop-continue and
> focus on team impediments. Scrum Master keeps list of impediments in
> priority order and team focuses on removing one impediment at the
> time. In essence, we do a more Lean retrospective.
> Empower teams to manage their talent. Teams decide who joins their
> team during recruiting process and they can move people out of the
> team (go to the bench) if the person becomes an impediment and are
> not helping the team. We evaluate teams, not individuals.
>
> One of the recent things we do to constantly evaluate teams is to use
> Henrik Kniberg's Scrum checklist which we added the following section:
>
> High Performing team behaviors:
> Team writes test code and product code together
> Team runs all their tests as often as possible (at least once every
> day)
> Team keeps tests passing all the time and stops working when a test
> fails
> Team runs their unit tests before check-in their code
> When a defect is found, the team creates a test case first to verify
> defect and then fixes the issue
>
> I can keep going, but it will be an even longer posting. We also
> have a few new changes coming up now like an Enterprise Definition of
> Done (EDoD). If you want to contact me directly, we can schedule a
> call and go over some of your specific questions. I just did an hour
> and half call with a company last week to share some of the changes
> we implemented.
>
> Best Regards,
>
> Rafael
>

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/agile-testing/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:agile-testing-digest@...
    mailto:agile-testing-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
    agile-testing-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

[rafaels_ulti]

Thanks, Lisa.

I find Henrik's checklist very useful.  I keep making minor changes
to the checklist as we go along.  

I think you bring a very good observation.  If I can point to a key
element of our transformation is how empower we felt by the Lean
ideas and how they made us an organization that is constantly
improving.  When I look at our organization today what I see is a
constant obsession with improving our value stream.  We did not have
that in the previous period when we implemented "Scrum out of the
box".

Best Regards,

Rafael
--- In agile-testing@..., "Lisa Crispin"
<lisa.crispin@...> wrote:
>
> Super post, thanks, Rafael! It's so cool to see in a nutshell how
you have
> evolved your software development and keep addressing obstacles and
> overcoming them. I like Henrik's checklist, it could be a quick way
to
> answer the question "is my team agile?"
> -- Lisa
>
> On Thu, Dec 11, 2008 at 9:42 AM, rafaels_ulti <
> rafael_santos@...> wrote:
>
> >   Hello Adam,
> >
> > Our company has over 70 developers and over 50 testers and we
started
> > doing Scrum 3.5 years ago. We went through a lot of pain to figure
> > out how to make it work in a large scale scenario.
> >
> > In our case we crossed a major threshold by re-organizing the
whole
> > development organization over a year and half ago. Basically we
were
> > introduced to Lean and that provided a set of tools that allow us
to
> > identify several issues. Probably the most useful tool is Value
> > Stream Mapping. Looking at our products and processes, we change
our
> > organization and our teams.
> >
> > Most of our Product Owners and their managers move to a new
> > department that determines the stuff we need to build and in what
> > order. The POs still seat close to the teams and work with them
all
> > the time, but they are part of an organization now that is only
focus
> > on the business side. On the development side we focus on building
> > teams to match the business organization created by this new
product
> > strategy department. They divide everything by business lines, so
we
> > create teams to work on those business lines. In essence, we end
up
> > with a bunch of feature teams.
> >
> > We have 27 Scrum teams. Of those 27 teams, we have what we have 21
> > customer facing teams. They are changing the product our customers
> > use. Because we are so large and our customers can be large
> > organizations as well, we have a group of teams who are mostly
> > infrastructure (build, environment management, etc.) and specialty
> > testing teams (performance, security and product acceptance). They
> > do not change the product but either test the product or support
the
> > other teams work. We have a few framework type teams.
> >
> > Now that most of the decisions regarding business value are
outside
> > of product development then we can concentrate on building strong
> > teams. We now focus on practices, standards and technology.
> >
> > As part of the re-org we brought Jeff Sutherland to teach
everyone in
> > our top management (Senior VPs, VPs and Directors) Scrum and
> > introduce them to how other companies handle their Enterprise
Agile
> > implementation.
> >
> > On the development side, we re-organize our Scrum meetings and
added
> > one meeting for top management.
> >
> > Team members go to Daily Scrum
> > Scrum Master goes to Business Line Scrum of Scrums – Keep Strategy
> > management aware of what the team is working on every day.
> > All Scrum Masters go to All Dev Scrum of Scrums – Scrum Masters
bring
> > impediments they can not resolve every day to this meeting. The
> > meeting is run by a Director who is responsible for working on
> > removing those impediments as soon as possible.
> >
> > Development Management has their own Daily Scrum where we reviewed
> > those impediments and other issues and try to resolve them as
soon as
> > possible.
> >
> > These are all short meetings to bring everyone in sync every
morning.
> >
> > Once a week the management of Dev and Strategy get together to
review
> > how we are doing on our release and discuss any priority changes
or
> > major unresolved impediments (MetaScrum).
> >
> > At the team level, we implemented some changes as well:
> >
> > Get teams to think more about Business Value by putting an
emphasis
> > on business value delivered (tell me when the story is done, not
when
> > the task is completed). Burn down story points, not hourly tasks.
> >
> > Change Retrospective to stop keeping lists of start-stop-continue
and
> > focus on team impediments. Scrum Master keeps list of impediments
in
> > priority order and team focuses on removing one impediment at the
> > time. In essence, we do a more Lean retrospective.
> > Empower teams to manage their talent. Teams decide who joins their
> > team during recruiting process and they can move people out of the
> > team (go to the bench) if the person becomes an impediment and are
> > not helping the team. We evaluate teams, not individuals.
> >
> > One of the recent things we do to constantly evaluate teams is to
use
> > Henrik Kniberg's Scrum checklist which we added the following
section:
> >
> > High Performing team behaviors:
> > Team writes test code and product code together
> > Team runs all their tests as often as possible (at least once
every
> > day)
> > Team keeps tests passing all the time and stops working when a
test
> > fails
> > Team runs their unit tests before check-in their code
> > When a defect is found, the team creates a test case first to
verify
> > defect and then fixes the issue
> >
> > I can keep going, but it will be an even longer posting. We also
> > have a few new changes coming up now like an Enterprise
Definition of
> > Done (EDoD). If you want to contact me directly, we can schedule a
> > call and go over some of your specific questions. I just did an
hour
> > and half call with a company last week to share some of the
changes
> > we implemented.
> >
> > Best Regards,
> >
> > Rafael
> >
> > --- In agile-testing@... <agile-testing%
40yahoogroups.com>,
> > "Adam White" <adam_white99@>
> > wrote:
> > >
> > > Lisa,
> > >
> > > Would you know of anyone at those companies who might know
someone
> > > that I could talk to about their experiences and how they use
scrum?
> > >
> > > Maybe we can facilitate this through the use of linked in?
> > > http://www.linkedin.com/in/adamwhite
> > >
> > > Adam
> > >
> > > --- In agile-testing@... <agile-testing%
40yahoogroups.com>,
> > "Lisa Crispin"
> > > <lisa.crispin@> wrote:
> > > >
> > > > I haven't worked on a project bigger than 30 developers, but
I've
> > > visited
> > > > and talked to large development organizations who did this
> > > successfully. I
> > > > hope some of the teams I know about (whose members are on this
> > > list) will
> > > > share their experiences - you know who you are!
> > > >
> > > > Generally the teams I've talked to divide into small teams of
10
> > > members or
> > > > fewer, with all the roles they need on each team and maybe
some
> > > supporting
> > > > teams of specialists who share time among different teams.
They
> > do
> > > scrum of
> > > > scrums on a daily basis so that everyone stays on the same
page.
> > CI
> > > and
> > > > builds can be a challenge if you have a huge code base and
builds
> > > take a
> > > > long time. Sometimes each team builds their own stuff
throughout

> > > the day or
> > > > the iteration and there is one big build a night or every few
> > days -
> > > I don't
> > > > think this is great and I'd like also like to know how other
> > teams
> > > deal with
> > > > this.
> > > >
> > > > I think it's a topic that can use a lot more thinking and

study -
> > > hope I
> > > > have the opportunity to do more of this myself.
> > > > -- Lisa
> > > >
> > > > On Tue, Dec 9, 2008 at 7:26 PM, Adam White <adam_white99@>
wrote:
> > > >
> > > > > I'm wondering how well agile scales as the teams get larger.
> > > Does
> > > > > anyone have any experiences they can share about using
scrum on
> > a
> > > team
> > > > > of 35-40 developers and testers for an enterprise level
product?
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Lisa Crispin
> > > > Co-author with Janet Gregory, _Agile Testing: A Practical
Guide

> > for
> > > Testers
> > > > and Agile Teams_ (Addison-Wesley 2009)
> > > > http://www.agiletester.ca
> > > > http://lisa.crispin.home.att.net
> > > > http://lisacrispin.blogspot.com
> > > >
> > >
> >
> >  
> >
>
>
>
> --
> Lisa Crispin
> Co-author with Janet Gregory, _Agile Testing: A Practical Guide for

Testers
> and Agile Teams_ (Addison-Wesley 2009)
> http://www.agiletester.ca
> http://lisa.crispin.home.att.net
> http://lisacrispin.blogspot.com
>

[rafaels_ulti]

Thanks, Steven.

Actually our product (Human Resources, Payroll and Talent management)
is more "monolithic" than you might think.  Most of those business
lines (what you called product lines) exist on the same platform.  
They shared databases, tables, security, workflow, business rules
engine, many common domain specific tables, etc.  That is why is so
important to have those All Dev Scrum of Scrums every morning.  That
way we find out issues and dependencies all the time that we need to
address as soon as possible.  Our product is more like a single suite
than a portfolio of separate products.

I did went back and read Adam's postings again and I was not able to
identify what kind of product he is working on.  So, I do not know
how "monolithic" this product might be.  I read that is an enterprise
product and they have Fortune 500 customers.  We have that kind of
customers as well.

I also read that he was asking for success stories.  I think we
became succesful in the last 1 year.  Before we were not successful.  
I know other companies like Salerforce.com with more people and teams
that are successful as well on their Enterprise Agile
implementation.  Actually, I remember several presentations on the
recent Agile conference from many Enterprise Agile companies.  Now,
we are not perfect, but we got out of a cycle of no improvement and
move into a cycle of constant improvement.

Best Regards,

Rafael

--- In agile-testing@..., "Steven Gordon"
<sgordonphd@...> wrote:
>
> Rafael,
>
> Very nice!  Perfect approach for product lines.
>
> However, I got the impression that Adam had a single "monolithic"
product.
>
> Adam,
>
> I order to do the equivalent, product management will need to find a
> way to subdivide their product vertically (perhaps by groups of
> related features) and then allocate teams along those subproducts.
>
> Many organizations with a single seemingly indivisible product fall
> into the trap of allocating teams horizontally (i.e., along the
lines
> of the architectural components) .  This would seem to be a more
> efficient use of "resources", but the specialization of teams
creates
> impediments to getting features completely done in single iterations
> and thereby inhibits agility.
>
> It is critical to find a way to subdivide into smaller teams.  Once
> team sizes become too big for everybody to directly communicate with
> everybody else, lines of communication will naturally devolve into a
> scale free network ( http://en.wikipedia.org/wiki/Scale-free_network
> ).  the result is that some people will naturally become
communication
> hubs and others will mostly communicate through those people,
creating
> potential of miscommunication and misunderstanding.  In other
words, a
> large team will naturally divide into smaller subteams, but not
> necessarily along functional lines.
>
> This is why it is best to preallocate subteams along functional
lines
> (and that those functional lines be vertical rather than
horizontal).
>
> Steven Gordon
>
> On Thu, Dec 11, 2008 at 9:42 AM, rafaels_ulti
> <rafael_santos@...> wrote:
> > Hello Adam,
> >
> > Our company has over 70 developers and over 50 testers and we
started
> > doing Scrum 3.5 years ago. We went through a lot of pain to figure
> > out how to make it work in a large scale scenario.
> >
> > In our case we crossed a major threshold by re-organizing the
whole
> > development organization over a year and half ago. Basically we
were
> > introduced to Lean and that provided a set of tools that allow us
to
> > identify several issues. Probably the most useful tool is Value
> > Stream Mapping. Looking at our products and processes, we change
our
> > organization and our teams.
> >
> > Most of our Product Owners and their managers move to a new
> > department that determines the stuff we need to build and in what
> > order. The POs still seat close to the teams and work with them
all
> > the time, but they are part of an organization now that is only
focus
> > on the business side. On the development side we focus on building
> > teams to match the business organization created by this new
product
> > strategy department. They divide everything by business lines, so
we
> > create teams to work on those business lines. In essence, we end
up
> > with a bunch of feature teams.
> >
> > We have 27 Scrum teams. Of those 27 teams, we have what we have 21
> > customer facing teams. They are changing the product our customers
> > use. Because we are so large and our customers can be large
> > organizations as well, we have a group of teams who are mostly
> > infrastructure (build, environment management, etc.) and specialty
> > testing teams (performance, security and product acceptance). They
> > do not change the product but either test the product or support
the
> > other teams work. We have a few framework type teams.
> >
> > Now that most of the decisions regarding business value are
outside
> > of product development then we can concentrate on building strong
> > teams. We now focus on practices, standards and technology.
> >
> > As part of the re-org we brought Jeff Sutherland to teach
everyone in
> > our top management (Senior VPs, VPs and Directors) Scrum and
> > introduce them to how other companies handle their Enterprise
Agile
> > implementation.
> >
> > On the development side, we re-organize our Scrum meetings and
added
> > one meeting for top management.
> >
> > Team members go to Daily Scrum
> > Scrum Master goes to Business Line Scrum of Scrums – Keep Strategy
> > management aware of what the team is working on every day.
> > All Scrum Masters go to All Dev Scrum of Scrums – Scrum Masters
bring
> > impediments they can not resolve every day to this meeting. The
> > meeting is run by a Director who is responsible for working on
> > removing those impediments as soon as possible.
> >
> > Development Management has their own Daily Scrum where we reviewed
> > those impediments and other issues and try to resolve them as
soon as
> > possible.
> >
> > These are all short meetings to bring everyone in sync every
morning.
> >
> > Once a week the management of Dev and Strategy get together to
review
> > how we are doing on our release and discuss any priority changes
or
> > major unresolved impediments (MetaScrum).
> >
> > At the team level, we implemented some changes as well:
> >
> > Get teams to think more about Business Value by putting an
emphasis
> > on business value delivered (tell me when the story is done, not
when
> > the task is completed). Burn down story points, not hourly tasks.
> >
> > Change Retrospective to stop keeping lists of start-stop-continue
and
> > focus on team impediments. Scrum Master keeps list of impediments
in
> > priority order and team focuses on removing one impediment at the
> > time. In essence, we do a more Lean retrospective.
> > Empower teams to manage their talent. Teams decide who joins their
> > team during recruiting process and they can move people out of the
> > team (go to the bench) if the person becomes an impediment and are
> > not helping the team. We evaluate teams, not individuals.
> >
> > One of the recent things we do to constantly evaluate teams is to
use
> > Henrik Kniberg's Scrum checklist which we added the following
section:
> >
> > High Performing team behaviors:
> > Team writes test code and product code together
> > Team runs all their tests as often as possible (at least once
every
> > day)
> > Team keeps tests passing all the time and stops working when a
test
> > fails
> > Team runs their unit tests before check-in their code
> > When a defect is found, the team creates a test case first to
verify
> > defect and then fixes the issue
> >
> > I can keep going, but it will be an even longer posting. We also
> > have a few new changes coming up now like an Enterprise
Definition of
> > Done (EDoD). If you want to contact me directly, we can schedule a
> > call and go over some of your specific questions. I just did an
hour
> > and half call with a company last week to share some of the
changes
> > we implemented.
> >
> > Best Regards,
> >
> > Rafael
> >
>

[ligerly]

Adam,It is interesting to read of the other large agile projects, but there
is one really big one not mentioned yet.
Dave Thomas did this with the Eclipse team delivering a very large system on
time every time (apart form a 3 week slippage due to infrastructure issues,
and amazing record for an annual build for a geographically distributed team
delivering a monolithic product!).  I was amazed at the size of the project
and delivering at a set date! (i haven't had time to blog on this yet).
 Dave recently did a speaking tour in Australia, but I'm not sure if the
material is on the web.  there is a brief description at
http://davethomas.net/talks_index.html  under Lean & agile in the large
cheers,
Erik

On Wed, Dec 10, 2008 at 1:26 PM, Adam White <adam_white99@...>wrote:

>   I'm wondering how well agile scales as the teams get larger. Does
> anyone have any experiences they can share about using scrum on a team
> of 35-40 developers and testers for an enterprise level product?
>
>  
>