Self-organization as a concept and a phenomenon has its root in systems theory. It is a key characteristic of a complex adaptive system.
In a complex adaptive system, a large number of diverse component parts interact and form a cohesive, robust, and resilient whole that exhibits emergent properties and achieves higher order in the absence of intervention or control from external forces.
Natural scientists first discovered these phenomena in natural systems such as a colony of ants, a forest, an ocean.
Engineers attempting to create better, larger, and more complicated systems such as a satellite, a power plant, or an enterprise information system apply the lessons and principles learned from the natural systems to improve the structure, design, and functions of the engineered systems.
Social scientists also find similar phenomena in social systems such as a community, a stock market, an economy and follow the suit by applying scientific knowledge and engineering techniques to effect social changes and to improve human conditions.
Self-organization enables a complex adaptive system to evolve from an initial chaotic, dysfunctional state into a cohesive, functional state. Self-organization also makes a complex adaptive system more robust and resilient to perturbations and uncertainties from its changing environments.
Now let's get back to our original question: Can a scrum team achieve self-organization? or phrase it in a different way: Is a scrum team a complex adaptive system?
A team consists of members who interact with each other to achieve a common goal. In traditional project management, a team is led by a team lead or a project manager and has well defined roles and responsibilities and top-down command and control hierarchy. Clearly, this type of team does not conform to the definition of a complex adaptive system which is a bottom-up grass-root organism without controlling forces imposed from the above.
A scrum team by definition consists of a scrum master which is a servant leader, a product owner which is the liaison of the end users and the champion for the end products, and a development team whose responsibility is to design, build and deliver the products to the end users in an incremental and iterative fashion.
The size of the development team within a scrum team can vary and even the guideline varies as well. Some say 5 plus or minus 2, others say 7 plus or minus 2. The rationale here is that the development team should be cross-functional in that all necessary disciplines are represented (for example, designers, developers, testers) but at the same time should have minimal number of people in order to achieve efficiency and effectiveness in communication and collaboration.
This reflects the emphasis of individuals and interactions from Agile Manifesto:
"Individuals and interactions over processes and tools"
This also reflects the emphasis of face to face conversation from agile principle #6:
"The most efficient and effective method of conveying information to and within a development team is face-to-face conversation."
This results in a overall scrum team size of less than a dozen people. In practice, the average scrum team size is even smaller. a complex adaptive system typically have large number of component parts. In another words, critical mass is needed to generate the dynamic behavior and to allow for the resilience and robustness to develop over time. Small group may be easier to communicate and collaborate, it does not generate the critical mass.
Another aspect of scrum is the emphasis of the role of the scrum master. A scrum master is the opposite of a traditional project manager or team lead in that it does not provide management function of command and control. In contrast, it is a servant leader and its primary goal is to protect the team from external influence, to facilitate communication and collaboration, to remove impediments, and to enable the team to focus on the tasks and deliver the products. In one of the agile projects I have worked on, we call the scrum master "Super Mom" to reflect the similarity of the role and the same initials (SM). This makes the scrum team somewhat like a closed system instead of an open system which a complex adaptive system is meant to be.
From the above, we can clearly see that agile methodology and scrum process put some structure and constrain around the makeup and function of a scrum team in terms of size, roles and responsibilities, and interaction with external entities such as end users, subject matter experts, and stakeholders.
The structure and constraints find its roots in the traditional project management and make the scrum team less a complex adaptive system.
Additionally, several characteristics of a complex adaptive system are missing in a scrum team.
First, a complex adaptive system typically starts in a organic way in that component parts are not "hand picked" by an external force but rather are naturally formed together organically such as a flock of birds or a schools of fishes.
Secondly, component parts within a complex adaptive system are typically equal and diverse without supervisor-subordinate relationship and clear distinction of roles and responsibilities. They are sometimes called agents. The emergent properties of a complex adaptive system come from the free and seemingly random interactions among the parts and also with the environments.
Lastly, it takes time for a complex adaptive system to form organically and to transition from chaotic state to a harmonic state. The time boxing nature of scrum and the limited time duration of projects do not provide the time needed for a scrum team to adapt and grow naturally.
The conclusion is that a scrum team is not a complex adaptive system and cannot achieve self-organization naturally or organically.
This is why scrum teams struggle to self-organize.
