Why is culture the biggest impediment to Agile adoption?

And what can we do about it?

Photo by Aaron Greenwood on Unsplash

The 2022 State of Agile survey, like many before it, states: “statistics show that Agile practices frequently clash with company culture. In fact, company culture is listed as the leading cause of unsuccessful delivery with Agile”.

Organisational culture can probably be defined most simply as “the way we do things around here”. More specifically, it comprises the behaviours, habits, beliefs, language, rituals, symbols and more that, together, are unique to that company. It is their identity. So what is it about that identity that make it clash with agile practices so often? And why is it so hard to overcome it?

Perhaps the problem is that we believe a company’s culture has to be in alignment with agile practices (an agile culture) in order for the transformation to be successful. Were that to be the case, agilists would have long ago defined a model for a conducive culture, and a framework for how to create that culture through an agile transformation.

Of course, that hasn’t happened. Mainly, I suspect, because we just want to sell them an agile.

A companies culture is its very identity

Organisational culture is complex and is never consistent across an entire organisation. For example, the culture will be very different in HR when compared to R&D or Sales, Finance or Technology. A Software Development department will necessarily have within it different cultures depending on leadership style, the products or services they produce and for whom, geographical distribution etc, etc.

It is therefore illogical to expect a specific culture to exist when commencing an agile transformation. In fact, the reverse is probably true. An agile adoption, if successful and sustained, will necessarily change the organisational culture. But it’s apparently the culture that stands in the way.

Which brings us to two key questions:

• Why is culture seen as the leading cause of agile ‘failure’?
• What can we do to avoid this conflict?

Why is culture seen as the leading cause of agile ‘failure’?

Culture is complex and includes things like values and beliefs, behavioural norms, traditions and rituals, jargon, leadership style and structure. Culture is the organisational identity, and just like your own personal identity, you’re not easily separated from it.

It is common among organisations seeking to adopt or improve their agility that such a transformation appears, to some, to comprise the imposition of certain practices (e.g. ‘more’ meetings, backlog refinement), roles (e.g. Release Train Engineer, Project Managers re-labelled as Scrum Masters and Product Owners), and tools (e.g. Jira, Miro, Rally) on development teams.

It is easy to see how these might clash with a team and departmental culture that values control, documentation and rigid plans over autonomy, working software and frequent releases. Worse, it is also not unknown for coaches to deride or denigrate existing tools and practices as ‘wrong’, ‘old’ or ‘legacy’, which simply serves to escalate resistance to change. No-one likes their culture being insulted.

Surely it is foolish to expect an organisation to change years — even decades — of deeply held beliefs, traditions and practices in a short space of time simple because… “agile”. When Management has always used spreadsheets and slide decks for planning, when contractual relationships between business and Tech teams is the way it has always been, when changing the plan is something to be avoided, change is hard.

What can we do to avoid this conflict?

If we hold the attitude that we as agile coaches know what is right and best for the organisation, and what they are doing is wrong, then meaningful, lasting change will be exceedingly difficult to achieve.

We do not change our habits because someone tell us we should. Try telling someone they should stop biting their nails or smoking and see how far you get. We change our habits only when we find for ourselves a compelling reason to change.

So the first thing we need to do is:

Respect what has gone before. If meaningful change is going to occur, we have to respect what has gone before: the existing roles, power structures and events. They may not be what we want, but they were originally created or adopted for good reasons.

Don’t tell people they are doing it wrong; don’t even tell them you have a better way of doing it. Instead, try to understand why they are doing it that way. Challenge their thinking. Ask if that’s the only way they’ve ever done it. Ask if they have considered alternatives. Look for the emotional attachment to whatever it is that is reinforcing the existing habit.

Treating your clients with respect is essential.

Now, I present to you two strategies for how to tackle this culture thing:

Know what you’re getting into (optional).

We could first try to identify the existing culture/s within the organisation we are working withThere are numerous models or frameworks available for assessing an organisation’s culture, such as:

• Denison Organisational Culture Survey: Developed by Daniel R. Denison, this survey assesses organisational culture based on four key traits: Mission, Adaptability, Involvement, and Consistency
• Competing Values Framework (CVF): Developed by Cameron and Quinn, the CVF identifies four organisational culture types: Collaborate, Create, Control, and Compete, or Clan, Adhocracy, Market, and Hierarchy
• Schein’s Three Levels of Organisational Culture: Developed by Edgar Schein, this model distinguishes between three levels of culture: Artifacts and Creations (visible symbols and behaviors), Espoused Values (explicitly stated beliefs and values), and Basic Assumptions (unconscious, taken-for-granted beliefs)
• Organisational Culture Profile (OCP): Created by O’Reilly, Chatman, and Caldwell, this model assesses seven cultural dimensions: Innovation, Stability, Respect for People, Outcome Orientation, Attention to Detail, Team Orientation, and Aggressiveness

We could use any of these before commencing an agile transformation to assess the client organisation’s existing culture, and compare that to some theoretical ideal, then tackle the differences.

Let’s use as an example the CVF:

If you were to assess the culture in your own Department or Division, Product Line or Value Stream, what would it be like?

It would have elements of all four types of culture in varying degrees, right? Now, what if we wanted to map out the culture you really wanted in your organisation? What is wrong with the current one? If for example, you wanted a bit more Clan and Adhocracy, a bit less Market and a lot less Hierarchy, what could you do to make that happen? And how would that ideal then match what you’d expect an ideal agile culture to be like?

But then, what is our priority — changing culture or adopting agile?

There is value in assessing organisational culture in advance of a transformation, and using it as a transformation strategy — a way to make the adoption of new practices, roles, and processes a little easier. If we can explain, for example, the logic behind smaller, more frequent releases as being more Market (customer-driven), that would help.

Adopt Outcomes, not Practices (alternative)

You are, by now, (hopefully) aware of the complexity of organisational culture and its impact on agile transformations, and you recognise that the problem lies in the imposition of new practices, roles, tools and processes that clash with existing traditions, beliefs and values.

I think we have established that, to rely on training and coaching teams in agile methods and frameworks can only take you so far, and statistically it is likely to fail. I think we can also agree that, while there is merit in assessing and using the organisational culture as a vehicle for transformation, it is complex and as yet unproven.

As an alternative, let’s keep in mind that while we try to get the organisation to become more agile, a lot of the changes we are expecting will clash with existing culture and make adoption much harder. Let us also remember that the more detailed the methods, practices and processes we are proposing, the greater the likelihood of cultural clashes.

So instead of mandating adoption of a specific framework, we could instead agree on specific general rules for agility and allow teams to evolve and continually improve their own practices, roles and tools to achieve these outcomes.

Step 1: I suggest a general transformational approach that starts, not with agreeing to adopt Scrum or SAFe (other frameworks are available), but with an agreement for the outcomes of the transformation initiative.

As examples, we could agree that:

• time-to-market for new product features should be less than n weeks
• customer-reported defects should be less than n per week
• customer satisfaction score should increase to and maintain at least n%

I’m sure you can come up with a load more, but you get the point. What are the measures by which you know the agile transformation is succeeding or has succeeded?

Step 2: Agree on a minimal set of Lean/Agile principles or rules to deliver these outcomes. I call this my Minimum Viable Framework (MVF) for agile adoption. The idea is to agree the shortest possible list of the things the organisation needs to do in order to meet their objectives. By establishing high-level agreement to these, we avoid direct clashes with organisational culture. Senior leaders very rarely disagree with these basic rules.

As an example, I recently suggested these four elements to a client:

• Prioritise by Value (meaning understanding and articulating value so that backlogs are prioritised by decreasing feature-value)
• Visualise all the work and the workflow (to enable identification of delays, waste, quality, handovers, dependencies, etc)
• Eliminate waste (as per Lean & Kanban)
• Limit work-in-process (to improve cycle-time, identify bottlenecks and manage flow)

Try to keep the list as short as possible, making it easy to remember. Communicate it over and over and over again through multiple events and channels. Make it very easy to find and refer to. Make this the “north star” of transformation efforts.

Step 3: Create training and coaching opportunities and capacity to help teams understand their MVF and help them find ways to make it happen. Empower them to change their own processes and practices.

Create KPIs (ideally, leading and lagging metrics) to measure these elements and ensure that they are improving. Publish and celebrate improvements regularly.

Summary

Existing culture is cited as the leading cause of agile failure because of the clash between new detailed practices on the one hand, and long-established organisational values, traditions and beliefs in the other.

We can either map both the organisation’s existing and ideal cultures and use them to drive transformation, or we can avoid the conflict by agreeing outcomes and empowering transformation from within.

More on the MVF in a future post, but for now I would love to hear your thoughts on the ideas expressed here.