jade rubick

The fundamental challenge in building engineering organizations is complexity

2020-09-24deliverycompany-culturecomplexityagilescaling

Delivery is the yardstick that engineering organizations are measured by. After all, if we’re not creating and maintaining something valuable in the world, what’s the purpose of our work?

Ward Cunningham has a wonderful way of describing the heart of agile software development. He compares it to the pedal of a wheel. Every turn of the wheel…

  1. You deliver something of value to your customers or the business.
  2. You do something that makes the next turn of the wheel smoother.

The purpose of this is to get a virtuous cycle going, so that your work gets easier and faster over time.

The challenge with growing startups is that this process is at odds with the exploding complexity of the software and organization you’re building. The increasing complexity conspires to gunk up the wheels and slow you down.

What are the problems of complexity?

  • More and more decision-makers. More effort to even understand how decisions are made.
  • A larger and more complex codebase to navigate
  • More teams to understand what they do
  • More dependencies between teams
  • More relationships to maintain
  • More work with people you don’t have relationships with (which requires more effort to navigate)
  • Multiple ways of doing things
  • More people to get on the same page
  • More leaders to second-guess you
  • Less effective meetings (if there are many people in the room)

What are the things you need as the organization grows?

  • Redundancy (in skillsets) to reduce the cost of vacation or departures on the whole system
  • Knowledge sharing to allow people to do greater areas of work
  • Architectural cohesion to reduce unnecessary complexity
  • Platforms to reduce duplication and accelerate velocity
  • Standards to simplify the decision space
  • Tenets to simplify decision making
  • Team definitions to narrow the problem space
  • Better definitions of the flow of work, so people are better coordinated (and so the amount of work is limited)
  • Role definitions, so people are better able to work together
  • Communication design (forums for communication, written processes, deliberate design of the spaces people communicate)
  • Self-service, so teams can be more decoupled from each other
  • Cross-functional teams, so less coordination is needed to build something valuable
  • About a million other things

You can think of growing an engineering organization as almost a mathemtical exercise, where you see growth in complexity in what you build and who you hire, and you reduce complexity with all the activities you take to streamline things.

A warning: one way of reading the Innovator’s Dilemna is that the company with the most complexity almost always loses. So you have to build your organizations very, very carefully.

Photo by Gavin Mills from FreeImages