After a decade of shipping software projects in various environments, I have started to notice patterns on projects that are more successful than others. The most surprising thing I'm seeing is how actual, day to day project management methodology is not the most important predictor of project success. Instead, the things I observed making or break software projects are pretty universal and little to do with methodologies like Scrum, Kanban or Waterfall-style approaches.
In this post, I'm summarizing the most important principles that I have seen make or break a project*. These are:
- Clarity from the start
- Milestones that are directional
- Transparency on an ongoing basis
- Dependency and Risk Management in a pragmatic way
*needless to say, your mileage might vary. What worked for in my case at tech companies like Skype, Skyscanner, Uber might hold true less in other environments.
Clarity from the Start
Many projects I've seen go south were ones where stakeholders were not aligned. An example is where the legal team discovered a large liability - but only after the feature was built. Back to the drawing board. Or when Team A built a new set of APIs for Team B to use. When it was done, Team B took a hard look and requested so many changes that Team A had to practically redo their work, doubling the time to finish the project.
I have seen few better time investments than ensuring clarity from the very start. A common tool I see used is via a project kickoff with all stakeholders present and involved. By all stakeholders, I mean everyone who will be working on the project or will have some sort of input. So every designer and PM who does the spec, engineers who write code, data scientist, people who will help test, roll out, communicate or market the project. By involved, I mean not a meeting where the project lead talks and everyone else purely listens.
Given this is an expensive meeting, the person leading the meeting typically prepares an overview about the background ("why"), goals ("what") suggested approach ("how") and end state. Showing off designs or other visuals is a great thing at this stage in order to get through to people who are more visual than textual types.
The key for a successful kickoff meeting is the interactivity.
After the brief, a good question to open things up for is "Does everyone understand why we are doing this project, how we will get there and what your role will be to help? Raise your hand if the answer is "no" or "maybe" to any of these.". It's a great sign when questions start pouring in. I've found it good practice to not kick off the project until all major questions and concerns are clarified.
Milestones that are Directional
Now that everyone knows what we are building, why and how, time to get started. However, another common trap I often see is the initial energy runs out soon after the start and the team starts to lose focus. In order to keep clarity throughout the project, as well as an ongoing sense of focus, milestones that help the team focus are something I've seen work exceptionally well.
What is a good milestone? It's an outcome that can be verified and moves the project towards the end goal. It is ideally something that does not take too long to get to - ideally something that can be hit in a couple of weeks in an ideal world.
Having good milestones in place means that the team can be honest with their progress. With good milestones in place, it makes no difference whether the team uses story points, engineer-days or any other way to measure progress. Whether the milestone has been hit or not, and how long it took over original estimates is pretty black and white. Milestones also help the team focus. Teams can decide if they can or will swarm on hitting a milestone, before starting a new one or how they parallelize these.
Transparency on an Ongoing Basis
I have seen countless software projects slip, then slip again. In some ways, it is the nature of crafting software. When building something new, surprises will happen and some of those surprises will translate in delays. However, people are teams are often late to admit to stakeholders when they come across trouble that they cannot fully mitigate.
A very simple tool I see help in creating this transparency is having a regular, no-BS update on where the team really is. For distributed teams, an email to all internal stakeholders can work well. For teams onsite, a team check-in with stakeholders locally present also works nice. This is not meant to be some high level, carefully composed status for upper management. It's meant to be a quickly thrown together description on where the team is against the milestones, what the upcoming goals are for the next period and what progress was made the last period. All team members are part in delivering/writing this update. This serves as a continuous reality check on what is actually happening, close to where the real work is going on: the engineers themselves.
A lot of projects slip because team members are not even honest with themselves.
"I'll finish this task by Friday", says an engineer and thinks "I hope, that is, it's more difficult than I expected. Oh well, I'll just work on the weekend a bit if I'm late.". By creating a culture where people commit to short-term goals, then hold themselves accountable and check in with each other helps keep both feet on the ground.
I have found transparency on the progress of the project to be a powerful tool in helping projects succeed, for two reasons. First, a team that is honest with itself tends to focus better when things are at risk. In teams that don't have much feel on how well they are doing, engineers will often cherry pick the cool or interesting task. In a team where everyone is aware of how good (or bad) things are going, people will pick up work that can help the team the most.
Second, being honest with immediate stakeholders creates a lot of trust over time. A lot of business stakeholders don't have much understanding or appreciation of what is easy or hard about software development. By exposing them to more granular details and helping them understand what tradeoffs the team is continuously making helps build empathy and more realistic expectations on both ends.
Dependency and Risk Management in a Pragmatic Way
For projects that are late, I noticed two common reasons. First, something along
"We did all that we could ahead of schedule, but we were blocked by {dependency X} and could not do a thing about it." This is a symptom of poor dependency management. The other thing is more like "We were on track, but then {big, unexpected issue} happened that took us a lot longer to resolve than estimated". This is a symptom of poor risk management.
Despite these being so important for most projects I've seen, most popular software project methodologies barely mention dependency or risk management. People practicing things like Scrum or Kanban to also think a lot less of these areas and don't do it early enough. More heavyweight project management methodologies like PMI do have sections dedicated to managing project risks. To me, this approach seems too bloated and abstract for day to day software development, so here's something a lot simpler of what I have seen work.
For dependency management, use the same principles as with good delegation. There are people who are good at delegation, and people who are not great at it. However, good enough delegation is a totally learnable skill. Same applies to dependency management.
- Discovery. Figure out who your dependencies are and what they need to do.
- Agreement. Talk to them and agree what they will do and by when.
- Check-in before the due date. For teams that you don't have a good track record working with, do more frequent check-ins, to make sure they are on track.
- Give feedback and/or escalate. Once the work is complete, give feedback. If your dependent team did a great job, call this out clearly. If they did not do a good job, consider understanding why. If they are really late, consider escalating earlier, rather than later.
For risk management, have a culture that rewards raising concerns early on and be pragmatic in tradeoffs to mitigate risk. Most of the risk in software projects is discovered by engineers, on the ground. The best engineers love problem-solving and are naturally optimistic. So when they come across a problem, they see a challenge to solve, not a potential delay to the project.
To tackle this, create a culture of talking about interesting challenges coming up on a day to day basis. Start rewarding people who flag things that might take longer to do and bring tradeoffs to the table.
As a rule of thumb for software projects, between time, feature and people, to undo a negative change in one of the dimensions, another one also needs to be changed.
Whenever risk comes up, consider reacting to it ahead of time. Lots of unexpected work creeping up early on in the project? Perhaps it's time to cut scope or pull in more people, perhaps pull in more experienced people. Is scope creep happening, with more features being added to the project? Communicating the tradeoffs and deciding between fixing the scope, pushing back on timelines or adding more people early on in the project will go a long way in de-risking the work.
Finally: Get It Done, Learn and Celebrate
As software teams, project management and all the fluff around is never the goal itself. The goal is to get things done, delivering some specific value to the business. There are many ways to get there, the "optimal how" being different and ever changing in every environment.
When we eventually do get things done, looking back and figuring out where we can do better next time is a key part of individual and team growth. And of course, celebrating a big achievement - whether it was done through blood, sweat and tears or like a hot knife through butter - is a final, really important step. This last step really helps build a cohesive team who will be ready and hungry to deliver on the next, more complex project, in a way even better than this last one.
Subscribe to my weekly newsletter to get articles like this in your inbox. It's a pretty good read - and the #1 tech newsletter on Substack.