What waterfall taught me about agile

I worked for one of the largest banks in Brazil back in the day, with everything that entails, waterfall processes, PMBOK training, commercial proposals, weekend-long debugging, wearing a suit and tie. I’ve learned a lot, and don’t regret doing any of it.

Nowadays, waterfall is used as a cautionary tale. It’s the project management process that we don’t speak about. Back when I was in the middle of figuring out long-running projects that ran inside of the banking system, for a bank with tens of millions of customers, it was just how things were done. PMBOK and waterfall are the way projects are executed in most businesses, so naturally a bank, whose main concern is about making sure that everyone’s money is secure will lean towards it.

Over time, I had the chance to work at multiple different companies and experience agile methodologies with different levels of maturity. They all shared the same goal: iterate over a product, with a clear vision of the end goal. For these projects, agile methodologies worked great, they helped organise and frame deliveries, while staying close to product decisions and feedback.

This combination allowed me to call-out a group of senior managers and directors on how we were actually doing mini-waterfall projects, that lasted roughly a quarter.

What did working for a bank look like

When I joined the bank, a big project that sent emails to millions of customers on a daily basis was being built, following the waterfall strategy. The full project took more than 1 year to get built, and the developer who was leading the project went to the bank on a Saturday morning to deploy the project in production, and came back on Monday. It was the first deployment in the production environment, which meant a lot of moving parts. I was a bit shocked by the whole situation. It was my first job after all.

After a few months, the lead developer moved on to another project, and I took over as the lead developer. That’s when the fun started. I was responsible for the technical execution of the project, with the assistance of a consulting company that did part of the development. But I was also responsible for aligning with the product department at the bank, which meant multiple phone calls every week.

Every time a new project came along for the emailing product, we had to create a detailed project plan. So, I learnt how to use Microsoft Project, calculate consultant costs, do risk analysis, which would produce a commercial proposal to be sent and approved by the bank department responsible for it. Then, the implementation would follow with a mix of internal development and outsourced development. When the time to deploy would come, we had to go through a long approval and planning process, which took around 2 weeks every time.

As the time went by, the time to do performance optimisations in the web server came. This brought another unusual aspect forward, the only way to be able to ssh into the servers to do diagnosis was going to the operations room in the bank. The operations room was in an underground level of one of the bank’s buildings, one of those rooms with rows of computers with people facing huge monitoring dashboards on a wall, running 24/7. Going there required getting special permissions. The first time I went there was like going into a movie set.

I was also responsible for making sure the system was running 24/7, because a lot of the batch operations happened in the middle of the night to not overload the systems during the day. These were both COBOL routines and regular Java services. At one time, I got a call Saturday morning, around 05:00 because there was an error running the batches. This meant jumping in my car and driving directly to the bank’s underground room. I called my manager, got the needed permissions and went to the underground room. During the weekends the room was emptier and not so much was happening. The problem was hard to diagnose and fix, which meant coming back home only at Sunday night. On other occasions, I left early in the morning, after spending the whole night debugging and deploying. And as I waited for my cab, the executive board was coming to work in their helicopters.

When waterfall is not a villain

The main things that agile improves is the ability iterate fast, deliver small batches, and slowly progress towards the final product. For the bank this didn’t matter, the project scope was set and defined 1 year in advance, it went through all the approvals needed. The 2-week long process would make fast iteration impossible, and the bank wouldn’t allow the deployment of small unfinished batches in production. Also, the mainframe teams that writes COBOL code moves much slower than we are used to.

In this scenario, waterfall taught me a lot. The scope was not measured in fictional story points, it was measured in money. How much would it cost to build a project, and this had a long approval process, with an estimated deadline that could be a year in the future. And the bank would expect us to meet this deadline. Not meeting it would trigger adjustments, another round of approvals, possibly fines. I also understood risk analysis with higher stakes, an underestimated risk could derail the full project.

Moving past waterfall

Many years later, in a modern company that had an agile process in place, with all scrum rituals, I recognised some familiar patterns from my waterfall past. We would do quarterly planning meetings, in which the full delivery of the quarter was decided, with the business metric as the sole goal for the quarter. The delivery was composed by unrelated features, meant to drive a number higher, no unique vision, no single goal. The engineering team struggled to relate their work to company goals, and customers needs, everything felt top-down. This was not about iterating products, this was about building a set of features over a quarter.

Naming the pattern

Once, during a quarterly plan meeting with directors, senior managers, and managers I called out this pattern. I said: “We are not doing agile, we are doing micro-waterfalls”. At that moment, the room went quiet because they were not expecting me saying that at the time. This also meant that nothing changed in the upcoming months. A few months later, I suggested changing from Scrum to Kanban as an experiment. Kanban really works when the team is moving tickets, instead of iterating. I documented my proposal and got buy-ins from directors, senior managers, and managers both from product and engineering.

The experiment was a success, other teams in the track tried the same approach with my help. The team struggled to find a goal to the sprint. So instead of trying to fix the fact that Jira didn’t allow for multiple goals natively we focused on what mattered, delivering product features and measuring the commercial goals. A few years later, the company started an official agile training, focusing on ceremonies (including setting sprint goals), metrics, parallel work limits, and how retrospectives can help improve the process itself. I won’t pretend to have anything to do with the training, this was decided at much higher levels, but I saw the pattern way before, thanks to my time at the bank doing waterfall.

The lesson learned

I had the chance to witness other engineers questioning processes and suggesting improvements. But it always comes from having the experience with different approaches, methodologies, and domains. They don’t treat the methodology as their identity, they treat it as a tool, and change the tools when the work requires a different one, with intention, guardrails, measurements, and exit paths.

That’s what waterfall taught me about agile: the ability to see when a process has stopped fitting the work. You can only see that clearly if you’ve been inside more than one process and paid attention to what each one was actually good at.

I still don’t regret the PMBOK training.