Good software development is a lot like building a house – you need to build a strong foundation before you even start thinking about the marble countertops.
The less time you spend on building and verifying the integrity of the foundation, insulation, plumbing & electricity, the more issues you will run into later.
If you could choose between contracting a plumber for two weeks during the building stages, or a flooded basement after you moved in – which one would you pick?
Infrastructure can be a very broad term, so let’s get specific.
When we talk about infrastructure driven development, we are talking about smart, well thought-out architecture that takes into consideration important decisions like the software stack, data design, compartmentalization of services, marshalling/unmarshalling between different parts of the application, third party integrations, and more.
Perhaps most importantly, infrastructure driven development is a way of critically thinking about the current problem space, and the variety of challenges, new features, and curve balls that will inevitably be thrown our way in the future.
What infrastructure driven development is not, is building out features first and connecting the dots later. That leads to common issues like technical debt, scaling difficulties and inability to keep up with the product roadmap – the technology starts holding the business back instead of propelling it forward.
By investing in the infrastructure and laying the groundwork, we’re preparing the software solution (and ourselves) to respond instead of react.
Software development does not happen in a vacuum – the external push & pull forces of customers, marketing teams, and executives means there are always trade-offs when project management decisions are being made.
In companies that do not follow infrastructure driven development (or haven’t been burned enough times by the absence of it), features very often take priority over things like infrastructure investment and technical debt reduction.
New features with observable outcomes and immediate ROI are valued over changes that happen beneath the surface, with no clear benefit to customers. This isn’t surprising - good software infrastructure is invisible to non-technical people – but a failing underlying infrastructure is immediately obvious to everyone.
Just like plumbing in a house, nobody is thinking about it when it’s working appropriately, but when your basement starts flooding and destroying your furniture, the issue is front and center in everyone’s mind.
The classic symptoms of an underdeveloped or failing software infrastructure are: seemingly simple new features requiring extensive development time, regression bugs constantly popping up, and an overall feeling of the technology team that they can’t keep up with product.
In these scenarios hiring additional technical resources might seem like a straightforward solution, but there’s only so many plumbers you can hire to speed up the repair of a burst pipe.
And the worst part is – you don’t know when the next pipe will burst. You know it will, but when and which one?
But why, you might ask, does infrastructure need to come first? Why can’t it be developed along the way, in parallel with new features?
Like we mentioned before, infrastructure driven development includes important considerations like the software stack, data design, compartmentalization of services, marshalling/unmarshalling between different parts of the application, third party integrations, and more.
Without having critically thought about these topics and made informed decisions, it’s very difficult for a team of developers to build a product in a logical and consistent way. Changes to any of the underlying software infrastructure decisions usually has far reaching consequences.
That’s why when technical teams are facing a failing software infrastructure and a new feature request, they very often have to decide between “hacking it” and further increasing the technical debt or rewriting a significant amount of code. Either way, the efficiency and output of the developers is frustratingly hampered.
There’s another good reason for why you should build infrastructure first and features second – optics.
Let’s say your company is planning on building a new product, and you’re asked to put together a budget for the MVP – in scenario A, you think that without investing a lot of resources into the infrastructure, you can build it for $200,000.
Your budget gets approved, you build the product, launch it, and it’s having some early success.
You then get a request for another quote – adding 5 small new features.
If those 5 small new features are a logical progression of the MVP, but they will require a budget of $150,000 because a codebase rewrite is going to be required, someone’s going to be mad at you, and rightly so.
However, consider scenario B – if you had first thought about putting the upfront investment into the infrastructure, and gotten a $300,000 budget approved – then adding those 5 new features may only cost you $50,000.
The net cost for each scenario is exactly the same, but the optics are vastly better in scenario B. It’s better to build it right the first time than to hit technical snags right when the MVP is starting to see success.
The idea of infrastructure driven development doesn’t necessarily contradict lean startups or rapid prototyping – they are necessary methodologies and strategies to get quick wins.
When we are working with a client it’s important for us to know their current market position, product roadmap, and in general have a very clear understanding of where they are and where they want to go. That information helps inform our product and technical strategy, and plan of attack on how we want to help them build and succeed.
Again, just like a house – the size of your foundation will depend on whether you’re planning to build a two-story house or a thirty-story condominium. And if you’re building a shed in the backyard – you don’t need a foundation at all!
The purpose of keeping infrastructure in the back of your mind is to understand the limits it places on your development team, and the risk you take on when you knowingly exceed those limits.
