The role of the build engineer is undergoing a transformation. Build engineers are becoming the key to success for any software producing company. But this transition is not happening without a lot of pain.
Traditionally the responsibility of a build engineer was to create and pass on artifacts from development to production. More and more the focus is shifting to improving developer productivity and enabling fast and frequent shipping of software.
Companies that rely on software to run their business might find themselves at a competitive disadvantage (and even out of business) if they do not aggressively optimize developer productivity. But across the industry, developer productivity is much lower than what it should be.
Software stacks are becoming more complex, and the code bases are growing which, without taking any actions, will slow down your builds and make build failures harder and harder to debug. At the same time, there is great pressure to become more agile and release fast and often, which requires fast feedback cycles and thus fast builds. This friction and frustration start hurting the team. Why?
Over the past several years as founder and CEO of Gradle, I have met with hundreds of build teams. In this post, I want to discuss root causes of this pain — both organizational issues and how technology gaps work against the build engineers and entire development team.
Common organizational anti-patterns
Build teams are understaffed and considered 2nd class engineering
In many organizations build engineers are in a vulnerable position, without the resources to do a good job.
Traditionally build engineering has been considered a less important engineering discipline compared to application development. This function is often a scapegoat for developer productivity problems, often for behavior that is not their responsibility.
Developers often change the behavior of the build and yet the build engineers are made responsible for the result. The build is slow — but did the build engineers introduce the language with a very slow compiler? Who wrote the slow tests and who publishes dependencies that always break downstream teams? Not the build team!
But in this case, the build engineers don’t have a strong standing organizationally and also don’t have sufficient data to explore and show what impacts the build and who is accountable. How the builds behave on local machines is a complete black box.