Shifting Left: How Operations Can Bring Security Into a Process Earlier
As an Operations pro at a cloud security company, I have a unique perspective on how security and operations can—and should, in an ideal world—work together. One common issue that we see in the tech industry today is that security is almost always brought in too late in the development process.
It can feel like security is a separate discipline entirely, and incorporating it into tight DevOps feedback loops may seem daunting. But I’m here to tell you that not only can it be done, but it’s not as difficult as it looks—and doing it right will make a big difference in your overall security posture while removing a host of pain points.
In particular, the concept of “shifting left” allows teams to take security processes they previously did much later in the deployment process (if at all) and introduce them earlier to ensure that security is built in, rather than as an afterthought. In this post, I’ll explain how to do this.
What Does “Too Late” Mean?
First, let’s take a quick look at how security is brought into DevOps processes today. Often, it’s just not. This manifests as engineers deploying code whenever it’s ready and not including security in the process at all. Of course, the developers who do this are just doing their jobs. They’re under pressure from product teams to release features, and they’re simply doing what they have to do to get the code out as fast as they can.
In other cases, security teams will get injected into that flow and become the gatekeepers, requiring a security review before code can go out. It’s good to see security folks being pulled in and code being reviewed for security flaws. But waiting until the last minute can hamper continuous delivery cycles, and in today’s fast-paced climate, that’s often not an option. Not to mention, it can lead to some real tension between these teams by setting them up in opposition to one another.
How to Start Shifting Left
Over the last few years, more and more organizations are optimizing their DevOps teams to move faster and deliver software on a continuous basis. But it’s time to bring security into the fold so here are my tips for doing that.
Use the Same Tools
It’s important to get your security teams familiar with the same continuous integration and delivery tools that your DevOps teams use. This is how Dev and Ops started working well together in the first place—by teaching Ops people how to code and developers how to use configuration management tools like Chef. It’s time to do this with security.
Jenkins is a great example. If your developers are already using Jenkins to test and deploy code, then why not teach your security team to use it too? If they use the same tool for security testing, it’s much easier to bring them into the process early on.
Another great tool is called Gauntlt. While I have some reservations about the term “rugged DevOps,” the idea behind Gauntlt is a good one. From their site:
“Gauntlt provides hooks to a variety of security tools and puts them within reach of security, dev and ops teams to collaborate to build rugged software. It is built to facilitate testing and communication between groups and create actionable tests that can be hooked into your deploy and testing processes.”
The concept of inserting security into the middle of your operational and development processes is fundamentally sound. This way, security doesn’t slow delivery down, and you can tighten up the feedback loop when an issue is uncovered. Security teams can write code using Gauntlt and test it before it goes out, facilitating successful, continuous delivery.
Try Static Analysis
Another option is to try out static analysis using a tool like Veracode. Then your security teams can analyze code before deployment and try to catch potential issues. Static analysis essentially analyzes software without executing the code. Security teams can use it to check for potential vulnerabilities in an automated fashion, without interrupting the software delivery cycle.
Align Incentives
Historically, one of the challenges with getting DevOps and security to work together has been that they have conflicting incentives. DevOps wins when they get code out fast. Security only wins when secure code goes out.
DevOps teams and security teams work better together when they use the same or compatible tools, as explained above. But to take things a step further, you also need to make sure that you align incentives across these teams. That means rewarding DevOps for releasing secure code and rewarding security for moving faster.
Provide Ownership
One common mistake we see is teams where security does not even have access to production. If security has no direct ownership over the code, you can bet they aren’t going to be tightly integrated into DevOps cycles.
On the other hand, when you give security folks ownership and teach them how to resolve issues in the code on their own, they are able to become part of a well-oiled machine instead of a silo. For example, Threat Stack can alert your team about security issues. If you teach security teams how to resolve those issues and give them access to configuration management code, they can go in and make system changes directly.
Similarly, Chef has open-sourced a tool called InSpec, which is an auditing and testing framework. With InSpec, security teams can write code to audit servers and ensure compliance. This, again, gives them far more direct ownership over the process. Empower security teams to level up their DevOps skills, and you’ll see the proof in the pudding.
Another way to approach this (especially if you don’t have a dedicated security team) is to put backend engineers in charge of fixing code when security vulnerabilities arise. When engineers are directly responsible for the health of their code, they are much more likely to focus on building it securely in the first place. At Threat Stack, we have 2 Ops people and 10 backend engineers on call. So when an issue arises with the product, it’s usually the backend engineering teams who wrote the code that are called on to fix it. This level of code ownership incentivizes them to build it right in the first place.
The Way Forward Is Sideways
Shifting security left is better for the overall health of your business. Rather than security teams catching issues, opening tickets, and waiting around for developers or Ops pros to fix things, they become empowered to do it themselves. This not only saves time and manpower, but ensures that code can be released both continuously and securely. It’s up to folks on all sides of this equation to become more proactive about learning how other teams work and how they can be more tightly integrated into those workflows.
The tips above should give you a starting place to make “shifting security left” a reality at your organization. Remember, as with many aspects of security, it does require a cultural shift (not just bringing in new tools.) But it can be done with the right mindset, incentives, and feedback loops in place.