By Glenn Mason, Principal Consultant
Continuous testing has gained popularity with elite performing DevOps teams due to its ability to enable a cost-effective way to deliver high quality software at speed throughout the entire delivery pipeline.
Online, you’ll find many definitions of what continuous testing is, but I prefer to keep the definition straight forward and define it simply as: “…testing all of the things, all of the time…”
Continuous testing is enabled by automation and some say that automation should be > 80% to achieve continuous testing. I would argue that to be an Elite DevOps performer, it should likely be higher. This allows you to focus on the areas that can’t be automated and put greater value back into human eyes and human thinking.
Let’s address the robot in the room
It is not uncommon for people to be anxious and somewhat apprehensive about increasing automation in any profession and the move to continuous testing is no different. People fear they will be automated out of a job. I would contend that this argument has been going on since the start of the industrial revolution. While our current information revolution may be increasing the speed of automation, it is also increasing the speed at which we need humans to use their brains for what differentiates us from the robots (automated test scripts). This allows us to take a broader view of quality in our systems and take on higher value tasks. No longer are our product features performing to binary expectations, a pass or fail, but to how the component operates in the context of the broader system. We need to be constantly asking ourselves, is our product getting better for the user? Such questioning has become even more important as many companies are focusing on delivering the highest customer experience to differentiate and compete effectively in an increasingly crowded marketplace.
When exploring continuous testing, I like to explore how testing can be applied using DevOps concepts such as shift left and shift right as a way to maximise efficiency and output.
Shifting Left (into agile teams)
Shifting the testing thinking left brings an increase in collaboration between the tester and the developer. We often see this collaborative work focusing on acceptance criteria definition in user stories. One way to increase continuous testing in shift-left teams may be to add some acceptance criteria in the user story to enforce automated testing to be part of the completed feature. My preferred way is to embrace behaviour driven development (BDD) and use the given-when-then structure as a way of defining user stories. When a team fully adopts BDD, often automated regression testing is done by the developer and viewed to be part of the feature development. Teams who are mature in Agile and BDD will also ensure that automation of the acceptance criteria is a requirement in the team’s definition of done. When BDD is fully embraced in this way, the tester works with the team to ensure there is sufficient coverage of happy path scenarios and anticipated edge cases in the given-when-then structure of the user story. Once this is done, knowing that the automated regression will be taken care of in development, the tester is freed up to spend time looking at other tasks. Here we are creating the capacity to allow the tester to move into higher-order thinking tasks such as exploratory testing and ad-hoc testing.
With high levels of confidence gained from our continuous testing practices, and by taking advantage of our continuous delivery pipeline, we can start exploring the world of shift-right.
This emphasises my initial statement of testing all of the things, all of the time. Often, testing is considered “complete” before the code is released into production. However, to empower teams to ensure quality at each stage (including production), testing beyond the release to production is essential to ensure fast and reliable functionality to users. To enable shift-right testing, we need to make provisions in our code to handle functionality to be toggled on and off for a particular user, group of users or in some instances to the entire user base. This allows us to test the functionality to a controlled group before releasing it to the wider application audience. One additional benefit of feature toggling is the creation of an in-built kill switch, which can be used if something unexpected happens or in the case of advanced product teams, the switching off of unused functionality overtime as a way to manage the size and complexity of an application.
DevSecOps is an area that has been gaining traction for some time over the last 12 to 18 months. While anecdotally, security professionals seemed apprehensive about embracing automated security testing to begin with, they are seeing that shifting appropriate security controls left, into the automated delivery pipeline is giving them the capacity to focus on more value adding tasks. Sounds like a familiar story? In addition, by embracing DevSecOps, we see a mindset shift to ‘everybody is now responsible for a secure product’ rather than, leaving security to the last minute as a checkbox.
In DevSecOps we take advantage of the delivery pipeline to test many things such as software and container dependency vulnerabilities and license compliance. Continuous security testing, using your continuous delivery pipeline, ensures security exceptions are fixed in the context of the work being done. This is a massive benefit to teams working to optimise flow in their systems, as is reduces unplanned work later in the delivery cycle, which is a killer of throughput and any chance of achieving elite DevOps performance potential.
As we can see from this blog, moving to an agile continuous testing mindset not only gives us better quality on the like-for-like tasks that were completed by separate teams, often manually in the past, it also frees up individuals to work on higher-value tasks. By having people focus on this next-level work, we create a virtuous cycle that again, increases quality in the system.
One often overlooked benefit of moving to continuous automated everything is the increased satisfaction in the worker’s day-to-day work. By correctly utilising high levels of automation we will increase capacity in the system. This leads to less burnout in staff as the pressure valve is released, freeing up the humans to do the work in which they excel.
To explore how continuous testing enables elite performing DevOps teams further, watch our on-demand-webinar that looks at this topic in greater detail. Alternatively, reach out to the team via firstname.lastname@example.org to continue the conversation.