Transitions to DevOps ... A Panel Discussion

Last month I was invited to speak on a panel at the Boston DevOps Meetup discussing the changes in roles moving from a traditional software development organization to one embracing DevOps. Development, QA, and Ops were represented (video below staring at the 48:10 mark). A spirited panel ensued with a great number of discussions post panel at the bar (fortunately not captured). Give it a watch if you're into that sort of thing.

Where Do We Go From Here?

Soundtrack

Image "Which Way Now?" by Nick Page CC 2.0 Attribution

In my last post a talked about the changing nature of the QA department in the face of the upswing of Agile / DevOps in software development. So that inevitably leads to the question "Where do us testers and QA folk go when the separate Testing/QA Department goes away?"

The good news is that the characteristics that make up a good tester - curiosity, desire to check assumptions, empathy for the end user, a knack for breaking things and passion to make software and processes suck less - are valuable beyond the scenario where software is hurled over a wall at testers. Just because a department called "QA" doesn't exist doesn't mean that testing isn't happening. In fact "testing" or "QA specialists" can leverage their expertise to greater effect outside a dedicated QA department.

Automation / Test Infrastructure

The "A" in CAMS stands for automation and a mature DevOps release pipeline relies on robust and efficient test automation. As part of an Agile development team a person with coding skills specializing in testing is valuable asset. They can assist in developing unit tests as well as creating and maintaining test frameworks for use with continuous integration to help find errors earlier. They can also help determine if the team has clear and testable stories and proper acceptance criteria for features (that will be automated). Finally they can help the team keep in mind quality aspects as part of the "Definition of Done" for an iteration.

Specialize

Another option is to specialize the type of testing by developing deep knowledge. Both security and performance testing remain important for deploying applications at scale in the cloud. Knowledge of how to do various types of stress and load tests and benchmarking can give the business the certainty that their application will scale when it counts. Skilled penetration testing can give the business the confidence that customer data will be secure when the application is deployed to the public. 

Test Infrastructure

The implication of "Infrastructure as code" is that code still needs to be tested, so there will be a need for people with code testing skills to test the infrastructure and deployment automation frameworks. QA folk with operations skills or experience would be a valuable addition to teams tasked with maintaining infrastructure.

Manage the Process

An efficient continous delivery / deployment pipeline relies on a mature and efficient software engineering process. This process is continually improved via fast feedback loops. Process minded QA folk can help "test" this process as an Agile project manager or Scrum master, helping ensure the teams are performing as efficiently as possible. Engineering process design as a program manager is also a possible option for the process minded.

Help Others

Specific companies will have differing needs depending on their own particular business and place on their journey into DevOps. A smart QA engineer should always have something to contribute - even if QA or test is not in the job title.

It's the End of QA as We Know It ...

Soundtrack

As software development has moved to Agile and now to DevOps and continuous deployment there are those questioning the need to have a QA or testing department. From the infamous GTAC 2011 Keynote Test is Dead to Eli Lopian's post The Day the QA Department Died to Prezi's How did dissolving the QA team enable the DevOps culture at Prezi there has been some discussion about how QA fits in with Agile and DevOps. Mirco Hering in a recent post on Not A Factory Anymore called The Testing Centre of Excellence is dead, Long Live the Centre of Quality Engineering argues that due to the rise of the importance of speed to market and the cost of the temporal delay between finding defects and the actually creation of the code means that a separate team that comes in tests the code doesn't really work in this scenario.

Cheap, Fast and Good Enough

Another factor, especially true for cloud/SaaS offerings with a mature continuous deployment pipeline, is the plummeting cost of failure. Back in the days when software was shipped on physical media a defect making it into GA could be extremely costly. New versions of a CD ROM might need to be burned. Box art and manuals printed. Translations updated. All these materials shipped and assembled into product and shipped to customers. Existing stock shipped back and destroyed. 

All this cost real money so it was in the interest of the business to be cautious about defects escaping into production. Even moving to digital delivery still had associated costs. The updated release needed to be gathered, staged and pushed to the delivery channel like an e-commerce site or content delivery network. The additional costs in this case are mostly labor (rather than physical goods) but time your release engineering team spends on pushing errata is time people aren't spending creating new value.

Today's SaaS offerings can roll back in minutes. Why pay to try to detect all possible defects when you can push an experiment to production and canary or A/B test it fast and roll it back faster if it fails?

We Wanted This

In this new environment the key is to move to a model of "continuous quality". For years QA folk have been advocating moving quality practices "to the left" in the software development process. We wished developers would write unit tests, we advocated for pervasive test automation and static analysis frameworks. Code reviews. We dreamed about continuous integration systems. We crossed the aisle and worked closely with our developers to test and debug hairy problems. We cavorted with our brothers and sisters in support over customer reported issues. We wanted everyone in software delivery to care about quality. Now more people do.

Years ago a QA colleague of mine would often joke "I'm just trying to automate my way out of a job." With the rise of continuous deployment pipelines it is now a possibility. File under "Be careful what you wish for ..." So what will be come of us in this new reality? Personally I'm excited and hopeful because I believe that there are numerous places where a "tester's mindset" can be of value to business in this brave new world. I'll share my thoughts on that in my next post.

It's the end of QA as we know it ... and I feel fine.

Out of the Bubble ...

After 12 years at Red Hat I'm finally out ... and taking a bit of a sabbatical. So I thought rather than just lay on the couch and eat chocolate bon bons the first month I'd start a blog about software testing.  Which brings us here and now. We'll see how this develops over time but it should be a fun way for me to share some thoughts, insights and opinions on software quality and software in general. Stay tuned ...