“Everyone” is an engineer. Developers and testers do not work on contradictory paradigms.
They are part of the “One” team.
These two statements form the basis of a successful DevOps team.
The key goal of an effective DevOps process is to keep the software artifact releasable, deployable and shippable at all points in time even while new functionalities and features are being worked on.
Unfortunately, there is a lot of misunderstanding around the word “DevOps”. Organizations seem to be focusing too much on automating their deployments as much as possible (Not necessarily a bad thing!) and on demand virtual infrastructure.
Two major (flawed) implementations have been observed:
- Form a tiger team and name it “DevOps” who do nothing but work on state of the art build pipelines using tools galore – Jenkins, Bamboo and the likes.
- A “Strategy” geared towards CD (Continuous Delivery) with great emphasis on automating the deployments.
The truth is that there is a lot more to DevOps than what we have just outlined. We just cannot treat DevOps as an isolated activity. It is a mindset, it is a culture.
DevOps creates a culture where the “Team” (Dev, QA and Operations) would work seamlessly to create, test, release, update and monitor applications quicker and more efficiently.
There needs to be an obsession with quality in the “Team” with a clear focus and understanding on how a particular feature would deliver exceptional value, without which there is a high chance of building something wrong resulting in wastage of efforts and all automation done would be of no help!
Changing role of QA in DevOps
Traditionally, there was a clear demarcation between Dev and QA. QA would get a build which would be deployed on their QA environment and QA would then commence their smoke, sanity, functional & regression testing. The build would sit with the QA for a couple of days and then finally a QA sign-off on the build.
All this changes with DevOps:
Reactive to Proactive
The primary role of the QA and test team traditionally has been to “Detect” faults and bugs. They were the last line of defense and were the gatekeepers to defect leakage. This was reactive. The QA teams now need a mindset change, they need to be “Proactive. What does that mean? Means closer collaboration with the Dev and product teams, constant checks and balances in the dev process and to an extent even acting as drivers and mentors to the overall quality of the code. The mantra here is “Continuous improvement”.
Continuous Quality does not just mean that we check for features and verify that they work as designed but to question whether the design is the best or to a level even if we picked the right feature!
This is driven by:
- A very high level of collaboration which is aimed to build a clear understanding of what the deliverables of the “Team” are. Practices that assist here can be impact analysis, story analysis, BDD (Behavior Driven Development)
- A right mix of technical practices which CI (Continuous integration), Automation of the UAT process, TDD (Test Driven Development).
DevOps teams function under a very high level of coordination which erases the boundaries between different roles defined as part of the “One” team. The idea is that everyone contributes across the software delivery value chain. Everyone is responsible for quality. Deployment engineers can add test cases if needed, Dev can configure deployment if needed and QA can configure automation test cases.
- Development Engineers – Continuous Testing with tests to include functional, performance, UI, API and more along with regression tests to ensure a comprehensive test coverage. Increase breadth of test coverage to cover more modules, units, and functionality.
- Operations Engineers - Continuous Integration environment so as to use the right tests.
- QA Engineers – Increase depth of continuous testing and enhancing on developer test cases to include in depth functional, performance, UI, API and more along with regression tests to ensure comprehensive test coverage.
- Automate the acceptance test cases for CD (Continuous delivery).
- Production monitoring for app performance and alert mechanisms on failures.
Continuous testing is a pre-cursor to an effective Continuous delivery process.
Automation is key
This really is the cornerstone we need to achieve - "Speed, "Quality" and "Agility". We need to:
- Automate all the testing processes and configure them to run automatically when the deployment is completed on the QA environment.
- Integrate into the CI (Continuous Integration) process.
- Build an automation testing framework that supports easy and error prone addition of test scripts.
- Factor in parallel and distributed test strategies for quicker testing.
- Test selection and execution based on module, code, functionality being changed is desirable. Running a regression on every check in or build can prove to be expensive.
- Factor in test run schedule that require no manual intervention, giving 24/7 test running capabilities.
- Bring up test infrastructure on the fly to save initial investments costs. Leverage the cloud whenever possible for on demand scalable test infrastructure.
- Integrate your test framework to your defect management system.
- Focus on code coverage, your test automation framework should be aimed at constantly increasing this value.
Skills must for a QA Engineer in DevOps
Coding and Ingenuity
QA Engineers need to enhance and build on their coding skills. Coding knowledge helps QA engineers identify possible code issues and enables them to test more thoroughly in the affected areas. Coding is a necessity to transition to writing and running automated test cases. These scripts need to be then constantly maintained and optimized by a QA engineer.
QA professionals need to be up-skilled to learn the new techniques in this practice. QA engineers must realize that adopting automation and coding knowledge will help them become better at white-box and black-box testing. DevOps QA foundation is based on excellent automation.
Communication and Collaboration
DevOps drives this and a clear communication and collaboration practice needs to be in place. This will help sharing of a common understanding and purpose DevOps team and the value being delivered. This also helps establish quality practices which are proactive rather than reactive. Some practices like impact analysis, story analysis, BDD (Behavior Driven Development).
Awareness of new tools and technologies
DevOps is seeing a continuous increase in tools and technologies that have been sprouting in this landscape. It is absolutely essential to keep pace with them for better application of QA and testing processes.
- Container Concepts
- Source Control
- Continuous Integration
- Deployment Automation & Orchestration
- Infrastructure Automation
Is manual testing dead? Definitely not, Manual testing in the form of exploratory testing, usability testing and so on, will still continuously be done throughout the lifecycle of an application and would still play a pivotal role. A QA engineer will be responsible for exploring and providing feedback about the product quality, specification and design, as early as it is feasible to do so. That being said, QA engineers would have to go through a paradigm shift in thought process, mind set and culture and adapt to the changing landscape of DevOps providing value throughout the lifecycle of the application. However automation and technology re-skilling would be of paramount importance for a QA engineer to succeed in the world of DevOps.