My first real job was in manufacturing. I ran several different types of Computerized Numerically Controlled (CNC) machinery. Some machines cut particle board, others drilled holes, and another was a router capable of both cutting and drilling. The problem we had was trying to coordinate all of the activities in the manufacturing plant. If we cut and drilled too much wood, there would be too much material waiting in the pre-assembly area. But cut and drill too little and the assembly workers went home early. Later I worked in receiving and we had the same problem in a different way. We had to send the correct quantity of the right parts to different areas in the manufacturing plant at the right time. Any mistake meant that we had to fix it somehow – which usually meant wasted time. We also believed in quality management. We wanted to manage production efficiently with little waste and produce products that our customers wanted. We all had to work together to achieve this. This is often called Lean Manufacturing. The last job that I had in manufacturing was implementing an ISO9000 quality management system. The implementation was successful because we had everyone work together to achieve it. The benefit of implementing ISO9000 was that we all had a common understanding of who our customer was and what they expected.
I was part of several efforts to implement continuous quality improvement methods at the same time. The goal was to improve product quality as the product moved through manufacturing. We had a huge success when we improved how preventative maintenance was performed on two critical drilling machines. The problem was that no one had explained to the drill operators what to inspect on the equipment. We created a daily checklist for them together with the equipment maintenance team. After two weeks, we saw a dramatic drop in machine down time. The drill machines used to break down almost 4 out of every 16 hours each day. Now the drill machines were never breaking down. We soon applied the same principles to all of the other critical machinery in manufacturing.
The concept of DevOps is similar; but applied to software development.
- Try to prevent problems from happening or, at least, test and fix them immediately.
- Everyone should work together to solve problems as soon as possible. Don’t wait until the very end to find out how big a problem is.
- Reduce the amount of time where work is waiting in a queue.
I will try to explain the parts of DevOps that I feel are important based on my experience. DevOps is the integration of software development, quality assurance, and IT Operations. The goal is to rapidly produce software with fewer gaps between the three main components. That is, there should be no gap between developing software and the required testing. Much of the deployment and testing can be automated so that the feedback loop on software development is as short as possible. Thus, developers will not have to wait several days for QA Testers to test and provide results. The feedback should come back soon after the changes are deployed. In addition, the testing environment should be created each time the software is deployed and should represent the production environment. Other software dependencies should be tested, too. In fact, if other software is being changed at the same time, it should also be deployed and tested at the same time. No software development should assume that there will be a final week of integration testing. Integration testing should take place all of the time.
I took the screenshot below from one of the sessions that Sanjeev Sharma provided. He provided an excellent explanation of where the bottle necks are before implementing DevOps.
To summarize the bottlenecks and solutions on the above image:
|Rigid ‘One-size-fits-all’ Development process||Agile Transformation with ‘Risk-Value’ based Process Variants|
|Ticket Based Environment Provisioning||Cloud Hosted Developer ‘Self-Service’|
|Weekend long Deployments that often fail||Frequent Deployment of Smaller Development|
|Late Discovery of Architectural Fragility||‘Shift Left’ Integration Testing to early in LifeCycle|
A successful implementation of DevOps will improve deployment frequency. It will also lead to faster time to market, lower failure rate of new releases, and shortened lead time between fixes. My experience with DevOps for the products that I managed was very good. I felt like I could release a hotfix or new build every day. The development team had test environments, new builds, and tests running every night. The test results were reviewed every day. Fixes to new code were implemented and retested. The team was constantly adding new tests to ensure that the software was stable and functioning as required.
The development team used deployment automation to create new environments (via configurations) whenever needed. Amazon Web Services was used for the automated deployment and running the environments.
It does take some effort to automate the creation of the environments, processes, and configurations. A considerable amount of time and frustration is saved once it is done and implemented.
I used the following resources to learn more about DevOps: