Microsoft Azure and DevOps

I recently took the online course for DevOps: An IT Pro Guide. It took me about four hours to go through all of the modules and assessments.

https://mva.microsoft.com/en-US/training-courses/devops-an-it-pro-guide-8286

The course information is accurate. This course is the foundation for how to implement and use DevOps with Azure and other tools. The hosts first explain the business concepts of DevOps and then how to implement. The hosts provide a lot of practical examples with step-by-step details (along with a good sense of humor).

I recommend that you take the course if you are interested in using DevOps with Azure. I will briefly review some of the material covered in the modules. I will use screenshots of the material presented.

The explanation as to why DevOps should be implemented was simpler than what I saw in similar training videos provided by IBM – see my previous blog. Of course, I agree with the benefits listed for DevOps:

DevOpsBenefits

Avoid creating VMs and manually copying code over to test. Instead, automate with continuous integration and build. You can choose a Microsoft-only ecosystem, an open source ecosystem, or a hybrid. Obviously, there was not enough time in the sessions to show all of the possible ecosystems. Still, I see that there is a lot of flexibility in what tools can be used for supporting DevOps.

If you want to use DevOps with Microsoft-only technologies, your ecosystem will look like below:

MicrosoftEcosystem

See below if you want to use an open source ecosystem. You can still use Azure!

OpenSourceEcosystem

You can use Release Management software to create environments using scripts. The ability to configure and manage the environment is what makes it possible to automate it. Yes, one of the hosts is in the screenshot.

ReleaseManagement

I saw one example where Jenkins is notified when new source code is committed to source code control. The notification is then passed along to Puppet Master and to the Puppet agent on the Azure VM. The new build is downloaded and installed on the Azure VM. The entire process is automated!

PuppetMaster

You can also de-provision resources when you are done.

Deprovisioning

You can use Systems Center Operations Manager to install and operate agents on VMs. This allows monitoring of the VMs during automated testing.

HybridMonitoring

It is a lot of material to cover in the sessions. You do have to pay attention if you want to take and pass the assessment questions – which I recommend. However, the education does not end here. There are technical resources provided that can take you further:

TechNet Blogs: http://blogs.technet.com/b/devops/

Learn More about DevOps: http://www.microsoft.com/en-us/server-cloud/solutions/development-operations.aspx

Again, I recommend the training sessions!

Why I like DevOps

My first real job was in manufacturing. I ran several different types of Computerized Numerically Controlled (CNC) machinery. Some machines cut particle board, others drilled holes, and another was a router capable of both cutting and drilling. The problem we had was trying to coordinate all of the activities in the manufacturing plant. If we cut and drilled too much wood, there would be too much material waiting in the pre-assembly area. But cut and drill too little and the assembly workers went home early. Later I worked in receiving and we had the same problem in a different way. We had to send the correct quantity of the right parts to different areas in the manufacturing plant at the right time. Any mistake meant that we had to fix it somehow – which usually meant wasted time. We also believed in quality management. We wanted to manage production efficiently with little waste and produce products that our customers wanted. We all had to work together to achieve this. This is often called Lean Manufacturing. The last job that I had in manufacturing was implementing an ISO9000 quality management system. The implementation was successful because we had everyone work together to achieve it. The benefit of implementing ISO9000 was that we all had a common understanding of who our customer was and what they expected.

I was part of several efforts to implement continuous quality improvement methods at the same time. The goal was to improve product quality as the product moved through manufacturing. We had a huge success when we improved how preventative maintenance was performed on two critical drilling machines. The problem was that no one had explained to the drill operators what to inspect on the equipment. We created a daily checklist for them together with the equipment maintenance team. After two weeks, we saw a dramatic drop in machine down time. The drill machines used to break down almost 4 out of every 16 hours each day. Now the drill machines were never breaking down. We soon applied the same principles to all of the other critical machinery in manufacturing.

The concept of DevOps is similar; but applied to software development.

  • Try to prevent problems from happening or, at least, test and fix them immediately.
  • Everyone should work together to solve problems as soon as possible. Don’t wait until the very end to find out how big a problem is.
  • Reduce the amount of time where work is waiting in a queue.

I will try to explain the parts of DevOps that I feel are important based on my experience. DevOps is the integration of software development, quality assurance, and IT Operations. The goal is to rapidly produce software with fewer gaps between the three main components. That is, there should be no gap between developing software and the required testing. Much of the deployment and testing can be automated so that the feedback loop on software development is as short as possible. Thus, developers will not have to wait several days for QA Testers to test and provide results. The feedback should come back soon after the changes are deployed. In addition, the testing environment should be created each time the software is deployed and should represent the production environment. Other software dependencies should be tested, too. In fact, if other software is being changed at the same time, it should also be deployed and tested at the same time. No software development should assume that there will be a final week of integration testing. Integration testing should take place all of the time.

I took the screenshot below from one of the sessions that Sanjeev Sharma provided. He provided an excellent explanation of where the bottle necks are before implementing DevOps.

DeliveryPipeline

To summarize the bottlenecks and solutions on the above image:

Bottlenecks Solutions
 Rigid ‘One-size-fits-all’ Development process  Agile Transformation with ‘Risk-Value’ based Process Variants
 Ticket Based Environment Provisioning Cloud Hosted Developer ‘Self-Service’
 Weekend long Deployments that often fail Frequent Deployment of Smaller Development
 Late Discovery of Architectural Fragility  ‘Shift Left’ Integration Testing to early in LifeCycle

A successful implementation of DevOps will improve deployment frequency. It will also lead to faster time to market, lower failure rate of new releases, and shortened lead time between fixes. My experience with DevOps for the products that I managed was very good. I felt like I could release a hotfix or new build every day. The development team had test environments, new builds, and tests running every night. The test results were reviewed every day. Fixes to new code were implemented and retested. The team was constantly adding new tests to ensure that the software was stable and functioning as required.

The development team used deployment automation to create new environments (via configurations) whenever needed. Amazon Web Services was used for the automated deployment and running the environments.

CloudEnvironment

It does take some effort to automate the creation of the environments, processes, and configurations. A considerable amount of time and frustration is saved once it is done and implemented.

I used the following resources to learn more about DevOps:

DevOps for Dummies http://www.ibm.com/ibm/devops/us/en/resources/dummiesbooks/

Understanding DevOps https://www.youtube.com/playlist?list=PL-u6s3rs05hUuEe4bLahtULFGQZXruAoq

DevOps Wikipedia https://en.wikipedia.org/wiki/DevOps