In this article, we discuss about the working and different stages of CI/CD Pipeline process and CI/CD need for every organization. Before getting into the topic, let’s brush up about CI\CD pipeline. Continuous integration (CI) is a process of developing software and performing initial tests. Continuous deployment (CD) is the process of integrating code with infrastructure. Finally, deploying the code into the desired environment.

CI/CD Stages: Understanding People, Process and Technology

Most application development teams consist of Developers, Testers/QA engineers, operation engineers, and SRE (Site Reliability Engineers) or IT Operations teams. They collaborate together to ensure that consumers receive high-quality applications. Continuous Integration and Continuous Deployment(CI/CD) are two different processes that make up CI/CD pipeline.

Continuous Integration(CI)

• People: Developers and Engineers, Database Administrator (DBA), Infrastructure Team.
• Technology: GitHub, Gitlab, SVM, BitBucket.
• Process: A commit is an action which sends a developer’s most recent changes to the repository. After getting a clear vision about the software requirements and specifications, developers will start writing the code and commit. Source Code Management (SCM tool) is the repository where the edits & commit changes are managed. Finally, the code changes are incorporated into the base code branch and stored in a central repository like GitHub.

Continuous integration(CI): Static Code Analysis

• People : Developers and Engineers, Database Administrator (DBA), Infrastructure team, Testers.
• Technology: GitHub, Gitlab, SVM, BitBucket.
• Process :  Next stage in CI is checking the code for static policies. SAST(Static Application Security Test) is a white-box testing method for finding software bugs by examining the code from the inside using SAST tools. Some SAST tools are SonarQube, Veracode, Appscan, and others. (For example, SQL injection). This is a quick test that looks for any syntactic errors in the code. However, this stage lacks the ability to search for runtime errors.

Continuous integration(CI): Build

• People: Developers and Engineers.
• Technology: Jenkins, Bamboo CI, Circle CI, Travis CI, Maven, Azure DevOps.
• Process: The CI process aids in the faster detection of bugs by ensuring that new modules are compatible with existing modules. This cuts down on the time it takes to test a new code update. Based on the programming language of source code, construct tools assist in compiling and generating executable files or packages (.exe,.dll,.jar…). In addition, SQL scripts, as well as infrastructure configuration files, were created and tested. Artifactory Storage, Build Verification, and Unit Tests are other sub-activities of the Build process.

Build Verification Test (BVT)/Smoke Tests and Unit Tests:

The next stage is smoke test. The components implementation process and the program’s essential functions are verified using BVT. The aim is to reject a badly damaged application so that QA team doesn’t waste time installing and checking it.

In addition to the above test, unit tests were also performed.

Artifactory Storage in CI:

Jfrog Artifactory or Repository tool is a centralized location or database used to store the packages and store binary files such as .rar, .war, .exe, Msi, etc.. Every day, a large number of builds are produced, making it difficult to keep track of them all. Finally, the build is sent to the repository for storage. From here, testers can choose and deploy an artefact in a test environment manually.

Continuous Integration(CI): Test Stages

• People: Testers, QA Engineers.
• Technologies: Selenium, Appium, Jmeter, SOAP UI, Tarantula.
• Process: A set of automated checks verify the code veracity following the build process. This stage prevents errors from making it to the production line. This check can take anything from seconds to hours, depending on the size of the build. QA Engineers (Testers) performs the automated tests. Testers perform regression analysis and also stress tests to check deviations from the expected output. Sanity test, integration tests and stress tests are also part of the testing process. Here, we came to know about the issues which is probably unknown to the developer.

Integration Testing:

Integration testing reveals flaws in the interaction of integrated units. The tool used to perform integration testing are Cucumber, Selenium, etc.. Performance testing is the next text. Before getting into the performance testing, someone should give approval to the set of updates made in the previous stage. This verification procedure can be time-consuming, but it is an essential part of the overall procedure.

Load and Stress Testing:

 This testing is used to find the stability, performance when the software is subjected to a large volume of traffic. These tests were used after multiple updates are clustered and full performance testing is performed. Finally, Canary testing is performed when a single update is being moved to the next stage.

Continuous Deployment: Bake and Deploy

• People: Infrastructure Engineer, Site Reliability Engineer (SRE), Operation Engineer.
• Technology: Spinnaker, Argo CD, Tekton CD.
• Process: At last, the passed codes gets stored into the servers. In servers, where the code will integrate with the main application. Before getting deployed into the production, Bake and Deploy are two substages through which the builds must move =. Spinnaker is home to both of these stages.

Continuous Deployment(CD): Bake

Baking is the process of constructing an immutable image instance from source code with the current development configuration. Database modifications and other technology upgrades are examples of these configurations. Example Spinnaker.

Continuous Deployment(CD): Deploy

Spinnaker will send the baked picture to the deploy stage automatically. Here, the server is configured for cluster deployment. During the deployment stage, a functionally equivalent procedure is carried out, similar to the testing processes mentioned above. Following the approvals and reviews, deployments are transferred to test, level, and finally production environments.

Continuous Deployment(CD): Verification

Teams may also use this space to optimize the overall CI/CD operation. Any failure at this stage, rectified as quickly as possible to minimize the effect on the end user. Teams should also think about automating this part of the process. The deployment techniques used to deploy to production are Blue-Green, Canary Analysis, Rolling Update, and others.

Continuous Deployment(CD): Monitoring

• People: SRE, Ops Team.
• Technology: Zabbix, Nagios, Prometheus, Elastic Search, Splunk, AppDynamics, Tivoli.
• Process: The performance metrics of updates, such as CPU usage and latency, will be tracked by application monitoring tools. Log analyzers can sift through reams of logs produced by the underlying middleware. And operating system to spot problems and track down the source of problems. Furthermore, the monitoring stage aids companies in gathering information about how new technological updates affect sales. And assisting the infrastructure team in tracking device activity trends and capability planning.

Continuous Deployment: Feedback and Collaboration Tool

• People: SRE, Ops, and Maintenance Team.
• Technology: JIRA, ServiceNow, Slack, Email, Hipchat.
• Process: The DevOps team’s mission is to release software more frequently and consistently with reduced errors and performance issues by providing regular feedback to developers, project managers. Also promptly raising tickets in ITSM tools. Feedback systems are usually a part of the entire software development process.

Summary of the article

In conclusion, we have discussed about different stages in CI/CD pipeline and also about different testing stages. With this overview, finally we came to know the various process involved in software development and also about some developing tools.