# Streamlined and Secure: Building a Modern CI/CD Pipeline on AWS with Open Source Tools(Jenkins, ArgoCD, Prometheus,Grafana, Trivy , SonarQube) This project demonstrates how to deploy a Netflix clone application using a DevSecOps approach on Amazon Web Services (AWS) using Jenkins for CI, Prometheus& Grafana for monitoring, SonarQube & Trivy for security Checks, and later deploying it in K8s Ckuster using ArgoCD as Gitops tool. This project has been published by [Nasiullha Chaudhari](https://www.linkedin.com/in/nasiullha-chaudhari-5a80601a8/overlay/about-this-profile/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base%3BOxgh5Y3xS2CSoXMxKFPAfw%3D%3D) which can be found [here](https://github.com/N4si/DevSecOps-Project) . This blog post builds upon the valuable foundation provided by the project's GitHub README, offering a deeper exploration of the implementation process. Before we dive deep into the project lets breakdown the project into the phases in which the actions are to be performed: **Phase 1: Initial Setup and Deployment** Setup an EC2 Instance and run the application in docker container **Phase 2: Security** Implement Security checks using scanning tools **Phase 3: CI/CD Setup** 1. **Install Necessary Plugins:** Jenkins plugins are installed for tools like SonarQube Scanner, NodeJS, and dependency management. Credentials for DockerHub are also added securely. 1. **Install Jenkins:** This is used to automate the deployment process. 2. **Configure CI/CD Pipeline:** A Jenkins pipeline is created to automate the following stages: * Clean workspace * Checkout code from Git repository * SonarQube Analysis: Code is scanned for vulnerabilities and quality issues using SonarQube. * Quality Gate: The pipeline can be halted if quality thresholds are not met. * Install Dependencies: Any additional dependencies required by the application are installed. * OWASP FS SCAN: The code is scanned for potential vulnerabilities in dependencies using the OWASP Dependency-Check plugin. * TRIVY FS SCAN: The container image is scanned for vulnerabilities using Trivy. * Docker Build & Push: The application is built as a Docker image, tagged with your credentials, and pushed to a Docker registry. * TRIVY: The pushed image is scanned again for vulnerabilities using Trivy. * Deploy to container: The application container is deployed and runs on the server. **Phase 4: Monitoring** 1. **Install Prometheus and Grafana:** These tools are used to monitor the application's performance and health. Prometheus collects metrics, while Grafana visualizes them for easy analysis. 2. **Configure Prometheus:** Prometheus is configured to scrape metrics from the application and other relevant sources like Node Exporter (which collects system metrics from the server). 3. **Install Node Exporter:** This tool is installed to collect system metrics from the server running the application. 4. **Configure Prometheus Plugin Integration:** Jenkins can be integrated with Prometheus to monitor the CI/CD pipeline itself. 5. **Install Grafana:** This is installed and configured to display visualizations of the collected metrics from Prometheus. **Phase 5: Notification** 1. **Implement Notification Services:** This could involve setting up email notifications or other mechanisms to receive alerts when issues arise. **Phase 6: Kubernetes (Optional)** This phase covers deploying the application to a scalable environment using Kubernetes, a container orchestration platform. It also involves integrating monitoring with Prometheus and Node Exporter within Kubernetes. [ ](https://www.linkedin.com/in/nasiullha-chaudhari-5a80601a8/overlay/about-this-profile/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base%3BOxgh5Y3xS2CSoXMxKFPAfw%3D%3D) ## [1\. Hardware Req](https://www.linkedin.com/in/nasiullha-chaudhari-5a80601a8/overlay/about-this-profile/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base%3BOxgh5Y3xS2CSoXMxKFPAfw%3D%3D)ui[rements: Server Con](https://www.linkedin.com/in/nasiullha-chaudhari-5a80601a8/overlay/about-this-profile/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base%3BOxgh5Y3xS2CSoXMxKFPAfw%3D%3D)figuration and port on which each service is configured to run: ```jsx Server 1(Application/Pipeline Instance) : T2.large 1. Jenkins(8080) 2. sonarqube(9000) 3. Netflix Container(8081) ---------------------------------------------------------------------= On server 2(Monitoring Instance): T2.medium 3. Node Exporteer(9100) 4. Prometheus(9090) -> yaml configuration 5. Grafana(3000) data source ``` --- ## 2\. Application Architecture: ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711896682786/b98dfb30-3ec0-487e-9c31-6d4eaf53d38a.png align="center") ## 4\. Implementation. ### **Phase 1: Initial Setup and Deployment** Instance: **i-0a9b2965ff3babddd (devSecOps)** **Step 1: Launch EC2 (Ubuntu 22.04):** * Provision an EC2 instance on AWS with Ubuntu 22.04. * Connect to the instance using SSH. **Step 2: Clone the Code:** * Update all the packages and then clone the code. * Clone your application's code repository onto the EC2 instance: ```plaintext git clone https://github.com/N4si/DevSecOps-Project.git ``` **Step 3: Install Docker and Run the App Using a Container:** * Set up Docker on the EC2 instance: ```plaintext sudo apt-get update sudo apt-get install docker.io -y sudo usermod -aG docker $USER # Replace with your system's username, e.g., 'ubuntu' newgrp docker sudo chmod 777 /var/run/docker.sock ``` * Build and run your application using Docker containers: ```plaintext docker build -t netflix . docker run -d --name netflix -p 8081:80 netflix:latest #to delete docker stop docker rmi -f netflix ``` Once the container is run we can access the website but the page is empty because the contents are being delivered via an external API(TMDB) which were supposed to pass through the argument in our docker command. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711897289611/55a042f9-399c-44a7-9109-1aa7251087d0.png align="center") **TMDB**: The movie database api credentials(personal) for the netflix clone website **Step 4: Get the API Key:** * Open a web browser and navigate to TMDB (The Movie Database) website. * Click on "Login" and create an account. * Once logged in, go to your profile and select "Settings." * Click on "API" from the left-side panel. * Create a new API key by clicking "Create" and accepting the terms and conditions. * Provide the required basic details and click "Submit." * You will receive your TMDB API key. * ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711897407000/05044281-17ae-4af3-b2d2-3f198a7cd0db.png align="center") Now recreate the Docker image with your api key: ```plaintext docker build --build-arg TMDB_V3_API_KEY= -t netflix . ``` ```jsx sudo docker run -d -p 8081:80 netflix2 docker run -d --name sonar -p 9000:9000 sonarqube:lts-community ``` ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711897423995/93a7c612-4560-46b6-9726-e676f20e3b52.png align="center") **Phase 2: Security** 1. **Install SonarQube and Trivy:** * Install SonarQube and Trivy on the EC2 instance to scan for vulnerabilities. sonarqube ```plaintext docker run -d --name sonar -p 9000:9000 sonarqube:lts-community ``` To access: publicIP:9000 (by default username & password is admin) To install Trivy: ```plaintext sudo apt-get install wget apt-transport-https gnupg lsb-release wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add - echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list sudo apt-get update sudo apt-get install trivy ``` to scan image using trivy ```plaintext trivy image ``` 2. **Integrate SonarQube and Configure:** * Integrate SonarQube with your CI/CD pipeline. * Configure SonarQube to analyze code for quality and security issues. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711897463942/ed41ce6c-bf5b-4e07-98e5-f5b4f4efd55f.png align="center") ### **Phase 3: CI/CD Setup** 1. **Install Jenkins for Automation:** * Install Jenkins on the EC2 instance to automate deployment: Install Java ```plaintext sudo apt update sudo apt install fontconfig openjdk-17-jre java -version openjdk version "17.0.8" 2023-07-18 OpenJDK Runtime Environment (build 17.0.8+7-Debian-1deb12u1) OpenJDK 64-Bit Server VM (build 17.0.8+7-Debian-1deb12u1, mixed mode, sharing) #jenkins sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \ https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \ https://pkg.jenkins.io/debian-stable binary/ | sudo tee \ /etc/apt/sources.list.d/jenkins.list > /dev/null sudo apt-get update sudo apt-get install jenkins sudo systemctl start jenkins sudo systemctl enable jenkins ``` * Access Jenkins in a web browser using the public IP of your EC2 instance. publicIp:8080 2. **Install Necessary Plugins in Jenkins:** Goto Manage Jenkins →Plugins → Available Plugins → Install below plugins 1 Eclipse Temurin Installer (Install without restart) 2 SonarQube Scanner (Install without restart) 3 NodeJs Plugin (Install Without restart) 4 Email Extension Plugin ### **Configure Java and Nodejs in Global Tool Configuration** Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save ### SonarQube Create the token Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this After adding sonar token Click on Apply and Save **The Configure System option** is used in Jenkins to configure different server **Global Tool Configuration** is used to configure different tools that we install using Plugins We will install a sonar scanner in the tools. Create a Jenkins webhook 1. **Configure CI/CD Pipeline in Jenkins:** * Create a CI/CD pipeline in Jenkins to automate your application deployment. Certainly, here are the instructions without step numbers: **Install Dependency-Check and Docker Tools in Jenkins** **Install Dependency-Check Plugin:** * Go to "Dashboard" in your Jenkins web interface. * Navigate to "Manage Jenkins" → "Manage Plugins." * Click on the "Available" tab and search for "OWASP Dependency-Check." * Check the checkbox for "OWASP Dependency-Check" and click on the "Install without restart" button. **Configure Dependency-Check Tool:** * After installing the Dependency-Check plugin, you need to configure the tool. * Go to "Dashboard" → "Manage Jenkins" → "Global Tool Configuration." * Find the section for "OWASP Dependency-Check." * Add the tool's name, e.g., "DP-Check." * Save your settings. **Install Docker Tools and Docker Plugins:** * Go to "Dashboard" in your Jenkins web interface. * Navigate to "Manage Jenkins" → "Manage Plugins." * Click on the "Available" tab and search for "Docker." * Check the following Docker-related plugins: * Docker * Docker Commons * Docker Pipeline * Docker API * docker-build-step * Click on the "Install without restart" button to install these plugins. **Add DockerHub Credentials:** * To securely handle DockerHub credentials in your Jenkins pipeline, follow these steps: * Go to "Dashboard" → "Manage Jenkins" → "Manage Credentials." * Click on "System" and then "Global credentials (unrestricted)." * Click on "Add Credentials" on the left side. * Choose "Secret text" as the kind of credentials. * Enter your DockerHub credentials (Username and Password) and give the credentials an ID (e.g., "docker"). * Click "OK" to save your DockerHub credentials. Now, you have installed the Dependency-Check plugin, configured the tool, and added Docker-related plugins along with your DockerHub credentials in Jenkins. You can now proceed with configuring your Jenkins pipeline to include these tools and credentials in your CI/CD process. ```jsx pipeline{ agent any tools{ jdk 'jdk17' nodejs 'node16' } environment { SCANNER_HOME=tool 'sonar-scanner' } stages { stage('clean workspace'){ steps{ cleanWs() } } stage('Checkout from Git'){ steps{ git branch: 'main', url: '' } } stage("Sonarqube Analysis "){ steps{ withSonarQubeEnv('sonar-server') { sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \\ -Dsonar.projectKey=Netflix ''' } } } stage("quality gate"){ steps { script { waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token' } } } stage('Install Dependencies') { steps { sh "npm install" } } stage('OWASP FS SCAN') { steps { dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check' dependencyCheckPublisher pattern: '**/dependency-check-report.xml' } } stage('TRIVY FS SCAN') { steps { sh "trivy fs . > trivyfs.txt" } } stage("Docker Build & Push"){ steps{ script{ withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){ sh "docker build --build-arg TMDB_V3_API_KEY=8ddb5be8173d1f5790522f4d62bf3937 -t netflix ." sh "docker tag netflix lajahshrestha/netflix-clone:latest " sh "docker push lajahshrestha/netflix-clone:latest " } } } } stage("TRIVY"){ steps{ sh "trivy image lajahshrestha/netflix-clone:latest > trivyimage.txt" } } stage('Deploy to container'){ steps{ sh 'docker run -d -p 8081:80 lajahshrestha/netflix-clone:latest' } } } } ``` ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711897543265/998b2b8c-f0b3-4674-bf43-923519a3fa00.png align="center") ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1711897550582/18c01a69-cc05-4384-afbb-1eaa7a0b2afb.png align="center") ### **Phase 4: Monitoring** Prometheus and Grafana are a powerful duo for monitoring applications and infrastructure. Prometheus, the unsung hero, acts as a data collector, constantly gathering metrics on various aspects like CPU usage, memory consumption, or API request counts. It keeps a watchful eye on these metrics, and if they stray from expected levels, it throws up an alert. But raw data can be overwhelming. This is where Grafana steps in. It acts as the visualizer, transforming the data collected by Prometheus into easy-to-understand graphs, charts, and dashboards. You can customize these dashboards to focus on key performance indicators (KPIs) that matter most to you. Together, they provide a clear picture of your application's health and performance, allowing you to identify and address potential issues before they snowball. 1. **Install Prometheus and Grafana:** Set up Prometheus and Grafana to monitor your application. **Installing Prometheus:** First, create a dedicated Linux user for Prometheus and download Prometheus: ```plaintext sudo useradd --system --no-create-home --shell /bin/false prometheus wget ``` Extract Prometheus files, move them, and create directories: ```plaintext tar -xvf prometheus-2.47.1.linux-amd64.tar.gz cd prometheus-2.47.1.linux-amd64/ sudo mkdir -p /data /etc/prometheus sudo mv prometheus promtool /usr/local/bin/ sudo mv consoles/ console_libraries/ /etc/prometheus/ sudo mv prometheus.yml /etc/prometheus/prometheus.yml ``` Set ownership for directories: ```plaintext sudo chown -R prometheus:prometheus /etc/prometheus/ /data/ ``` Create a systemd unit configuration file for Prometheus: ```plaintext sudo nano /etc/systemd/system/prometheus.service ``` Add the following content to the `prometheus.service` file: ```plaintext [Unit] Description=Prometheus Wants=network-online.target After=network-online.target StartLimitIntervalSec=500 StartLimitBurst=5 [Service] User=prometheus Group=prometheus Type=simple Restart=on-failure RestartSec=5s ExecStart=/usr/local/bin/prometheus \\ --config.file=/etc/prometheus/prometheus.yml \\ --storage.tsdb.path=/data \\ --web.console.templates=/etc/prometheus/consoles \\ --web.console.libraries=/etc/prometheus/console_libraries \\ --web.listen-address=0.0.0.0:9090 \\ --web.enable-lifecycle [Install] WantedBy=multi-user.target ``` Here's a brief explanation of the key parts in this `prometheus.service` file: * `User` and `Group` specify the Linux user and group under which Prometheus will run. * `ExecStart` is where you specify the Prometheus binary path, the location of the configuration file (`prometheus.yml`), the storage directory, and other settings. * `web.listen-address` configures Prometheus to listen on all network interfaces on port 9090. * `web.enable-lifecycle` allows for management of Prometheus through API calls. Enable and start Prometheus: ```plaintext sudo systemctl enable prometheus sudo systemctl start prometheus ``` Verify Prometheus's status: ```plaintext sudo systemctl status prometheus ``` ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/406d53e2-f0b7-4644-8900-387a607db1fe/Untitled.png align="left") You can access Prometheus in a web browser using your server's IP and port 9090: `http://:9090` **Installing Node Exporter:** Create a system user for Node Exporter and download Node Exporter: ```plaintext sudo useradd --system --no-create-home --shell /bin/false node_exporter wget ``` Extract Node Exporter files, move the binary, and clean up: ```plaintext tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz sudo mv node_exporter-1.6.1.linux-amd64/node_exporter /usr/local/bin/ rm -rf node_exporter* ``` Create a systemd unit configuration file for Node Exporter: ```plaintext sudo nano /etc/systemd/system/node_exporter.service ``` Add the following content to the `node_exporter.service` file: ```plaintext [Unit] Description=Node Exporter Wants=network-online.target After=network-online.target StartLimitIntervalSec=500 StartLimitBurst=5 [Service] User=node_exporter Group=node_exporter Type=simple Restart=on-failure RestartSec=5s ExecStart=/usr/local/bin/node_exporter --collector.logind [Install] WantedBy=multi-user.target ``` Replace `--collector.logind` with any additional flags as needed. Enable and start Node Exporter: ```plaintext sudo systemctl enable node_exporter sudo systemctl start node_exporter ``` Verify the Node Exporter's status: ```plaintext sudo systemctl status node_exporter ``` You can access Node Exporter metrics in Prometheus. 2. **Configure Prometheus Plugin Integration:** Integrate Jenkins with Prometheus to monitor the CI/CD pipeline. **Prometheus Configuration:** To configure Prometheus to scrape metrics from Node Exporter and Jenkins, you need to modify the `prometheus.yml` file. Here is an example `prometheus.yml` configuration for your setup: ```plaintext scrape_configs: - job_name: "prometheus" static_configs: - targets: ["localhost:9090"] - job_name: 'node_exporter' static_configs: - targets: ['localhost:9100'] - job_name: 'jenkins' metrics_path: '/prometheus' static_configs: - targets: ['13.232.109.143:8080'] ``` Make sure to replace `` and `` with the appropriate values for your Jenkins setup. Check the validity of the configuration file: ```plaintext promtool check config /etc/prometheus/prometheus.yml ``` Reload the Prometheus configuration without restarting: ```plaintext curl -X POST ``` You can access Prometheus targets at: `http://:9090/targets` ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/8d87a139-55a8-48ab-8df3-5bc765b5c420/Untitled.png align="left") ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/c47cf561-7cea-4de0-8afb-94030d9d1332/Untitled.png align="left") ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/b4d073bb-0360-439b-b459-05b4bea78696/Untitled.png align="left") **Install Grafana on Ubuntu 22.04 and Set it up to Work with Prometheus** **Step 1: Install Dependencies:** First, ensure that all necessary dependencies are installed: ```plaintext sudo apt-get update sudo apt-get install -y apt-transport-https software-properties-common ``` **Step 2: Add the GPG Key:** Add the GPG key for Grafana: ```plaintext wget -q -O - | sudo apt-key add - ``` **Step 3: Add Grafana Repository:** Add the repository for Grafana stable releases: ```plaintext echo "deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list ``` **Step 4: Update and Install Grafana:** Update the package list and install Grafana: ```plaintext sudo apt-get update sudo apt-get -y install grafana ``` **Step 5: Enable and Start Grafana Service:** To automatically start Grafana after a reboot, enable the service: ```plaintext sudo systemctl enable grafana-server ``` Then, start Grafana: ```plaintext sudo systemctl start grafana-server ``` **Step 6: Check Grafana Status:** Verify the status of the Grafana service to ensure it's running correctly: ```plaintext sudo systemctl status grafana-server ``` **Step 7: Access Grafana Web Interface:** Open a web browser and navigate to Grafana using your server's IP address. The default port for Grafana is 3000. For example: `http://:3000` You'll be prompted to log in to Grafana. The default username is "admin," and the default password is also "admin." ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/07b28349-be1d-4f22-86f0-819400dac00e/Untitled.png align="left") **Step 8: Change the Default Password:** When you log in for the first time, Grafana will prompt you to change the default password for security reasons. Follow the prompts to set a new password. **Step 9: Add Prometheus Data Source:** To visualize metrics, you need to add a data source. Follow these steps: * Click on the gear icon (⚙️) in the left sidebar to open the "Configuration" menu. * Select "Data Sources." * Click on the "Add data source" button. * Choose "Prometheus" as the data source type. * In the "HTTP" section: * Set the "URL" to `http://localhost:9090` (assuming Prometheus is running on the same server). * Click the "Save & Test" button to ensure the data source is working. ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/0ee2321c-eed8-4264-a54c-65ab4e53e755/Untitled.png align="left") **Step 10: Import a Dashboard:** To make it easier to view metrics, you can import a pre-configured dashboard. Follow these steps: * Click on the "+" (plus) icon in the left sidebar to open the "Create" menu. * Select "Dashboard." * Click on the "Import" dashboard option. * Enter the dashboard code you want to import (e.g., code 1860). * Click the "Load" button. * Select the data source you added (Prometheus) from the dropdown. * Click on the "Import" button. You should now have a Grafana dashboard set up to visualize metrics from Prometheus. ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/bcdba78d-f288-42da-9115-4c909463c61d/Untitled.png align="left") Grafana is a powerful tool for creating visualizations and dashboards, and you can further customize it to suit your specific monitoring needs. That's it! You've successfully installed and set up Grafana to work with Prometheus for monitoring and visualization. 1. **Configure Prometheus Plugin Integration:** * Integrate Jenkins with Prometheus to monitor the CI/CD pipeline. ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/d0304725-49f8-4382-bca5-72d3d34a0577/Untitled.png align="left") ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/946ce4eb-f5e1-4422-b914-6d2ab0bcf71f/Untitled.png align="left") **Phase 5: Notification** 1. **Implement Notification Services:** * Set up email notifications in Jenkins or other notification mechanisms. ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/2b6e24b2-d461-4715-b119-00ceea61b072/Untitled.png align="left") ```jsx lajah.aws@gmail.com ktut rfsu nkog avvt ``` ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/67df652a-97c6-4ab4-a32e-72238eac9f8d/Untitled.png align="left") ![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/74341641-7150-4a30-b048-ed32fbd55682/0e081cb1-4c82-4003-a634-190a162d161c/Untitled.png align="left") **Phase 6: Kubernetes** **Create Kubernetes Cluster with Nodegroups** In this phase, you'll set up a Kubernetes cluster with node groups. This will provide a scalable environment to deploy and manage your applications. **Monitor Kubernetes with Prometheus** Prometheus is a powerful monitoring and alerting toolkit, and you'll use it to monitor your Kubernetes cluster. Additionally, you'll install the node exporter using Helm to collect metrics from your cluster nodes. **Install Node Exporter using Helm** To begin monitoring your Kubernetes cluster, you'll install the Prometheus Node Exporter. This component allows you to collect system-level metrics from your cluster nodes. Here are the steps to install the Node Exporter using Helm: 1. Add the Prometheus Community Helm repository: ```plaintext helm repo add prometheus-community ``` 2. Create a Kubernetes namespace for the Node Exporter: ```plaintext kubectl create namespace prometheus-node-exporter ``` 3. Install the Node Exporter using Helm: ```plaintext helm install prometheus-node-exporter prometheus-community/prometheus-node-exporter --namespace prometheus-node-exporter ``` Add a Job to Scrape Metrics on nodeip:9001/metrics in prometheus.yml: Update your Prometheus configuration (prometheus.yml) to add a new job for scraping metrics from nodeip:9001/metrics. You can do this by adding the following configuration to your prometheus.yml file: ```plaintext - job_name: 'Netflix' metrics_path: '/metrics' static_configs: - targets: ['node1Ip:9100'] ``` Replace 'your-job-name' with a descriptive name for your job. The static\_configs section specifies the targets to scrape metrics from, and in this case, it's set to nodeip:9001. Don't forget to reload or restart Prometheus to apply these changes to your configuration. To deploy an application with ArgoCD, you can follow these steps, which I'll outline in Markdown format: **Deploy Application with ArgoCD** 1. **Install ArgoCD:** You can install ArgoCD on your Kubernetes cluster by following the instructions provided in the [EKS Workshop](https://archive.eksworkshop.com/intermediate/290_argocd/install/) documentation. 2. **Set Your GitHub Repository as a Source:** After installing ArgoCD, you need to set up your GitHub repository as a source for your application deployment. This typically involves configuring the connection to your repository and defining the source for your ArgoCD application. The specific steps will depend on your setup and requirements. 3. **Create an ArgoCD Application:** * `name`: Set the name for your application. * `destination`: Define the destination where your application should be deployed. * `project`: Specify the project the application belongs to. * `source`: Set the source of your application, including the GitHub repository URL, revision, and the path to the application within the repository. * `syncPolicy`: Configure the sync policy, including automatic syncing, pruning, and self-healing. 4. **Access your Application** * To Access the app make sure port 30007 is open in your security group and then open a new tab paste your NodeIP:30007, your app should be running. ## Conclusion By implementing a DevSecOps approach with tools like Prometheus and Grafana, you gain a comprehensive view of your application's security, performance, and health. This allows for proactive management, ensuring a secure and well-functioning application throughout its lifecycle. Remember, DevSecOps is an ongoing process, so continuously refine your approach as your application evolves and make sure to cleanup the resources used while doing the process to cut off ongoing charges from the vendor. Happy learning.