In the last post MLOPS - Lab Setup, I was able to set the local ML lab environment, and run validation in Jupyter Notebook to test the CODA device and performance on my local PC. Although Jupyter Notebooks can be user-friendly tools for ML practice, offering easy interaction and immediate feedback, which simplifies testing and debugging, it has limitations such as reproducibility issues, challenges in collaboration and version control, scalability concerns for larger projects, and a lack of automation for tasks like retraining. ...

Transitioning from DevOps to MLOps can be achieved by leveraging existing DevOps expertise by adding new layers specific to machine learning. Key Differences: Model Lifecycle Management: MLOps handles model training, deployment, and retraining. Data Versioning: Tools like DVC ensure dataset version control. Experiment Tracking: MLflow and Weights & Biases track model training parameters and results. Model Serving: Deploy models with TensorFlow Serving or TorchServe. Model Drift: Monitor data changes over time to trigger retraining. ...

AWS Managed Prometheus & Grafana is the “plug-and-play” choice for production workloads requiring minimal management, while on the other hand installing helm kube-prometheus-stack offers maximum control but requires more effort to maintain and scale effectively. Hence for cost control and full customization, I decided to install kube-prometheus-stack on my local lab cluster. Understand Prometheus Pull-based Monitoring Flow Expose Metrics: Applications expose metrics in Prometheus format. Discover Targets: Kubernetes-native targets: Discovered via the Kubernetes API. Non-cloud-native targets: Defined statically or exposed through exporters. Scrape Metrics: Prometheus scrapes metrics periodically from /metrics endpoints. Store Metrics: Metrics are stored in Prometheus’s time-series database. Visualize Metrics: Grafana (in kube-prometheus-stack) is often used to query and visualize metrics. Prometheus metrics issue ...

‘Kubernetes Release vs EKS EOL’ A Kubernetes version encompasses both the control plane and the data plane. While AWS manages and upgrades the control plane, we (cluster owner/customer) hold the responsibility for initiating upgrades for both cluster control plane as well as the data plane. When we initiate a cluster upgrade, AWS manages upgrading the control plane, and we are still responsible for initiating the upgrades of the data plane, which includes worker nodes provisioned via Self Managed node groups, Managed Node Groups, Fargate & other add-ons. If worker nodes are provisioned via Karpenter Controller, we can take advantage of Drift or Disruption Controller features (spec.expireAfter) for automatic node recycling and upgrade. ...

‘Karpenter vs Cluster Autoscaler’ Karpenter is more modern, flexible, and cost-efficient, making it a better choice for dynamic, complex, or large-scale workloads on EKS. Cluster Autoscaler is simpler and integrates seamlessly with AWS Managed Node Groups, making it suitable for basic scaling needs. Transitioning to Karpenter from Cluster Autoscaler is a logical step when EKS cluster demands evolve toward more complex scaling with diverse workloads, cost optimization, fine-grained control over node provisioning. ...

‘when EKS ConfigMap meet AWS Secret manager Here I will demo a mysql as database and a wordpress deployment as backend in EKS to reference ConfigMap and Secret via environment variables during run time by reading the environment-specific configuration (like environment names or feature flags) from a ConfigMap and sensitive information (like a database password) from a Secret. Create demo resources # vim cf-demo.yaml to create demo namespace, configmap, secret and deployment root@asb:~/cf# cat secret.yaml apiVersion: v1 kind: Secret metadata: name: mysql-root-secret type: Opaque data: MYSQL_ROOT_PASSWORD: c2VjdXJlcGFzc3dvcmQ= # Base64 for "securepassword" root@asb:~/cf# cat mysql.yaml apiVersion: v1 kind: Service metadata: name: mysql-service spec: ports: - port: 3306 selector: app: mysql --- apiVersion: apps/v1 kind: Deployment metadata: name: mysql spec: selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:5.7 env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-root-secret key: MYSQL_ROOT_PASSWORD root@asb:~/cf# cat wordpress.yaml apiVersion: v1 kind: ConfigMap metadata: name: wordpress-config data: WORDPRESS_DB_HOST: "mysql-service:3306" WORDPRESS_DB_NAME: "wordpress" --- apiVersion: v1 kind: Secret metadata: name: wordpress-secret type: Opaque data: WORDPRESS_DB_USER: d29yZHByZXNz # Base64 for "wordpress" WORDPRESS_DB_PASSWORD: c2VjdXJlcGFzc3dvcmQ= # Base64 for "securepassword" --- apiVersion: apps/v1 kind: Deployment metadata: name: wordpress spec: replicas: 1 selector: matchLabels: app: wordpress template: metadata: labels: app: wordpress spec: containers: - name: wordpress image: wordpress:latest ports: - containerPort: 80 env: - name: WORDPRESS_DB_HOST valueFrom: configMapKeyRef: name: wordpress-config key: WORDPRESS_DB_HOST - name: WORDPRESS_DB_NAME valueFrom: configMapKeyRef: name: wordpress-config key: WORDPRESS_DB_NAME - name: WORDPRESS_DB_USER valueFrom: secretKeyRef: name: wordpress-secret key: WORDPRESS_DB_USER - name: WORDPRESS_DB_PASSWORD valueFrom: secretKeyRef: name: wordpress-secret key: WORDPRESS_DB_PASSWORD --- apiVersion: v1 kind: Service metadata: name: wordpress-service spec: selector: app: wordpress ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer # Or NodePort if LoadBalancer is not available Apply and verify the Configuration and Secrets in the Pod root@asb:~/cf# kubectl create ns cf-test root@asb:~/cf# kubectl apply -f . -n cf-test configmap/wordpress-config created secret/wordpress-secret created deployment.apps/wordpress created service/wordpress-service created secret/mysql-root-secret created service/mysql-service created deployment.apps/mysql created root@asb:~/cf# kubectl get all -n cf-test NAME READY STATUS RESTARTS AGE pod/mysql-fdff667f8-xzblb 1/1 Running 0 6s pod/wordpress-6dff4575b9-sgn8t 1/1 Running 0 12m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/mysql-service ClusterIP 10.106.191.27 <none> 3306/TCP 6s service/wordpress-service LoadBalancer 10.97.169.213 pending 80:31208/TCP 12m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mysql 1/1 1 1 6s deployment.apps/wordpress 1/1 1 1 12m NAME DESIRED CURRENT READY AGE replicaset.apps/mysql-fdff667f8 1 1 1 6s replicaset.apps/wordpress-6dff4575b9 1 1 1 12m # Kubectl exec into pod to verify the configmap and secret root@asb:~/cf# kubectl logs wordpress-6dff4575b9-sgn8t -n cf-test WordPress not found in /var/www/html - copying now... Complete! WordPress has been successfully copied to /var/www/html No 'wp-config.php' found in /var/www/html, but 'WORDPRESS_...' variables supplied; copying 'wp-config-docker.php' (WORDPRESS_DB_HOST WORDPRESS_DB_NAME WORDPRESS_DB_PASSWORD WORDPRESS_DB_USER WORDPRESS_SERVICE_PORT WORDPRESS_SERVICE_PORT_80_TCP WORDPRESS_SERVICE_PORT_80_TCP_ADDR WORDPRESS_SERVICE_PORT_80_TCP_PORT WORDPRESS_SERVICE_PORT_80_TCP_PROTO WORDPRESS_SERVICE_SERVICE_HOST WORDPRESS_SERVICE_SERVICE_PORT) AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.48.245. Set the 'ServerName' directive globally to suppress this message AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.48.245. Set the 'ServerName' directive globally to suppress this message [Wed Nov 06 23:39:29.456836 2024] [mpm_prefork:notice] [pid 1:tid 1] AH00163: Apache/2.4.62 (Debian) PHP/8.2.25 configured -- resuming normal operations [Wed Nov 06 23:39:29.456923 2024] [core:notice] [pid 1:tid 1] AH00094: Command line: 'apache2 -D FOREGROUND' root@asb:~/cf# kubectl exec -it wordpress-6dff4575b9-sgn8t -n cf-test -- env | grep WORDPRESS_DB WORDPRESS_DB_NAME=wordpress WORDPRESS_DB_USER=wordpress WORDPRESS_DB_PASSWORD=securepassword WORDPRESS_DB_HOST=mysql-service:3306 IRSA with AWS Secret Manager to ensure security best practices in EKS ...

Valero is an open-source tool for storing, restoring, and migrating Kubernetes cluster resources and persistent volumes. Valero provides a way to hold the entire state of a Kubernetes cluster, all its objects and their consistent numbers, store backup files to Cloud storage like AWS S3, and then restore them to a previous state to ensure K8S data resilience, disaster recovery results and easy transport between clusters. Velero for EKS Backup EKS cluster using Velero, can be followed by the below path: ...

“After scalling, let’s go EKS Security !” In the last post, I was able to implement EKS cluster autoscaler and Horizontal Pod Autoscaler (HPA), in this post I will continue with EKS security practice with Kube-Bench and OPA Gatekeeper. kube-bench: kube-bench is a tool that checks Kubernetes clusters against the CIS (Center for Internet Security) benchmarks, a set of best practices for securing Kubernetes. It is critical to ensure that a cluster complies with these security guidelines, helping identify potential vulnerabilities and misconfigurations. Key features include generating detailed audit reports, performing automated compliance checks, and easily integrating into existing CI/CD pipelines for continuous security assessments. ...

“Explore how pods and EC2 nodes can be scaled in EKS cluster " Autoscaler and Horizontal Pod Autoscaler (HPA) Practice on EKS In last post, I was able to deploy EKS cluster via Jenkins and Terraform: Jenkins - Multi-Destnation Continuse Deployment with Terraform This post walks through the process of setting up an Autoscaler and Horizontal Pod Autoscaler (HPA) in an Amazon EKS cluster. We will explore how to dynamically scale the number of nodes in EKS cluster and how to autoscale K8S pods based on CPU utilization using HPA. ...

“Design a Jenkins pipeline can handle multiple destination: EC2, EKS across different environments” Jenkins CD Pipeline Design In last post, I was able to create a Jenkins Universal CI Pipeline to create blog docker image and push to DockerHub: Jenkins - Universal CI Pipeline with Ansible & Terraform Now it is time to design the continuous deployment pipeline with Ansible and Terraform for infrastructure provision and application configuration and deployment. ...