Here’s a summary of the actions I took to delete an AWS account using an open source tool aws-nuke as it is at the end of free tier I donot need it anymore to aviod cost: 1. Install aws-nuke: apt install aws-nuke This command installs aws-nuke, a tool used to delete AWS resources in an account. It requires AWS credentials and the correct configuration file to specify what resources to remove. ...
Ray is an open-source framework designed for scalable and distributed ML workloads, including training, tuning, and inference. It provides a simple API for scaling Python applications across multiple nodes. Distributed Training: Easily scale PyTorch, TensorFlow, and other ML jobs across multiple GPUs/instances. Hyperparameter Tuning: Integrates with Optuna and Ray Tune for distributed hyperparameter optimization. Parallel Inference: Supports inference pipelines that scale out dynamically based on demand. Fault Tolerance: If a node fails, Ray can reschedule tasks on other available nodes. Together with EKS, Karpenter, and Ray, a modern ML team can achieve dynamic Auto-Scaling and GPU resource allocation from local deployment to Running Distributed ML Jobs in Cloud Production environment: ...
Finally, machine learning workload into AWS EKS with Karpenter In the previous post, I was able to complete both local Pytorch ML and AWS SageMaker practice, and containerize and deploy ML docker image locally. In this post, I will explore the model deployment to EKS cluster with Karpenter to simulate a more scalable and real-life production-ready environment. Challenges when moving to Cloud deployment Here are key differences between local PyTorch model vs AWS SageMaker artifacts, and how they impact the Docker image size and Performance considerations for LLM images in EKS deployment. ...
Continue Pneumonia Classifier by moving to AWS Serverless Go AWS Serverless Deployment Deploying machine learning models in production requires additional considerations to address latency, scalability, cost-efficiency, and monitoring. A modern approach to hosting an ML application in AWS can be considered as a serverless architecture. This allows users to upload images via a S3 static web page and send them to API Gateway. The API Gateway receives the HTTP POST request and forwards it to the Lambda function, which handles the image preprocessing, inference, and postprocessing logic. The Lambda function sends the image payload to the SageMaker endpoint, by calling the SageMaker endpoint using the SageMaker Runtime SDK (invoke_endpoint) which hosts my trained model, and then retrieves the prediction. The prediction result is sent back to the frontend for display. ...
Continue Pneumonia Classifier by moving to AWS SageMaker In this post, we will continue Pneumonia Classifier practice by leveraging AWS managed AI service SageMaker. We will: Provision AWS Sagemaker AI domain and workspace. Run a JupiterLab to download, process, and upload chest X-ray dataset from Kaggle to S3 for SageMaker access. Use SageMaker Pre-built image classification algorithm, to define SageMaker Estimator to configure the SageMaker training job, including compute resources, training duration, and data input method. Count the number of training samples and set the hyperparameters, perform hyperparameter tuning to find the best configuration for the model. Launching the Hyperparameter Tuning Job. Using CloudWatch and SageMaker Train Job to monitor and troubleshoot. In the AWS SageMaker AI JupiterLab ...
Last post we were able to build and train the pneumonia classifier model. In this post we are moving a step forward to: Containerizing the model and creating a frontend-backend application to allow users to upload images and get predictions. Overview of the Application Architecture A high-level diagram or description of the system architecture: Backend: A Dockerized Flask/FastAPI application serving the trained model. Frontend: A Dockerized React/Streamlit app for image upload and displaying predictions. Interaction: Frontend sends images to the backend, which processes them and returns predictions. Step 1: Containerizing the Backend ...
Pneumonia causes over 2.5 million deaths annually worldwide. Traditional diagnosis through chest X-ray analysis is time-consuming and requires expert radiologists. This project aims to develop an automated deep learning system to classify pneumonia from chest X-rays with high accuracy, potentially assisting healthcare professionals in faster diagnosis. Purpose Here I will use a local environment with Pytorch and Jupyter Notebook to: Demonstrate end-to-end development of a medical image classifier Using Kaggle dataset for NIH Chest X-Ray image processing Create a baseline model for pneumonia detection (Normal vs Pneumonia) Local GPU acceleration for model training and prediction Typical ML steps to achieve image classifier: ...
Machine Learning workload deployed in K8S and minikube!! So far the local ML practice is just the beginning, in a real world production environment, ML projects typically follow a structured lifecycle and often deployed in scalable, cloud-based environments. Cloud Providers like AWS with Managed Kubernetes Services (EKS) provide orchestration, scaling, and fault tolerance to handle ML workloads. Path for deploying MLOps workload in K8S Transition from local ML practice to a K8S-based deployment (This post) Start with MLOps tools like Kubeflow. Shift from local to cloud platforms (AWS) to deploy ML workload on EKS. Practice deploying models using REST APIs (local) and APT Gateway (AWS). Try local data engineering (ETL pipelines, data lakes, etc.), then move to AWS data services and solutions for ML workload. Setting up Minikube with GPU on WSL Ubuntu ...
‘Deep Seek is really hot at the moment! In this post, I want to build a local Knowledge Base using WSL, Docker, Ollama, Open WebUI and DeepSeek R1 7b. Ollama is a desktop application designed to run and interact with large language models (LLMs) locally on a machine. It provides an easy interface for downloading, managing, and using various LLMs, ensuring privacy and local execution. Open WebUI is a web-based user interface for interacting with AI models. It is often used in conjunction with locally hosted or remote LLMs, providing a customizable and user-friendly platform to input queries and manage model interactions. ...
Recently I got a task from the Company’s app team, to install an agent on EC2 instances across several AWS accounts and enable connection via AWS private link to a target AWS account where the management server is hosted. To achieve this task, here I will see how to use AWS Systems Manager for software distribution and installation for multiple AWS accounts and install and configure AWS private link using Terraform. ...