Machine Learning on Zack's Blog

Can AI Actually Build a Complex Project? A Brutal RAG System Post-Mortem

Sun, 01 Mar 2026 05:27:11 +0000

Last month, I did something that, looking back, was probably the most educational practice of my year. I tried using Claude and GPT to build a complete multi-source RAG system from scratch. This wasn’t a “toy demo” for social media; it was a real internal tool for my lab designed to handle messy document data, query rewriting, hybrid search, and deployment for my team.

I set a strict rule for myself: Let the AI write almost all the code, while I handle the requirements and decision-making.

Improve Billing & Health Analyzer with Bedrock v2

Wed, 21 Jan 2026 04:50:08 +0000

I constantly rely on AWS Billing and Cost Management and Trusted Advisor to monitor costs and security. However, manual billing reviews are time-consuming, and raw Cost Explorer data doesn’t provide the full picture. What if an intelligent program could proactively call the Billing and Trusted Advisor APIs, retrieve the raw data, and pass it to an LLM for in-depth analysis of account health—then deliver actionable insights and AI-powered recommendations? That would save a significant amount of time and effort.

Serverless Billing & Health Analyzer with Bedrock

Sat, 18 Oct 2025 11:42:05 +0000

Boost EKS RAG with LangChain

Sat, 11 Oct 2025 05:21:03 +0000

After building a custom RAG system on Amazon EKS, my son asked why the application could only handle one question at a time, while ChatGPT allows users to continue conversations through follow-up questions with contextual memory. That made me realize there was an opportunity to leverage the power of the LangChain framework for better abstractions, conversational memory, and access to the broader LangChain ecosystem.

This post documents the journey of migrating to a LangChain-powered solution — the challenges faced, and the new capabilities and benefits gained.

AWS Bedrock with Open WebUI

Tue, 16 Sep 2025 14:56:30 +0000

Recently Data team reached out, trying to build an EC2 running Open WebUI, connecting to AWS Bedrock, to offer team member AI application. This guide provides step-by-step instructions for deploying and integrating with AWS Bedrock models. .

The architecture consists of two Docker containers operating on a single EC2 instance: Open WebUI serves as the user-facing chat interface, while a Bedrock Access Gateway acts as middleware. This gateway securely forwards requests from Open WebUI to the AWS Bedrock API using the EC2 instance’s attached IAM role. The containers communicate over a private Docker network, isolating traffic between them.

AI Coding - Build an Electricity and Gas Analytic Platform

Thu, 14 Aug 2025 07:45:20 +0000

In the past few days, I got an idea from my recent electricity and gas bills—thinking about how to run data analysis to examine usage patterns and trends using the Python approach: employing Pandas for reading data, using Numpy for normalisation, and Matplotlib to create a few visualisations.

However, this time I wanted to try talking to Gemini CLI via the terminal to see if it could generate a web-based analytical platform for me, as someone with no developer background.

Exploratory Data Analysis (EDA) with Peppa Pig

Sun, 27 Jul 2025 05:56:03 +0000

This was a linguistic analysis project where the primary goal was not just to count words, but to evaluate the language, themes, and emotional tone of the children’s show “Peppa Pig” (specifically, the first four seasons) to determine its suitability for a pre-kindergarten audience. In this study, I will try to answer a broader question:

“Beyond a simple word list, what can a multi-faceted data analysis tell us about the show’s true educational and emotional value?”

MLOPS - Enhancing Oscar Model with LightGBM

Fri, 21 Mar 2025 13:36:27 +0000

In a previous post - MLOps - Build a Oscar Best Picture Winner Model, I was able to establish a baseline model using a RandomForestClassifier to predict the Oscar for Best Picture. This classic workflow involved data cleaning, training, and prediction, providing a solid starting point.

However, a deeper look at the results revealed critical weaknesses that an experienced machine learning engineer would immediately flag:

Inadequate Model Choice: The initial model wasn’t powerful enough for the task. The classification report showed a recall of 0.00 for the “winner” class. This is a major red flag, indicating the model completely failed to identify any actual winners, likely due to the severe class imbalance.
Misleading Evaluation Metrics: I think I relied too heavily on accuracy. On an imbalanced dataset, a model can achieve high accuracy simply by always predicting the majority class. Better to shift our focus to more robust metrics like the F1-score, ROC AUC, and Precision-Recall AUC.

This analysis led to idea to enhance this Oscar prediction with a more sophisticated LightGBM (LGBM) classifier model, known for its high performance, speed, and efficiency on tabular data.

MLOps - Build a Oscar Best Picture Winner Model

Sun, 23 Feb 2025 09:37:43 +0000

In this post, we will continue to build a basic machine learning model to predict the Best Picture winner at the Academy Awards (Oscar).

We will use our previous processed dataset that includes information about the nominees and winners from the 72nd to the 96th Oscar ceremonies. The goal is to predict the winner based on various features like IMDb ratings, Metascore, Tomatometer percentage, Golden Globe and BAFTA wins/nominations.

Step 1: Understand the features and model

MLOps - Data Processing for Oscar Winner Model

Sun, 23 Feb 2025 05:35:54 +0000

After downloading a dataset from Kaggle about historical Oscar nominations and winners, it is time to move on to data cleansing and enrichment.

The original data contained seven headers, but only a few were useful: the film name, ceremony year, and winner status. Additionally, the dataset spanned from 1927 to 2024 and included all Oscar categories, making it quite complex—perhaps too overwhelming for me as a beginner. So, I decided to focus on Best Picture as it is always the most important award among all the other categories.

MLOps - How About Predict Oscar Winner

Fri, 21 Feb 2025 11:50:12 +0000

The Idea

The 97th Academy Awards ceremony, presented by the Academy of Motion Picture Arts and Sciences (AMPAS), will take place on March 2, 2025, at the Dolby Theatre in Hollywood, Los Angeles. Last year 2024 I had some greate experience with some great movies like Dune Part2 and Wicked, not sure if some of my faviourate actors can grab a Oscar.

so Why not go and use historical Oscar nominations dataset for the past 20 years, to create and train a model by feeding categories and results, to predict the winners each year, then input this years’ nominations to get a prediction ??

MLOps - Get Started with KubeRay

Tue, 11 Feb 2025 04:35:22 +0000

Ray is an open-source framework designed for scalable and distributed ML workloads, including training, tuning, and inference. It provides a simple API for scaling Python applications across multiple nodes.

Distributed Training: Easily scale PyTorch, TensorFlow, and other ML jobs across multiple GPUs/instances.
Hyperparameter Tuning: Integrates with Optuna and Ray Tune for distributed hyperparameter optimization.
Parallel Inference: Supports inference pipelines that scale out dynamically based on demand.
Fault Tolerance: If a node fails, Ray can reschedule tasks on other available nodes.

Together with EKS, Karpenter, and Ray, a modern ML team can achieve dynamic Auto-Scaling and GPU resource allocation from local deployment to Running Distributed ML Jobs in Cloud Production environment:

MLOps - Deploy ML workload into EKS with Karpenter

Sat, 01 Feb 2025 04:35:11 +0000

Finally, machine learning workload into AWS EKS with Karpenter

In the previous post, I was able to complete both local Pytorch ML and AWS SageMaker practice, and containerize and deploy ML docker image locally.

In this post, I will explore the model deployment to EKS cluster with Karpenter to simulate a more scalable and real-life production-ready environment.

Challenges when moving to Cloud deployment

Here are key differences between local PyTorch model vs AWS SageMaker artifacts, and how they impact the Docker image size and Performance considerations for LLM images in EKS deployment.

MLOps - Deploy Classifier with AWS Serverless

Sat, 25 Jan 2025 04:34:58 +0000

Continue Pneumonia Classifier by moving to AWS Serverless

Go AWS Serverless Deployment

Deploying machine learning models in production requires additional considerations to address latency, scalability, cost-efficiency, and monitoring.

A modern approach to hosting an ML application in AWS can be considered as a serverless architecture.

This allows users to upload images via a S3 static web page and send them to API Gateway. The API Gateway receives the HTTP POST request and forwards it to the Lambda function, which handles the image preprocessing, inference, and postprocessing logic. The Lambda function sends the image payload to the SageMaker endpoint, by calling the SageMaker endpoint using the SageMaker Runtime SDK (invoke_endpoint) which hosts my trained model, and then retrieves the prediction. The prediction result is sent back to the frontend for display.

MLOps - Move Image Classifier to AWS SageMaker

Sat, 25 Jan 2025 04:34:41 +0000

Continue Pneumonia Classifier by moving to AWS SageMaker

In this post, we will continue Pneumonia Classifier practice by leveraging AWS managed AI service SageMaker. We will:

Provision AWS Sagemaker AI domain and workspace.
Run a JupiterLab to download, process, and upload chest X-ray dataset from Kaggle to S3 for SageMaker access.
Use SageMaker Pre-built image classification algorithm, to define SageMaker Estimator to configure the SageMaker training job, including compute resources, training duration, and data input method.
Count the number of training samples and set the hyperparameters, perform hyperparameter tuning to find the best configuration for the model.
Launching the Hyperparameter Tuning Job.
Using CloudWatch and SageMaker Train Job to monitor and troubleshoot.

In the AWS SageMaker AI JupiterLab

MLOps - Containerize Classifier Application

Sat, 25 Jan 2025 04:34:14 +0000

Last post we were able to build and train the pneumonia classifier model.
In this post we are moving a step forward to: Containerizing the model and creating a frontend-backend application to allow users to upload images and get predictions.

Overview of the Application Architecture

A high-level diagram or description of the system architecture:

Backend: A Dockerized Flask/FastAPI application serving the trained model.
Frontend: A Dockerized React/Streamlit app for image upload and displaying predictions.
Interaction: Frontend sends images to the backend, which processes them and returns predictions.

Step 1: Containerizing the Backend

MLOps - Build a Image Classifier Model

Sat, 25 Jan 2025 04:34:02 +0000

Pneumonia causes over 2.5 million deaths annually worldwide. Traditional diagnosis through chest X-ray analysis is time-consuming and requires expert radiologists. This project aims to develop an automated deep learning system to classify pneumonia from chest X-rays with high accuracy, potentially assisting healthcare professionals in faster diagnosis.

Purpose

Here I will use a local environment with Pytorch and Jupyter Notebook to:

Demonstrate end-to-end development of a medical image classifier
Using Kaggle dataset for NIH Chest X-Ray image processing
Create a baseline model for pneumonia detection (Normal vs Pneumonia)
Local GPU acceleration for model training and prediction

Typical ML steps to achieve image classifier:

MLOps - Deploy ML workload to K8S

Mon, 20 Jan 2025 04:33:48 +0000

Machine Learning workload deployed in K8S and minikube!!

So far the local ML practice is just the beginning, in a real world production environment, ML projects typically follow a structured lifecycle and often deployed in scalable, cloud-based environments. Cloud Providers like AWS with Managed Kubernetes Services (EKS) provide orchestration, scaling, and fault tolerance to handle ML workloads.

Path for deploying MLOps workload in K8S

Transition from local ML practice to a K8S-based deployment (This post)
Start with MLOps tools like Kubeflow.
Shift from local to cloud platforms (AWS) to deploy ML workload on EKS.
Practice deploying models using REST APIs (local) and APT Gateway (AWS).
Try local data engineering (ETL pipelines, data lakes, etc.), then move to AWS data services and solutions for ML workload.

Setting up Minikube with GPU on WSL Ubuntu

MLOps - Build a Knowledge Base with DeepSeek R1

Sun, 19 Jan 2025 04:33:34 +0000

‘Deep Seek is really hot at the moment!

In this post, I want to build a local Knowledge Base using WSL, Docker, Ollama, Open WebUI and DeepSeek R1 7b.

Ollama is a desktop application designed to run and interact with large language models (LLMs) locally on a machine. It provides an easy interface for downloading, managing, and using various LLMs, ensuring privacy and local execution.

Open WebUI is a web-based user interface for interacting with AI models. It is often used in conjunction with locally hosted or remote LLMs, providing a customizable and user-friendly platform to input queries and manage model interactions.

MLOps - Explore ML tools

Wed, 02 Oct 2024 04:32:58 +0000

In the last post MLOPS - Lab Setup, I was able to set the local ML lab environment, and run validation in Jupyter Notebook to test the CODA device and performance on my local PC.

Although Jupyter Notebooks can be user-friendly tools for ML practice, offering easy interaction and immediate feedback, which simplifies testing and debugging, it has limitations such as reproducibility issues, challenges in collaboration and version control, scalability concerns for larger projects, and a lack of automation for tasks like retraining.

MLOps - Setup a Home Machine Learning Lab

Tue, 01 Oct 2024 04:32:35 +0000

Transitioning from DevOps to MLOps can be achieved by leveraging existing DevOps expertise by adding new layers specific to machine learning.

Key Differences:
Model Lifecycle Management: MLOps handles model training, deployment, and retraining.
Data Versioning: Tools like DVC ensure dataset version control.
Experiment Tracking: MLflow and Weights & Biases track model training parameters and results.
Model Serving: Deploy models with TensorFlow Serving or TorchServe.
Model Drift: Monitor data changes over time to trigger retraining.