In a previous post - MLOps - Build a Oscar Best Picture Winner Model, I was able to establish a baseline model using a RandomForestClassifier to predict the Oscar for Best Picture. This classic workflow involved data cleaning, training, and prediction, providing a solid starting point.
However, a deeper look at the results revealed critical weaknesses that an experienced machine learning engineer would immediately flag:
Inadequate Model Choice: The initial model wasn’t powerful enough for the task. The classification report showed a recall of 0.00 for the “winner” class. This is a major red flag, indicating the model completely failed to identify any actual winners, likely due to the severe class imbalance. Misleading Evaluation Metrics: I think I relied too heavily on accuracy. On an imbalanced dataset, a model can achieve high accuracy simply by always predicting the majority class. Better to shift our focus to more robust metrics like the F1-score, ROC AUC, and Precision-Recall AUC. This analysis led to idea to enhance this Oscar prediction with a more sophisticated LightGBM (LGBM) classifier model, known for its high performance, speed, and efficiency on tabular data.
...