Michael Tefula

Today I decided to try out Google’s Vertex AI — a machine learning platform that makes it easier to train and deploy ML models.

In the last few weeks I’ve been writing code in Python to train models, but Vertex AI provides a low-code platform and a set of tools that simplify the deployment of ML models at scale.

Despite the promises of ease-of-use, I struggled to get Vertex AI to work on a small dataset (1k rows of data). Both training runs I tried went for 5-12 minutes before throwing up errors I couldn’t figure out.

Meanwhile, I was able to get a simple logistic regression model going in Python and trained locally in seconds. Here’s the performance of the model on fictional heart attack data.

Doing the same thing on Vertex AI should be quicker, and easier to deploy into a app. But it looks like I still have lots to figure out about the platform.

SVM, Precision & Recall.

I completed the final honors exercise today. Here’s a confusion matrix of my support vector machine model model. This chart is read diagonally left to right, and then diagonally right to left. In other words, we start with the diagonal top left to bottom right:

• Top Left (True Positives): the model correctly predicted rain in 98 cases.
• Bottom Right (True Negatives): the model correctly predicted that it would not rain in 445 cases.

Then we read the chart going from top right to bottom left:

• Top Right (False Negatives): the model incorrectly predicted ‘No Rain’ in 84 cases where it actually rained.
• Bottom Left (False Positives): the model incorrectly predict ‘Rain’ in 28 cases where it did not actually rain.

Key takeaways:

• Useful metrics that come from this chart include:
  • Precision: This is a metric that tells us how many predictions actually happened out of all the predictions for rain. e.g. in the example above:
    • Precision for ‘Rain’ predictions: The model predicted rain cases of 126 total (i.e. 98 + 28) but only 98 were actually true rain cases in the test data. This gives a precision rate of 77.8%.
  • Sensitivity (or Recall): This is a metric that tells us what proportion of predictions were correct vs. wrong. e.g. in the example above:
    • Sensitivity for ‘Rain’: The model correctly predicted rain cases of 98 rain cases out of a total of 182 (98 + 84) actual rain occurrences. This gives a sensitivity ratio of 53.8%.

Update on where I am with the honors labs:

1. Linear Regression – completed.
2. KNN – completed
3. Decision Trees – completed
4. Logistic Regression – completed
5. SVM – completed

I’m almost done with the honors assignments and today I went through a logistic regression model exercise. The code is on Github.

I used ChatGPT to help me write out the code for the visualizations. Here’s one I found particularly helpful. It shows the top 10 coefficients (aka features) that are most impactful on the model. Overall this model also had over 80% accuracy on predicting rain for the next day.

Update on where I am with the honors labs:

1. Linear Regression – completed.
2. KNN – completed
3. Decision Trees – completed
4. Logistic Regression – completed
5. SVM – to do

Today I trained a decision tree model on weather data from the honors lab I’m completing.

The model achieves a 83% accuracy on predicting if it will rain or not, using historical data. The chart below shows the decision tree steps.

More performance metrics can be viewed below.

I’ve put the code on Github.

A quick update on the honors assignments I’m completing:

1. Linear Regression – completed.
2. KNN – completed
3. Decision Trees – to do
4. Logistic Regression – to do
5. SVM – to do

Today, I caught up on some AI news.

There are lots of cool AI demos floating around on the net. Some look super impressive, but they fall apart when you dig into actual use cases.

In December last year, we learned that the Google Gemini model was edited favourably to look more impressive than it actually is. There’s no doubt that these models are incredible, but the demos are often designed to stir up more hype than is warranted.

Recently, there’s been talk of tools like Devin replacing software engineers. But experts in the field are more sceptical. Here’s a great breakdown video that tames the hype. (The Reddit discussion is good, too.)

And what of large language models in general? People are starting to see limits. Here’s a good perspective from one person who’s used LLMs on half a billion tokens in an enterprise use case.

Overall it’s great to see all the cool things generative AI can do. But we need to tame expectations! These tools have limits.

Today I completed the KNN lab. The work is ungraded and I haven’t yet checked whether my results are in line with other students’ but I’ll dig into this once I have all the modes. Here are the results for the KNN model I built today. The code is on Github.

1. Linear Regression – completed.
2. KNN – completed
3. Decision Trees – to do
4. Logistic Regression – to do
5. SVM – to do

Today I started the bonus material of the intro to ML course I’ve been doing. I’ll be working with weather data will practise all the key topics I covered in the course. Key areas include:

1. Linear Regression – completed.
2. KNN – to do
3. Decision Trees – to do
4. Logistic Regression – to do
5. SVM – to do

I’ll then assess all the above techniques using the following evaluation metrics.

1. Accuracy Score
2. Jaccard Index
3. F1-Score
4. LogLoss
5. Mean Absolute Error
6. Mean Squared Error
7. R2-Score

I’ve started with linear regression. The code for this is here. I’ll work through the remainder in the coming days.

I completed the Machine Learning with Python introductory course today and received a fun digital certificate for it here.

The course is designed as a six-week, part-time program, but I completed it in less than half the time by dedicating a little time every day. I’m also keen to move on to other, more practical building bits with LLMs. However, before that, I will revise what I’ve learnt in the last three weeks and apply the concepts to a few novel challenges.

83 more days of AI learning to go!

K-Means Clustering Continued…

I went through a lab exercise today on k-means. I’ve put the code on my Github page here.

I remain in awe that you can fit a model with a few lines of code. A snippet is provided below. In this example, a k-means algorithm is run on data assigned to the variable ‘X’. How cool is that?

Here are some cool charts from the code on Github.

Key takeaways:

• K-means is an unsupervised machine learning technique to cluster data into a ‘k’ number of groups.
• Example use cases include customer segmentation, fraud detection, and grouping biological markers.
• It’s useful where the data isn’t labelled, and you want to explore and find patterns that might not be immediately obvious.

K-Means Clustering.

Today I learnt about k-means clustering. This is another one of those fancy-sounding machine learning techniques that seems intimidating to the uninitiated. But actually, the intuition behind it is simple. Thanks to Youtube, you can understand the rudimental elements of this unsupervised machine learning algorithm fairly quickly. Here’s a video that got me over that hurdle.

Tomorrow, I’ll train a model using this technique.

Key takeway:

• Machine learning has so many techniques that it seems to me part of solving any problem with ML is first figuring out what data you need, and then researching and running experiments to figure out what the best algorithm to use is.
• Some of these algorithms are very old. Even when it comes to the latest breakthroughs with large language models — Geoffrey Hinton was using similar methods as far back as 1985 (albeit at a fraction of the scale today).