A friendly introduction to Machine Learning with ml5

Ricardo Ramirez
By Ricardo Ramirez
2020-06-11

Learning fundamental building blocks for you to experiment with Machine Learning in your browser.

Have you ever wondered how your phone’s keyboard knows what words you’ll use next when sending a text? Short answer: using Machine Learning. Long answer: the keyboard on your phone is probably using different mathematical and statistical concepts to learn/predict what you’ll do before you do it. Sounds interesting right? Let’s explore a little bit deeper into how.

People who create these types of applications (like your keyboard app) don’t know you personally, much less know your particular way of writing, however, their responsibility is to make your keyboard smart enough to learn so then you can write better and faster every day. That’s Machine Learning in a nutshell: teaching a computer to perform a task without explicitly telling it how to do it.

Sure, that’s a fairly broad definition of Machine Learning, but it’s a starting point. You can also watch this fun video from CGP Gray about How Machines Learn and have another good explanation. Many other online resources can help you with more details and even expand on more advanced topics. This introduction is different, here I’ll try to share some fundamentals of Machine Learning building blocks and show you how to use them right away in your projects.

How to get started

Let’s say you are ready to create your own app using Machine Learning. We want an app that guesses what object you put in front of your phone’s camera. On one hand, you may be interested in taking a very complete Machine Learning course such as Google’s ML Crash Course or you can learn with me and start with an awesome tool called ml5.js

What is ml5.js?

ml5.js is a JavaScript library (siblings with p5.js, this is important) whose goal is to help you create and experiment with Machine Learning in the browser.

This is really cool for artists, developers, students, and anyone interested to be able to play, experiment and understand through real-life examples.

How easy, you may ask? Well, ml5.js gives you a very friendly access to algorithms and functions that underneath use another very powerful library to allow you to experiment with Machine Learning right in your browser.

ml5.js uses TensorFlow.js as a base, which is a very powerful and popular library to handle mathematical operations and machine learning algorithms in JavaScript. You can learn more about TensorFlow.js on their official site.

As mentioned on its official site, ml5.js provides immediate access in the browser to pre-trained models for detecting human poses, generating text, styling an image with another, composing music, pitch detection, and common English language word relationships, and much more.

What are pre-trained models? We’ll see that in a moment.

Before writing code, let’s learn some important concepts that will help you along the way.

Key concepts

Model

You can understand a model as a box that receives data input, does something, and generates an output. It’s important to know that you can create your own box (train a model) or work with an existing one (pre-trained model). Again, there is a lot of complexity going into the concept so don’t worry if it seems confusing to you. This video from Microsoft about Machine Learning Models can give you more details but, for our own good, our initial definition should give you a good idea to start.

Training is known as the process of finding patterns and relationships from a training dataset.

In general terms, models can be classified as supervised learning, unsupervised learning and reinforced learning. For the sake of this introduction, we’ll not go into the definition of each one, we’ll prefer to show you how a supervised model works and use the pre-trained model provided by ml5.js for image classification.

Algorithm

Methods or procedures that aim to perform a task or solve a problem. Speaking specifically of Machine Learning, these processes can be repetitive and involve corrections and adjustments to itself.

Dataset

You can think of it as a table of values, usually ordered and classified with labels. They are used both to train a model (training dataset) or to test a model (test dataset).

Prediction

The stage where using a pre-trained model, we provide an input dataset and wait for values as output, usually containing labels and scores.

Now with those building blocks in mind, we can start experimenting with some code.

Code example

For this example, we’ll build a small app that tries to guess what object is in front of your camera. If you are impatient you can see the result here.

We are using ml5.js functions for image classification and p5.js (optional) to manage the camera in the browser. If you would like to learn more about p5.js you can visit their official site.

Some JavaScript knowledge is ideal but not required for you to explore. I encourage you to change the code and see what happens.

Key Features

Let’s briefly look at the relevant lines of code that make this app work. Feel free to play with the code first and then come back for understanding the key parts. You can check the complete code here:

1. First, we need our model, in this case we use ml5.js imageClassifier function to create an object that classifies images (in this case a video) using a pre-trained model called MobileNet (already included with ml5.js).

2. Did you notice modelLoaded when creating our model? That is the name of the callback function that is executed once our model is available.

3. Now that we have our model we need it to make predictions by using the function with the same name.

classifier.predict(modelResults);

4. Now modelResults is the name of another callback function that receives the prediction results and stores them into a dictionary called results from where we’ll extract our predictions ready to be displayed.

 

5. There it is, our little app using ml5.js!

As you can see, with a few lines of code we can start experimenting with Machine Learning. You don’t like a video? No problem. If you go to the ml5.js documentation you will find many other examples using images, sounds, and text. There is no excuse not to start playing with Machine Learning!

What other ideas can you think of?

This article is heavily inspired by Dan Shiffman’s “A Beginner’s Guide to Machine Learning with ml5.js“, so If you would like to continue learning I highly recommend watching his videos.

I hope you have enjoyed learning with me and feel inspired to continue creating using Machine Learning.

 

About the author

Ricardo Ramírez

Graduated from Computer Science Engineering from Universidad Don Bosco. Ricardo has over 8 years of experience in Software Development. He’s currently a Front End Developer at Applaudo Studios.