Edge Impulse enables developers to create the next generation of intelligent device solutions with embedded Machine Learning. Machine Learning at the very edge will enable valuable use of the 99% of sensor data that is discarded today due to cost, bandwidth, or power constraints.

Enabling machine learning (ML) on microcontrollers allows low power consumption devices to detect motions and sounds, as well as collect and analyze data. Now you can use Edge Impulse to easily create and deploy models.

Now, Wio Terminal is officially supported by the Edge Impulse. Let’s see how to get Wio Terminal started with the Machine learning at the very edge! We will introduce:

• How to get started on Edge Impulse with Wio Terminal
• Step-by-step tutorial: Using Wio Terminal with Edge Impulse to Distinguish Alcohol
• Wio Terminal Continuous Motion Recognition

How to get started on Edge Impulse with Wio Terminal

Install dependencies

To set Wio Terminal up in Edge Impulse, you will need to install the following software:

1. Node.js v12 or higher.
2. Arduino CLI
3. The Edge Impulse CLI and a serial monitor. Install by opening command prompt or terminal and run:

npm install -g edge-impulse-cli @serialport/terminal

Note: Problems with installing the CLI? Please check Installation and troubleshooting for more reference.

Connect to Edge Impulse

With all the software in place, it’s time to connect the development board to Edge Impulse.

1. Connect the development board to your computer

Connect Wio Terminal to your computer. Entering the bootloader mode by sliding the power switch twice quickly. For more reference, please also see here.

An external drive named Arduino should appear in your PC. Drag the the downloaded Edge Impulse uf2 firmware files to the Arduino drive. Now, Edge Impulse is loaded on Seeeduino Wio Terminal!

2. Setting Keys

From a command prompt or terminal run:

$ edge-impulse-daemon

NOTE: When connecting to a new device, run edge-impulse-daemon --clean to remove previous cached.

3. Verifying that the device is connected

That’s all! Your device is now connected to Edge Impulse. To verify this, go to your Edge Impulse project, and click Devices. The device will be listed here.

Next steps: build a machine learning model

With everything set up you can now build your first machine learning model with these tutorials:

• Building a continuous motion recognition system.

Looking to connect different sensors? The Data forwarder lets you easily send data from any sensor into Edge Impulse.

Demo: Using Wio Terminal with Edge Impulse to Distinguish Alcohol

In this wiki, we will introduce how to use Wio Terminal with Edge Impulse to simply deploy a machine learning project. Wio Terminal with the Grove systems can be very powerful, which brings hundreds of sensor data in for analysis and to possible evaluate different scenarios!

This is a project inspired by Benjamin Cabé’s Artificial nose project. With this wiki, you will get an understanding of the workflow of Edge Impulse using the Wio Terminal.

Hardware Required

• Wio Terminal
• Grove – Multichannel Gas Sensor v2
• Some Alcohol

Get Started

Let’s work you through the workflow of Edge Impulse with Wio Terminal.

1. Preparations

First, you will need to follow the Wio Terminal Edge Impulse Get Started to upload the Edge Impulse firmware to the Wio Terminal.

Once you have the Edge Impulse Firmware, connect Wio Terminal to your PC and run the Edge Impulse serial tool to establish connections. Run the following in your Terminal:

edge-impulse-daemon

Note Login your Edge Impulse account. You also need to name your device.

The terminal should look something like this:

Now, navigate to Edge Impulse homepage and login the same account. Navigate to Devices and you should see the device:

Now your Wio Terminal is all set and ready to collect data!

2. Collecting Data

Connect the Grove – Multichannel Gas Sensor v2 to Wio Terminal’s Grove I2C port. Place the Grove – Multichannel Gas Sensor v2 onto the alcohol that you are going to test, in my case it’s Beer first.

One thing to be pointed here, the Grove – Multichannel Gas Sensor v2 can be affected by the surrounding easily, you may want to use a cover to make sure it’s sensing the only test content.

On the Edge Impulse dashboard, navigate to Data acquisition, choose your device, and name your Label. As it implies, the label should name according to your test content so here will also be Beer.

Sample length (ms.) will be the length of your sampling time in ms, Sensor chooses External multichannel gas and Frequency choose 100Hz.

Click Start Sampling and it will start to collect data, which the terminal should look something like this:

In my test, I have obtained Beer data for 10s for 6 time and each time they have similar results. You will need to have data sets are fairy similar to each other.

Note If you have data fluctuating a lot and this may be caused by the surrounding.

Once you have enough data for one label you can do the exact same step for other labels! In my test, I have two more data sets Maotai and Liqueur:

• Maotai data set:

• Liqueur data set:

You should see that different alcohol will have different gas values which makes it perfect for machine learning! To add more, it is always better to have more data for it to be trained later, and so please feel free to collect more data!

Note Feel free to do more categories of alcohol!

3. Designing Impulse

Next up, we will need to design the impulse by clicking Impulse Design -> Create Impulse. An impulse takes raw data, uses signal processing to extract features, and then uses a learning block to classify new data. In this example, I added a processing block of raw data with all the input axis and added a Neural Network (Keras) learning block as followed:

Click Save Impulse and under Impulse design click Raw data and you should see the raw features of the data sets:

Click Save parameters then it will navigate to another page. Click Generate Features.

This will generate features from the previous data sets and you will see a graph on the right. If data sets are separated with each other meaning that the data sets are unique to each other which makes it better for machine learning the differences.

4. Training Data

Under Impulse Design, click NN Classifier to configure settings for Neural Network, the following are my settings:

You may want to adjust these settings according to your needs and configure your Neural network architecture, then click Start training! This will perform the training and may take a while.

Once trained, you will see a table of training performance. If your data sets are unique to each other you should get pretty good results! Here is my performance:

From this, you can see that the accuracy is not bad, this is because there is only 3 cases. You may want to add more cases/tests to this example.

5. Live Classification

Now we have the model trained, we can test the model with new data. Navigate to Live classification, and sample new sets of data for testing.

• Testing Example 1:

• Testing Example 2:

As we can see from the results, this is pretty good results where you are able to distinguish alcohol using the Grove – Multichannel Gas Sensor v2 with the help of Edge Impulse!

Further Development

For now, implementing the model back into Wio Terminal as a library is still being developed. It will be released very soon, and then you can detect alcohol ‘offline’ with the trained model loaded onto Wio Terminal!

Wio Terminal Continuous Motion Recognition

In this tutorial, you’ll use machine learning to build a gesture recognition system that runs on Wio Terminal. This is a hard task to solve using rule based programming, as people don’t perform gestures in the exact same way every time. But machine learning can handle these variations with ease. You’ll learn how to collect high-frequency data from real sensors, use signal processing to clean up data, build a neural network classifier, and how to deploy your model back to a device. At the end of this tutorial, you’ll have a firm understanding of applying machine learning in embedded devices using Edge Impulse.

There is also a video version of this tutorial:

1. Prerequisites

For this tutorial, you’ll need a supported device. Follow Wio Terminal Edge Impulse tutorial fist before the following.

Apart from Wio Terminal, here are other supported devices.

• ST B-L475E-IOT01A
• Arduino Nano 33 BLE Sense
• Eta Compute ECM3532 AI Sensor
• Any mobile phone

If your device is connected under Devices in the studio you can proceed:

!!!Note Edge Impulse can ingest data from any device – including embedded devices that you already have in production. See the documentation for the Ingestion service for more information.

2. Collecting your first data

With your device connected, we can collect some data. In the studio go to the Data acquisition tab. This is the place where all your raw data is stored, and – if your device is connected to the remote management API – where you can start sampling new data.

Under Record new data, select your device, set the label to idle, the sample length to 5000, the sensor to Built-in accelerometer and the frequency to 62.5Hz. This indicates that you want to record data for 10 seconds, and label the recorded data as idle. You can later edit these labels if needed.

!!!Note There is also another option for External multichannel gas, this uses the Grove – Multichannel Gas Sensor v2 for collecting gas sensor values.

After you click Start sampling move your device up and down in a continuous motion. In about twelve seconds the device should complete sampling and upload the file back to Edge Impulse. You see a new line appear under ‘Collected data’ in the studio. When you click it you now see the raw data graphed out. As the accelerometer on the development board has three axes you’ll notice three different lines, one for each axis.

!!!Note It’s important to do continuous movements as we’ll later slice up the data in smaller windows.

Machine learning works best with lots of data, so a single sample won’t cut it. Now is the time to start building your own dataset. For example, use the following three classes, and record around 3 minutes of data per class:

• Idle – just sitting on your desk while you’re working.
• Wave – waving the device from left to right.
• Circle – drawing circles.

!!!Note Make sure to perform variations on the motions. E.g. do both slow and fast movements and vary the orientation of the board. You’ll never know how your user will use the device. It’s best to collect samples of ~10 seconds each.

3. Designing an impulse

With the training set in place, you can design an impulse. An impulse takes the raw data, slices it up in smaller windows, uses signal processing blocks to extract features, and then uses a learning block to classify new data. Signal processing blocks always return the same values for the same input and are used to make raw data easier to process, while learning blocks learn from past experiences.

For this tutorial, we’ll use the ‘Spectral analysis’ signal processing block. This block applies a filter, performs spectral analysis on the signal, and extracts frequency and spectral power data. Then we’ll use a ‘Neural Network’ learning block, that takes these spectral features and learns to distinguish between the three (idle, circle, wave) classes.

In the studio go to Create impulse, set the window size to 2000 (you can click on the 2000 ms. text to enter an exact value), the window increase to 80, and add the ‘Spectral Analysis’ and ‘Neural Network (Keras)’ blocks. Then click Save impulse.

Configuring the spectral analysis block

To configure your signal processing block, click Spectral features in the menu on the left. This will show you the raw data on top of the screen (you can select other files via the drop down menu), and the results of the signal processing through graphs on the right. For the spectral features block you’ll see the following graphs:

• After filter – the signal after applying a low-pass filter. This will remove noise.
• Frequency domain – the frequency at which signal is repeating (e.g. making one wave movement per second will show a peak at 1 Hz).
• Spectral power – the amount of power that went into the signal at each frequency.

A good signal processing block will yield similar results for similar data. If you move the sliding window (on the raw data graph) around, the graphs should remain similar. Also, when you switch to another file with the same label, you should see similar graphs, even if the orientation of the device was different.

Once you’re happy with the result, click Save parameters. This will send you to the ‘Feature generation’ screen. In here you’ll:

1. Split all raw data up in windows (based on the window size and the window increase).
2. Apply the spectral features block on all these windows.

Click Generate features to start the process.

Afterward the ‘Feature explorer’ will load. This is a plot of all the extracted features against all the generated windows. You can use this graph to compare your complete data set. E.g. by plotting the height of the first peak on the X-axis against the spectral power between 0.5 Hz and 1 Hz on the Y-axis. A good rule of thumb is that if you can visually separate the data on a number of axes, then the machine learning model will be able to do so as well.

Configuring the neural network

With all data processed it’s time to start training a neural network. Neural networks are a set of algorithms, modeled loosely after the human brain, that is designed to recognize patterns. The network that we’re training here will take the signal processing data as an input, and try to map this to one of the three classes.

So how does a neural network know what to predict? A neural network consists of layers of neurons, all interconnected, and each connection has a weight. One such neuron in the input layer would be the height of the first peak of the X-axis (from the signal processing block); and one such neuron in the output layer would be wave (one the classes). When defining the neural network all these connections are initialized randomly, and thus the neural network will make random predictions. During training, we then take all the raw data, ask the network to make a prediction, and then make tiny alterations to the weights depending on the outcome (this is why labeling raw data is important).

This way, after a lot of iterations, the neural network learns; and will eventually become much better at predicting new data. Let’s try this out by clicking on NN Classifier in the menu.

Set ‘Number of training cycles’ to 1.. This will limit training to a single iteration. And then click Start training.

Now change ‘Number of training cycles’ to 2 and you’ll see performance go up. Finally, change ‘Number of training cycles’ to 100 or more and let training finish. You’ve just trained your first neural network!

!!!Note You might end up with a 100% accuracy after training for 100 training cycles. This is not necessarily a good thing, as it might be a sign that the neural network is too tuned for the specific test set and might perform poorly on new data (overfitting). The best way to reduce this is by adding more data or reducing the learning rate.

4. Classifying new data

From the statistics in the previous step we know that the model works against our training data, but how well would the network perform on new data? Click on Live classification in the menu to find out. Your device should (just like in step 2) show as online under ‘Classify new data’. Set the ‘Sample length’ to 5000 (5 seconds), click Start sampling, and start doing movements. Afterward, you’ll get a full report on what the network thought that you did.

If the network performed great, fantastic! But what if it performed poorly? There could be a variety of reasons, but the most common ones are:

1. There is not enough data. Neural networks need to learn patterns in data sets, and the more data the better.
2. The data does not look like other data the network has seen before. This is common when someone uses the device in a way that you didn’t add to the test set. You can add the current file to the test set by clicking ⋮, then selecting Move to training set. Make sure to update the label under ‘Data acquisition’ before training.
3. The model has not been trained enough. Up the number of epochs to 200 and see if performance increases (the classified file is stored, and you can load it through ‘Classify existing validation sample’).
4. The model is overfitting and thus performs poorly on new data. Try reducing the learning rate or add more data.
5. The neural network architecture is not a great fit for your data. Play with the number of layers and neurons and see if performance improves.

As you see there is still a lot of trial and error when building neural networks, but we hope the visualizations help a lot. You can also run the network against the complete validation set through ‘Model validation’. Think of the model validation page as a set of unit tests for your model!

With a working model in place we can look at places where our current impulse performs poorly…

5. Anomaly detection

Neural networks are great, but they have one big flaw. They’re terrible at dealing with data they have never seen before (like a new gesture). Neural networks cannot judge this, as they are only aware of the training data. If you give it something unlike anything it has seen before it’ll still classify as one of the three classes.

Let’s look at how this works in practice. Go to ‘Live classification’ and record some new data, but now vividly shake your device. Take a look and see how the network will predict something regardless.

So… how can we do better? If you look at the feature explorer on the accX RMS, accY RMS and accZ RMS axes, you should be able to visually separate the classified data from the training data. We can use this to our advantage by training a new (second) network that creates clusters around data that we have seen before, and compares incoming data against these clusters. If the distance from a cluster is too large you can flag the sample as an anomaly, and not trust the neural network.

To add this block go to Create impulse, click Add learning block, and select ‘K-means Anomaly Detection’. Then click Save impulse.

To configure the clustering model click on Anomaly detection in the menu. Here we need to specify:

• The number of clusters. Here use 32.
• The axes that we want to select during clustering. As we can visually separate the data on the accX RMS, accY RMS and accZ RMS axes, select those.

Click Start training to generate the clusters. You can load existing validation samples into the anomaly explorer with the dropdown menu.

Known clusters in blue, the shake data in orange. It’s clearly outside of any known clusters and can thus be tagged as an anomaly.

!!!Note The anomaly explorer only plots two axes at the same time. Under ‘average axis distance’ you see how far away from each axis the validation sample is. Use the dropdown menu’s to change axes.

6. Deploying back to the device

With the impulse designed, trained, and verified you can deploy this model back to your device. This makes the model run without an internet connection, minimizes latency, and runs with minimum power consumption. Edge Impulse can package up the complete impulse – including the signal processing code, neural network weights, and classification code – up in a single C++ library that you can include in your embedded software.

THIS IS STILL UNDER DEVELOPMENT FOR WIO TERMINAL, PLEASE STAY TUNED!

7. Conclusion

Machine learning is a super interesting field: it allows you to build complex systems that learn from past experiences, automatically find patterns in sensor data, and search for anomalies without explicitly programming things out. We think there’s a huge opportunity for machine learning on embedded systems. There are huge amounts of sensor data currently collected, but 99% of this data is currently discarded due to cost, bandwidth or power constraints.

Edge Impulse helps you unlock this data. By processing data directly on the device you no longer have to send raw data to the cloud, but can draw conclusions directly where it matters. We can’t wait to see what you will build!

Reference

• Benjamin Cabé’s Artificial nose project
• Edge Impulse Continuous motion recognition

About Author