Using Grafana for real-time AI-powered microscopy image analysis at the edge 

In June, Volkov Labs and Theia Scientific published a blog post about using NFS (Network File System) Server as a solution to share external storage between Docker containers on balenaCloud. The Theia web application is based on Balena, Grafana, Ray, NGINX, and PostgreSQL and consists of multiple Docker images.

The architecture is notably built with:

• A PostgreSQL database container with an external NVMe SSD.
• A REpresentational State Transfer (REST) Application Programming Interface (API) server built with Python, which connects to the PostgreSQL database. 
• A Grafana User Interface (UI) that allows users to analyze images and configure the application for optimal performance.
• An NGINX reverse proxy provides access to the REST API and Grafana UI running in separate containers. 

Theia Scientific and Volkov Labs also co-hosted this awesome webinar at GrafanaCONline 2022. The webinar demonstrates the deployment of machine learning (ML) models on an edge computing device for real-time microscopy image analysis using Balena for remote device management and Grafana for application control and visualization. 

Specifically, the Theiascope™ platform created by Theia Scientific provides real-time image and data analysis automation technology for scientists and engineers who conduct research utilizing optical, electron, and X-ray-based microscopy with instrumentation in network- and time-constrained environments. This technology can cut labor costs by 80%, reduce training time and operational expertise, and accelerate the delivery of unbiased results from years, months, days, to seconds in the energy, health, manufacturing, and transportation sectors.

Working with Theia Scientific, it is great to learn the latest from Dr. Christopher Field, Co-Founder, President, and Principal Investigator at Theia Scientific, “We have started development with the Jetson AGX Orin and Jetpack 5.0.2 for our next generation product development, and we are excited to unleash the full power of the hardware and latest drivers for greater performance and flexibility for real-time, quantitative image analysis at the point-of-acquisition.” Get the latest on Jetpack 5.0.2 from NVIDIA from this blog for highlights features. 

Theiascope™ platform

From a software perspective, besides Jetpack v5.0.2 and CUDA v11, the Theiascope™ platform also works with balenaOS and balenaCloud for fleet management, remote updates, and containerizing their application.

The Theia Scientific team utilized a combination of PyTorch, Ray.io, and custom Python code for the server side, a.k.a. “backend”, of their web application. While custom plugins and data sources from the open source community and/or developed by Volkov Labs for Grafana were utilized for the client-side, a.k.a. “frontend”, the component of the software. Volkov Labs has shared many of these developments with the open source and Grafana communities and currently maintains one of the largest collections of plugins for Grafana.

The entire software stack is hosted on NVIDIA Jetson devices, which act as a web server, and the application can be accessed from any computer with a web browser. The whole pipeline is designed and optimized for hosting and running various ML models with Graphical Processing Unit (GPU)-based hardware acceleration.

For hardware, they are using the Jetson AGX Xavier and the Jetson Xavier NX modules attached to the Seeed Studio Jetson Mate carrier board (available at Amazon) and packaged as the hardware for the Theiascope™ platform.

Why NVIDIA Jetson? 

“We initially targeted the Jetson family of embedded devices for the Theiascope™ platform because we needed hardware acceleration from a GPU [Graphical Processing Unit] to run our various ML [Machine Learning] models at 30 FPS [Frames per Second] or higher, to enable real-time automated image analysis and quantitation at electron and optical microscopes.

As part of our product R&D [Research and Development], we tested the entire family of Jetson devices, from the Nano to the AGX, for performance and found the Xavier NX was the minimum hardware specification to run our ML models at real-time speeds. We liked the AGX for its larger GPU memory, which allowed us to host and run larger ML models.

The Jetson Mates were selected to begin an investigation into running multiple ML models simultaneously and act as a portable GPU cluster for site visits and demonstrations.” — Dr. Christopher Field, Co-Founder, President, and Principal Investigator at Theia Scientific 

The following pics are recent deployments that include the Xavier NX (in a case under the network switch), two AGXs, and two Jetson Mates (Fully loaded, one mate has been attched with 3 Jetson Xavier NX modules and 1 Nano and the other cluster has 4 NXs). The fans on the Jetson Mate have been swapped to the white color and disabled the LED of the fan on the other. Many microscopes run in dark rooms, so the bright colors can cause problems.

Edge Computing is must-have for electron microscopes

Edge computing was essential for Theia Scientific because multi-million dollar electron microscopes and instrumentation typically do not have access to Cloud or Internet-based computational resources. Check this blog to learn what, why, and how to get started with Edge AI.

Microscopes are generally deployed in “network-constrained” environments and do not have dedicated GPUs for computation. Thus, it is essential to bring Cloud-like computational resources to the microscope instead of bringing microscopes to the Cloud.

“GPU-enabled edge computing devices allowed us to deploy our ML models and software for network-constrained microscopes without modification of the microscope hardware and zero software installation.”  – Dr. Christopher Field

About Theia Scientific, LLC

Theia Scientific is a technology company that provides unclouded machine vision to microscopy instrumentation and quantitative image analysis workflows. The team is built with experts in edge computing architectures for scientific instrumentation, data analytics, and AI model development. 

About Volkov Labs

Volkov Labs provides open-source and commercial custom plugin development for Grafana. You can learn more on their website and check out their Live project. 

About Grafana 

Grafana is a professional cross-platform open-source data visualization and metric analysis tool developed by Grafana Labs. Grafana allows users to query, visualize, alert, and understand metrics from multiple data sources. No matter where the data is or what type of database, Grafana can be perfectly matched. Check this tutorial to Build your own Grafana dashboard and interface with reTerminal.

About Balena

Balena provides a full technology stack to help customers develop, provision, deploy, and manage Internet of Things (IoT) fleets at any scale. balenaCloud is a Software-as-a-Service (SaaS) platform that enables both application and device management features.

With balenaCloud, one can build and provision application images to hundreds of thousands of devices, using various features like the API and SDKs. In addition, customers can manage fleets at scale using either a friendly GUI or automate tasks with the Command Line Interface.

• Read about our partnership
• ICYMI: Arm Dev Summit, Learn what is LoRa, LoRaWAN and build your first LoRaWAN gateway. Register now and save the date of October 26th.

We are having this workshop with Balena, and we would like to show how easy it is to build a LoRaWAN gateway with a Raspberry Pi and a LoRa concentrator running on balena. The goal of this workshop is twofold: first, to educate developers on this simple and amazing LPWAN technology, and second,  to highlight the partnership among Seeed and balena, which manages +100K LoRaWAN gateways all around the world successfully running on Raspberry Pi 4.

Other resources

• NVIDIA Orin: bring your next-gen AI products with Jetson AGX Orin and NX Orin
•  Compare NVIDIA Jetson AGX Orin with AGX Xavier: 8x AI performance, in-advance Ampere GPU, CPU, Memory & Storage.
• Webinar: Deploy and Scale Edge AI with Jetson Mate
• Tutorial by Janakiram MSV: Edge AI with Triton Inference Server, Kubernetes, Jetson Mate
• NVIDIA Blog: Building a Four-Node Cluster with NVIDIA Jetson Xavier NX

About Author