2D&3D Machine Vision for High Frame Rate Video Streaming Analytics with Depth Cameras

Seeed Hardware: reServer J2032 with NVIDIA Jetson Xavier NX 16GB

Use Case Provider: Aivero

Application: Depth Video Data Capturing and Management & Environmental Perception

Industry: Robotics & Logistics & Manufacturing

Deployment Location: Norway

Background

Analyzing visual data from depth images can be an extremely challenging task for any machine learning professional. Especially when it comes to creating accurate, stable, and automated solutions from a 3d environment. However, emerging tools and technologies are providing promising solutions that could revolutionize the way we capture and handle 2D and 3D visual data from depth cameras. Aivero, the excellent software company who’s focusing on the real-time processing and analysis of depth data demonstrates how ML vision can be leveraged to create streamlined solutions for processing massive amounts of user-generated depth video along with useful tips on how you can begin harnessing the power of advanced AI technology.

Challenge

Companies seeking to implement computer vision and ML automation face a significant challenge due to the complexity of systems, bandwidth limitations, and high storage requirements. One key issue is that the bandwidth bottleneck associated with 3D depth image compression is not illuminated in the market, which mainly focuses on the data rate associated with RGB-D video streams of ever-increasing resolution and frame rate. This makes it difficult to accurately capture real-world geometry, as traditional compression methods are not equipped to handle abrupt depth discontinuities.

Solution

Aivero provides the key technology to break the barrier between the visual data perceived from multiple dimensions and the streaming annotation/recognition in the next step. The cloud-based platform which is capable of camera device management, 2D/3D visual data point cloud preview and storage, is ready for you to transfer the high frame rate compressed depth video, complete the data processing and analyzing through reServer 2032, and then upload these visual data in real-time to ML training frameworks and inferencing tools for the next step.

Aivero’s 3D video compression technology provides easy access to annotation, training, and inference tools for 2D and 3D video combinations. Despite how many types of depth cameras you are using, and how many different data formats you accessed from multiple camera SDKs, Aivero simplifies the steps of producing a colorful depth map and converting the data to a point cloud. You can easily preview the point cloud image from specific depth cameras, store the data, and also complete fleet management of the cameras on the visual data capturing platform & dashboard developed by Aivero.

Business Impact/Result

Compared to the traditional lossless depth compression methods such as RVL, PNG, and LZ4, Aivero delivered an outperforming method not only maintaining a very high image quality level, but also achieving less computationally expensive, operating on an ROI that currently supports up to 3072 depth units, or 3.072 meters when using a 1 mm/step resolution. This real-time, low-latency streaming solution based on RTSP, RTP, and webRTC allows for storing compressed RGB-D video streams to disk. Compressed data can be accessed in C++, Rust, Python, MATLAB, NVIDIA Deepstream SDK, Samsung NNStreamer, GStreamer, and with common deep learning tools such as Tensorflow and MXnet.

Curious about the lossless compression method and how to value the performance metrics? Check out Aivero’s white paper about the experiment for a comparison of the most common depth video compression techniques.

About Aivero

Aivero is a leading software company based in Norway and Denmark.  Its technology is designed to provide real-time processing and analysis of depth data, enabling high-performance applications that require precise depth perception. Aivero simplifies the use of 2D and 3D visual information in computer vision and AI applications, delivering its product as a SaaS or an on-premise hosted system that can be used in a variety of applications such as manufacturing, security, and robotics.

Seeed NVIDIA Jetson Ecosystem

Seeed is an Elite partner for edge AI in the NVIDIA Partner Network. Explore more carrier boards, full system devices, customization services, use cases, and developer tools on Seeed’s NVIDIA Jetson ecosystem page.

Join the forefront of AI innovation with us! Harness the power of cutting-edge hardware and technology to revolutionize the deployment of machine learning in the real world across industries. Be a part of our mission to provide developers and enterprises with the best ML solutions available. Check out our successful case study catalog to discover more edge AI possibilities!

Take the first step and send us an email at [email protected] to become a part of this exciting journey! 

Download our latest Jetson Catalog to find one option that suits you well. If you can’t find the off-the-shelf Jetson hardware solution for your needs, please check out our customization service, and submit a new product inquiry to us at [email protected] for evaluation.