Autonomous Shuttle Bus at University of Waterloo: AI-powered Driving Environmental and Traffic Perception

Hardware: reComputer J4012 with NVIDIA Jetson Orin NX module

Use Case Provider: University of Waterloo

Application: Environmental Perception & Path Planning

Industry: Autonomous Driving & Transportation

Deployment Location: Canada

To concur with the complex road environment challenge and also provide a safer campus for a constant stream of pedestrians, bicycles, and vehicles, the University of Waterloo decided to start the research project of deploying the autonomous shuttle bus – WATonoBus equipped with Jetson Orin NX on the vehicle and the infrastructure sensor nodes at the roundabout off the ring road, to collect all perception information for assisting autonomous driving and predicting trajectories of objects at the busy and uncontrolled area.

Challenge

There might be challenges when dealing with the safe interaction of autonomous vehicles in a dynamic environment, where pedestrians, cyclists, and conventional vehicles coexist. This interaction depends on the accurate and effective collection of information from an array of sensors, including cameras, lidar, and radar. The challenge lies not only in the acquisition of this data but also in its precision to enable local mapping capabilities and enhance the estimation of pedestrian and vehicle intents on the road. The subsequent phases of path planning and decision-making are intricately tied to the prediction of potential obstacles and the dynamic states of moving objects. Navigating these complexities necessitates the seamless integration of advanced sensor technologies, robust data processing methods, and sophisticated algorithms to ensure the capability of the autonomous vehicle to make informed and safe decisions on campus.

Solution

Deployed on vehicles and infrastructure sensors at the roundabout

The Autonomous Bus is equipped with an advanced sensor suite:

• 3 front-facing cameras with a 32-line Lidar facing forward,
• 2 side cameras, and one rear-facing camera,
• a 32-line dome Lidar on the vehicle corners to provide full local coverage through an Ethernet port. There are also 2 Radars on the front and rear.
• The high-precision GPS, IMUs, and wheel encoders ensure the accurate position of the vehicle.

The NVIDIA Jetson Orin NX Edge device processes data from these sensors, capable of handling two Baslet dart board-level cameras at up to 160 fps with 1080p resolution each via USB 3 ports, although only 20 fps is required in this campus scenario.

Additionally, the environmental data is collected through solar-powered sensor poles, enabling the creation of a safe navigation space. These sensors track objects on the road, identify a secure drivable area, and share this information with autonomous vehicles upon request. The bus processes the data locally to make informed decisions, ensuring safe navigation in complex scenarios such as roundabouts with raised centers or busy parking lots.

Decision making

The Decision-Making module within the WATonoBus plays a key role in ensuring accurate high-level decisions for the motion of the ego vehicle across diverse driving scenarios. It captures data from an array of sensors responsible for localizing the ego vehicle and assessing the surroundings, incorporating parameters such as position, speed, classification, and confidence estimates.

Focused on anticipating the behaviors of pedestrians, cyclists, and vehicles over a future period, the module employs intelligent planning to facilitate safe decision-making. This is particularly crucial at uncontrolled pedestrian crossings and stop-sign-controlled intersections on campus. By extracting likely estimates of the intent of surrounding entities from rich perception data, the system enhances its ability to navigate these scenarios with foresight and precision.

Path planning

The primary goal of the path/motion planning in this system is to chart the trajectory of the vehicle to align with expected driving behaviors. These anticipated common scenarios include maintaining a regular distance, stopping at specific positions, seamlessly merging into traffic, and executing lane changes. The system should ensure that the vehicle navigates safely, steering clear of obstacles and formulating effective responses in emergency scenarios during each of these situations. Finally, the Vehicle Control module issues precise commands for the front steering angle and traction/braking torques, following the planned path while upholding stability, and facilitating a secure driving experience.

Edge device fleet management by Allxon

Through Allxon’s Fleet Management Solution, the University of Waterloo can remotely manage and monitor the status of the NVIDIA Jetson Orin NX devices in real time. In case of device abnormalities, Allxon Out-Of-Band Technology allows remote system rebooting, eliminating preliminary device issues that may jeopardize safety on the road. Additionally, Allxon offers Over-The-Air (OTA) update functionalities, enabling bulk updates of software, containers, or system configurations remotely. This allows the University of Waterloo to swiftly update all deployed devices on campus after each development, ensuring that all projects continuously run smoothly on the latest versions.

---

Allxon

Allxon envisions a world of open and optimized business operations. Partnering with key IHV and ISV leaders, Allxon rolls out seamless fleet management solutions for top SI and MSP enterprises. With years of deep industry experience, Allxon has pioneered “SaaS-In-Chip” for customizable Out-Of-Band management to make fleet management more open and inclusive. Allxon stands as an industry-first in providing exceptional SaaS solutions, with full hardware and software integration, for powerful fleet management services.

Seeed NVIDIA Jetson Ecosystem

Seeed is an Elite partner for edge AI in the NVIDIA Partner Network. Explore more carrier boards, full system devices, customization services, use cases, and developer tools on Seeed’s NVIDIA Jetson ecosystem page.

Join the forefront of AI innovation with us! Harness the power of cutting-edge hardware and technology to revolutionize the deployment of machine learning in the real world across industries. Be a part of our mission to provide developers and enterprises with the best ML solutions available. Check out our success case study catalog to discover more edge AI possibilities!

Take the first step and send us an email at [email protected] to become a part of this exciting journey! 

Download our latest Jetson product Catalog to find one option that suits you well. If you can’t find the off-the-shelf Jetson hardware solution for your needs, please check out our customization services, and submit a new product inquiry to us at [email protected] for evaluation.

About Author