AORTA — AI-based 3D Traffic Perception and Digital Twin
Within the AORTA research project, DC Vision Systems developed an AI-based 3D camera system for real-time traffic perception and digital twin generation for intelligent transportation infrastructure.
The system combines multiple stereo camera units, embedded FPGA/GPU processing and AI-based computer vision algorithms to continuously analyze complex traffic scenarios in real time. Vehicles, pedestrians and other traffic participants are detected, classified, tracked and reconstructed in 3D directly at the edge infrastructure.
A central part of the project was the generation of a dynamic digital twin of urban intersections. The system fuses information from multiple cameras and transforms the extracted scene understanding into a live 3D environment model. This enables infrastructure-assisted traffic monitoring, object tracking, trajectory estimation and emergency vehicle prioritization.
Video above: The video stream of the three cameras is shown on the bottom. On top the digital twin is visaulized in realtime in 3D with our Vision-SDKs visualisation.
The developed technology includes:
- Real-time 3D environment perception
- AI-based object detection and multi-object tracking
- Re-identification of vehicles across multiple cameras
- Edge-AI processing with FPGA/GPU acceleration
- Infrastructure-based digital twins
- 3D reconstruction from camera and drone data
- Smart-city and autonomous driving interfaces
The camera platform was specifically developed for outdoor infrastructure deployment and supports low-latency real-time processing for intelligent transportation systems.
Technology Highlights
- Stereo-based 3D depth estimation
- Real-time object tracking and trajectory prediction
- Digital twin visualization dashboard
- Multi-camera fusion
- Edge computing architecture
- Low-latency AI inference
- Infrastructure-assisted perception
Setup and system architecture
Three cameras have been installed per crossing. The cameras cover a complete3 view of the crosing scenario. Via Ethernet/PoE, the cameras are connected to an edge server at the crossing. The server itself can be accessed with a 4G/5G modem.
The below image shows the system architecture and the dataflow. The cameras are interfaced by Camera-Ineterface block. Object detection and tracing are perfomed for each camera. The object detections and tracks are fused together when processing the environment model. The output is the static 3D model of the crossing and the object list, which includes 3D bounding boxes of the tracked objects.
Scene representation
The scene is represented as a 3D model. The above video shows the reconstructed model.
3D model of the scene with the attached labels, like road, pedestrian, blocked areas, etc.
2D/3D Object tracking
Objects are detected in the images and tracked. Based on this, 3D bounding boxes of the objects are determined and located in the digital twin with high accuracy.
References
AORTA- Seite DE – AORTA Projekt
Research Impact
The technologies developed in AORTA form part of DC Vision Systems’ ongoing work in embedded vision, digital twins, autonomous systems and intelligent infrastructure.
Funding Reference
Parts of the presented technologies were developed within the research project:
AORTA – Automatisierte Bildung von Rettungsgassen in komplexen Szenarien durch intelligente Vernetzung
Funded by the German Federal Ministry for Digital and Transport (BMDV/BMVI) under funding references:
45AVF2002G / 01MM20002H
