AORTA — AI-based 3D Traffic Perception and Digital Twin
Within the AORTA research project, DC Vision Systems developed an infrastructure-based 3D perception system for real-time traffic monitoring and digital twin generation.
The system combines multiple stereo camera units with edge computing and AI-based computer vision. Vehicles, pedestrians and other road users are detected, classified and tracked in real time. Information from multiple cameras is fused into a common 3D environment model representing both the static infrastructure and dynamic traffic participants.
This enables applications such as infrastructure-assisted traffic perception, trajectory analysis, traffic monitoring and prioritization of emergency vehicles.
Video above: The video stream of the three cameras is shown on the bottom. On top the digital twin is visualized in realtime in 3D with our Vision-SDKs visualisation.
System Capabilities:
- Real-time 3D environment perception
- AI-based object detection and classification
- Multi-object tracking with 3D bounding boxes
- Multi-camera fusion and vehicle re-identification
- Stereo-based depth estimation
- Dynamic digital twin generation
- Trajectory estimation and prediction
- Edge processing with GPU and FPGA acceleration
- 3D reconstruction from camera and drone data
The camera platform was specifically developed for outdoor infrastructure deployment and supports low-latency real-time processing for intelligent transportation systems.
System Architecture
Three camera systems were installed at each monitored intersection to provide overlapping coverage of the relevant traffic area.
The cameras are connected via Ethernet/PoE to an edge server located close to the intersection. Image processing and object detection are performed locally, enabling low-latency operation without continuously transmitting raw image data to a remote data center.
The processing pipeline consists of camera acquisition, object detection and tracking for each individual camera, followed by multi-camera fusion and 3D environment modelling.
The resulting environment model contains both the static 3D representation of the intersection and a dynamic object list including position, orientation, object class, 3D bounding boxes and trajectories.
The below image shows the system architecture and the dataflow. The cameras are interfaced by Camera-Ineterface block. Object detection and tracking are perfomed for each camera. The object detections and tracks are fused together when processing the environment model. The output is the static 3D model of the crossing and the object list, which includes 3D bounding boxes of the tracked objects.
Scene representation
The scene is represented as a 3D model. The above video shows the reconstructed model.
3D model of the scene with the attached labels, like road, pedestrian, blocked areas, etc.
2D/3D Object tracking
Objects are detected in the individual camera images and tracked over time. Stereo depth information and multi-camera fusion are used to determine their 3D position within the scene.
The system generates 3D bounding boxes and trajectories for vehicles, pedestrians and other traffic participants. Re-identification allows objects to be associated across multiple camera views.
References
AORTA- Seite DE – AORTA Projekt
Technology Transfer
The technologies developed within AORTA contributed to DC Vision Systems’ work in real-time 3D perception, digital twins and embedded computer vision.
Core concepts such as multi-camera processing, 3D environment modelling, object tracking and real-time visualization are also relevant to today’s DC Vision SDK and embedded vision platforms.
Funding Reference
Parts of the presented technologies were developed within the research project:
AORTA – Automatisierte Bildung von Rettungsgassen in komplexen Szenarien durch intelligente Vernetzung
Funded by the German Federal Ministry for Digital and Transport (BMDV/BMVI) under funding references:
45AVF2002G / 01MM20002H
