SERN: Simulation-Enhanced Realistic Navigation for Multi-Agent Robotic Systems in Contested Environments

Jumman Hossain1, Emon Dey1, Snehalraj Chugh1, Masud Ahmed1, MS Anwar1, Abu-Zaher Faridee1,2, Jason Hoppes3, Theron Trout3, Anjon Basak3, Rafidh Chowdhury3, Rishabh Mistry3, Hyun Kim3, Jade Freeman4, Niranjan Suri4, Adrienne Raglin4, Carl Busart4, Timothy Gregory4, Anuradha Ravi1, Nirmalya Roy1
1Department of Information Systems, University of Maryland, Baltimore County, USA 2Amazon Inc., USA 3Stormfish Scientific Corporation, USA 4DEVCOM Army Research Lab, USA
Paper arXiv Video Code(Coming Soon..) Data(Coming Soon..)

SERN's Realistic Navigation with AuroraXR.

Abstract

The increasing deployment of autonomous systems in complex environments necessitates efficient communication and task completion among multiple agents.

This paper presents SERN (Simulation-Enhanced Realistic Navigation), a novel framework integrating virtual and physical environments for real-time collaborative decision-making in multi-robot systems. SERN addresses key challenges in asset deployment and coordination through our bi-directional SERN ROS Bridge communication framework.

Our approach advances the state-of-the-art (SOTA) through:

  • Accurate real-world representation in virtual environments using Unity’s high-fidelity simulator.
  • Synchronization of physical and virtual robot movements.
  • Efficient ROS data distribution between remote locations.
  • Integration of SOTA semantic segmentation for enhanced environmental perception.

Additionally, we introduce a Multi-Metric Cost Function (MMCF) that dynamically balances latency, reliability, computational overhead, and bandwidth consumption to optimize system performance in contested environments.

We further provide theoretical justification for synchronization accuracy by proving that the positional error between physical and virtual robots remains bounded under varying network conditions.

Our evaluations show a 15% to 24% improvement in latency and up to a 15% increase in processing efficiency compared to traditional ROS setups. Real-world and virtual simulation experiments with multiple robots (Clearpath Jackal and Husky) demonstrate synchronization accuracy, achieving less than 5 cm positional error and under 2° rotational error.

These results highlight SERN's potential to enhance situational awareness and multi-agent coordination in diverse, contested environments.

Video

SERN Framework

The SERN framework implementation demonstrates bi-directional communication between physical and virtual robots. The system enables synchronized path planning and cross-domain data sharing across separate physical and virtual networks, supporting coordinated multi-robot operations.

SERN framework implementation diagram

Results

Performance Comparison of SERN ROS Bridge framework vs. Traditional ROS
Performance Comparison of SERN ROS Bridge framework vs. Traditional ROS. We implemented the identical process of increasing the robot numbers by 2, 3 and 5 and recorded the CPU, memory usage, power consumption, and latency overhead of the AuroraXR server (integrated with SERN) and traditional ROS Master. We have chosen the point cloud data generated by each of the robots as data modality to be read and visualized and illustrated the averaged values after 20 iterations of each setup.

BibTeX

@misc{hossain2025sern,
  author       = {Hossain, Jumman and Dey, Emon and Chugh, Snehalraj and Ahmed, Masud and Anwar, M. S. and Faridee, Abu-Zaher and Hoppes, Jason and Trout, Theron and Basak, Anjon and Chowdhury, Rafidh and Mistry, Rishabh and Kim, Hyun and Freeman, Jade and Suri, Niranjan and Raglin, Adrienne and Busart, Carl and Gregory, Timothy and Ravi, Anuradha and Roy, Nirmalya},
  title        = {SERN: Simulation-Enhanced Realistic Navigation for Multi-Agent Robotic Systems in Contested Environments},
  year         = {2025},
  eprint       = {2410.16686},
  archivePrefix= {arXiv},
  primaryClass = {cs.RO},
  url          = {https://arxiv.org/abs/2410.16686}
}