Repository navigation
DenseRewardManager for R mode #15
Description
Hi,
and thank you for your great work on WorldEngine.
I am wondering, does the DenseRewardManager class support R mode? Looking at this code, it seems that reward is always computed using the logged detection tracks, and do not get updated, for instance when using IDM traffic simulation:
Am I interpreting correctly? What is the correct way to generate rewards when simulating rollouts in R mode?
Thank you!
Metadata
Metadata
Assignees
Labels
No labels
Activity
WCJ-BERT commentedon Oct 7, 2026
Thanks for pointing this out.
Your interpretation is correct: the current
DenseRewardManagerpreloads logged detection tracks into observations_list, and scoring still uses that buffer. Although live simulation tracks are collected in detection_tracks_list, they are not currently used for dense reward computation. Directly switching to IDM therefore does not make these rewards reflect reactive traffic.WorldEngine already records the simulated ego and surrounding-agent states through its DataManager, including the motion generated by IDM. We propose reusing these recordings in a two-stage workflow: first collect an R-mode rollout, then convert the recorded OpenScene metadata into a replayable scene and run
DenseRewardManagerwith all agents following the recorded trajectories.Would this two-stage workflow address your use case for computing dense rewards from R-mode rollouts?
DavideA commentedon Oct 8, 2026
Thank you for your answer.
I understand the two stage workflow. But what is the repercussion on the results with / without behavioral model? Were IDM / BWM training rewards scored against prelogged detection tracks, and if so do you expect higher performances with the two-stage workflow? Or were those results collected with the two-stage workflow in place?
WCJ-BERT commentedon Oct 9, 2026
Thank you for the follow-up.
The results reported in our paper were obtained using the two-stage workflow described above. But we have also become increasingly aware of the limitations of this workflow, particularly its separation of rollout collection and reward computation.
We are currently targeting November for an update introducing a main/auxiliary SimEngine setup to support online reward computation in R mode.This update is intended to address those workflow limitations.
We hope this upcoming update will address your concerns and better support your use case.
DavideA commentedon Oct 9, 2026
Thank a lot for your answer!
This clarified my doubts. Looking forward to the update then :)