Robosuite-Scenic Integration [Custom only] #383

sahil-tgs · 2025-09-08T08:16:45Z

latest PR: #391

RoboSuite-Scenic Interface

Setup

# Install Scenic
pip install -e .
# Install RoboSuite
pip install robosuite

Main Files

src/scenic/simulators/robosuite/simulator.py
src/scenic/simulators/robosuite/model.scenic

Overview

This new version now fully supports custom environment generation. We can now use native Robosuite Objects like Robots, arena elements like Table, Manipulation Objects like Cube, sphere etc, as components for Setting up different Environment configuration inside Robosuite via Scenic.

Native RoboSuite robots (Panda, UR5e, Sawyer, Jaco, IIWA)
Arena elements (Tables with configurable dimensions)
Primitive objects (Box, Ball, Cylinder, Capsule) with customizable sizes
Complex objects (Milk, Cereal, Bottle, etc.) with fixed dimensions
Dynamic behaviors for robot control

Architecture

We are no longer supporting Robosuite's standard set of test environments called "Tasks", which where earlier supported param use_environment:, now every environment is a custom environment.
The enviroment is custom generated but uses components of Robosuite enviroments like MultiTableArena for Manipulable Table, to build custom workspace/ arena configuration, and EmptyArena for an empty room which is (default env setup).
For any Robosuite Simulation instance, the bare minimum required object is a robot object as an ego object without this no simulation env can be generated. [Limitation]
Simmilarly If one Robot generates it lode ups the default RObosuite enviroment setup with a EmptyArena (empty room) [Limitation]
other than that, we have full controll over how many robots we can define or generate, or no ot table we generate and where we postion them or how many manipulation objects like cubes and spheres we generate. The current code also have extended support for some of the preexisting complex objects in Robosuite, so we can place them too.

Examples

File	Description
`dual_table_workspace.scenic`	Two-table setup with random object placement
`four_table_workspace.scenic`	Four tables with diverse object types
`stacked_cubes.scenic`	Stacked objects on single table
`stack_lift.scenic`	Benchmark - demonstrates behaviors, multi-object scenes

dfremont · 2025-09-16T17:10:30Z

examples/robosuite/stack_lift.scenic

+    with color (0.2, 0.3, 0.8, 1),
+    with width 0.06, with length 0.06, with height 0.06
+
+top_cube = new Box at (0.6, 0, 0.89)


Suggested change

top_cube = new Box at (0.6, 0, 0.89)

top_cube = new Box at (0.6, 0, 0.89),

dfremont · 2025-09-16T17:22:15Z

src/scenic/simulators/robosuite/simulator.py

+import mujoco
+
+try:
+    import robosuite as suite


This import causes robosuite to print various warnings for me even if we won't actually use robosuite (e.g. if you run scenic without the -S option). Unless it would cause other problems, it might be better to not import robosuite at all unless it's going to be used. That's what we do for MetaDrive and CARLA: we set up their model.scenic files to work without needing to import the simulator packages, but their simulator.py files raise an exception if those packages are missing.

dfremont · 2025-09-16T17:24:52Z

src/scenic/simulators/robosuite/simulator.py

+
+    size = config.get('size', [0.02, 0.02, 0.04])
+    if len(size) == 3:  # Convert from [width, length, height]
+        radius = (size[0] + size[1]) / 4  


This doesn't look right, although I'm not sure what you mean by "radius" here.

dfremont · 2025-09-16T17:28:47Z

src/scenic/simulators/robosuite/simulator.py

+    def _position_robots(self):
+        """Position robots based on scenic configuration."""
+        for i, robot_config in enumerate(self.scenic_config.get('robots', [])):
+            if i < len(self.robots):


I don't see where self.robots is initialized (in the superclass __init__?), so I'm not sure how this condition can fail. If it does fail, is skipping positioning one of the robots listed in scenic_config really the right behavior? I would have thought this would mean there is a problem somewhere.

dfremont · 2025-09-16T17:44:09Z

src/scenic/simulators/robosuite/simulator.py

+
+    def __init__(self, render: bool = True, real_time: bool = True, speed: float = 1.0,
+                 env_config: Optional[Dict] = None, controller_config: Optional[Dict] = None,
+                 camera_view: Optional[str] = None, lite_physics: Optional[bool] = None):


Docstring should explain what these optional arguments do. (See CarlaSimulator or MetaDriveSimulator for examples.)

dfremont · 2025-09-16T17:51:58Z

src/scenic/simulators/robosuite/simulator.py

+            if agent in allActions and allActions[agent]:
+                for action in allActions[agent]:
+                    if action and hasattr(action, 'applyTo'):
+                        action.applyTo(agent, self)


This seems to call applyTo twice (first in executeActions above, then here). Do you need this loop at all?

dfremont · 2025-09-16T17:52:29Z

src/scenic/simulators/robosuite/simulator.py

+        for name, body_id in self.body_id_map.items():
+            self.prev_positions[name] = self.data.xpos[body_id].copy()
+
+        action = (self.pending_robot_action if self.pending_robot_action is not None 


This seems to assume there's only a single robot. We should support multiple robots taking actions at the same time.

dfremont · 2025-09-16T17:58:20Z

src/scenic/simulators/robosuite/simulator.py

+        for obj in self.mujoco_objects:
+            @sensor(modality="object")
+            def obj_pos(obs_cache, name=obj.name):
+                return np.array(self.sim.data.body_xpos[self.obj_body_ids[name]])


Do we need to make these observables, or can we just call code like this to directly query the position in getProperties?

dfremont · 2025-09-16T18:03:44Z

src/scenic/simulators/robosuite/simulator.py

+            elif prop == 'gripper_state' and robot_idx is not None:
+                values[prop] = self._get_gripper_state(robot_idx)
+            else:
+                values[prop] = getattr(obj, prop, None)


This doesn't properly update the other dynamic properties like yaw, pitch, roll, velocity, etc. -- those need to have their current values looked up in robosuite.

dfremont · 2025-09-24T23:43:02Z

examples/robosuite/stack_lift.scenic

+    """Pick up the top cube from the stack and lift it."""
+    do PickObject(pickup_object)
+    do LiftToHeight(1.05)
+    for _ in range(10):


If this limit of 10 steps is exceeded, the behavior will return and so the simulation will never end (unless you imposed a time limit when running Scenic). Did you mean to write terminate simulation at the end of the behavior?

This was actually leftover from an earlier debugging session where I needed the simulation to continue running after the behavior completed, it allowed me to inspect the final robot state and manipulate the environment directly through Robosuite's viewer for testing purposes. And yes for the release version, this should properly terminate, I'll add the terminate simulation at the end.

dfremont · 2025-09-24T23:46:10Z

examples/robosuite/stacked_cubes.scenic

+    with color (0.2, 0.3, 0.8, 1),
+    with width 0.06, with length 0.06, with height 0.06
+
+top_cube = new Box at (0.6, 0, 0.89), 


Suggested change

top_cube = new Box at (0.6, 0, 0.89),

top_cube = new Box on bottom_cube,

This should work if the shape of the Box is set properly. If you want the top cube centered on the bottom cube, you can do on top of bottom_cube.

yes in the new PR and most recent commit, I have done exactly that.

dfremont · 2025-09-24T23:47:26Z

src/scenic/simulators/robosuite/model.scenic

+from .simulator import RobosuiteSimulator, SetJointPositions, OSCPositionAction
+
+# Global parameters with defaults matching Robosuite's defaults
+param env_config = {}


Please add documentation for these global parameters to the module docstring (see scenic.simulators.carla.model for an example).

dfremont · 2025-09-24T23:49:38Z

src/scenic/simulators/robosuite/model.scenic

+)
+
+# Default values dictionary
+DEFAULTS = {


I don't think gathering all the defaults into a dict like this makes the code any clearer. I'd just put the values directly into the classes where they are used, so that they're immediately present if you go to look up the default value of a property in the class definition.

dfremont · 2025-09-24T23:50:33Z

src/scenic/simulators/robosuite/model.scenic

+    friction: DEFAULTS['friction']
+    solref: DEFAULTS['solref']
+    solimp: DEFAULTS['solimp']
+    shape: BoxShape()


This is the default value, so you don't need to override shape.

dfremont · 2025-09-24T23:51:05Z

src/scenic/simulators/robosuite/model.scenic

+    solref: DEFAULTS['solref']
+    solimp: DEFAULTS['solimp']
+    shape: BoxShape()
+    allowCollisions: True


Why allow collisions by default? We don't want to be able to place cubes inside the table, for example.

dfremont · 2025-09-24T23:52:40Z

src/scenic/simulators/robosuite/model.scenic

+    width: DEFAULTS['table_width']
+    length: DEFAULTS['table_length']
+    height: DEFAULTS['table_thickness']
+    position: (0, 0, DEFAULTS['table_height'])


This should not be necessary if you create the table with new Table on floor or similar: Scenic will take into account the height of the table.

dfremont · 2025-09-24T23:53:45Z

src/scenic/simulators/robosuite/model.scenic

+    width: DEFAULTS['object_size'] * 2
+    length: DEFAULTS['object_size'] * 2
+    height: DEFAULTS['object_size'] * 2


Suggested change

width: DEFAULTS['object_size'] * 2

length: DEFAULTS['object_size'] * 2

height: DEFAULTS['object_size'] * 2

width: self.radius * 2

length: self.radius * 2

height: self.radius * 2

And likewise for the other shapes below.

dfremont · 2025-09-24T23:55:20Z

src/scenic/simulators/robosuite/model.scenic

+# Robots (matching RoboSuite's naming)
+class Robot(RoboSuiteObject):
+    """Base robot class."""
+    robot_type: "Panda"


For consistency with Scenic, please use camel case for these properties, e.g. robotType

dfremont · 2025-09-24T23:58:42Z

src/scenic/simulators/robosuite/simulator.py

+
+    def checkSuccess(self) -> bool:
+        """Check if task is successfully completed."""
+        if hasattr(self.robosuite_env, '_check_success'):


How can this condition ever be false? More broadly, what's the point of this method?

dfremont · 2025-09-25T00:01:08Z

src/scenic/simulators/robosuite/simulator.py

+
+    def applyTo(self, agent, sim: RobosuiteSimulation):
+        """Apply action to agent."""
+        if hasattr(sim, 'robots') and agent in sim.robots:


The hasattr check will always be true and so is redundant. The other check looks wrong: if agent is not a robot and so does not support this action, the interface should raise an exception rather than silently do nothing. You can use the Action.canBeTakenBy method to restrict which objects can take which actions.

dfremont · 2025-09-25T00:01:46Z

src/scenic/simulators/robosuite/simulator.py

+        """Apply action to agent."""
+        if hasattr(sim, 'robots') and agent in sim.robots:
+            robot_idx = sim.robots.index(agent)
+            if robot_idx < len(sim.robosuite_env.robots):


I'm not sure what this check is meant to do. Can it be false?

dfremont · 2025-09-25T00:02:16Z

src/scenic/simulators/robosuite/simulator.py

+
+    def applyTo(self, agent, sim: RobosuiteSimulation):
+        """Apply action to agent."""
+        if hasattr(sim, 'robots') and agent in sim.robots:


See similar comment for OSCPositionAction.

dfremont · 2025-09-25T00:03:28Z

src/scenic/simulators/robosuite/simulator.py

+        super().setup()
+
+        # Extract configuration from Scenic scene
+        scenic_config = self._extract_scenic_config()


Packing all the Scenic data into a dict here and then unpacking it later seems redundant to me. Can you just pass the RobosuiteSimulation object into ScenicManipulationEnv and access its attributes?

dfremont · 2025-09-25T00:06:25Z

src/scenic/simulators/robosuite/simulator.py

+}
+
+# Object type mapping
+OBJECT_FACTORIES = {


If all the objectType property is used for is to index into this dictionary, it would be simpler to change these lambdas into methods on the original classes. For example the Ball class's implementation of makeRobosuiteObject (or whatever you want to call it) would create the BallObject as below referring to the appropriate attributes of self.

sahil-tgs and others added 8 commits July 28, 2025 20:21

code_optimized

8cfbfe4

Joint Space Controll

8ec7beb

Robosuite Pre-config env only

bde69a3

import fix.scenic

9641225

Robosuite Pre-config env only fixed

d8dfc69

Custom Env

6ba582d

removed redundant class attributes

4ea472f

removed suite.make integration

70272a4

dfremont reviewed Sep 16, 2025

View reviewed changes

dfremont reviewed Sep 24, 2025

View reviewed changes

dfremont reviewed Sep 25, 2025

View reviewed changes

This was referenced Sep 29, 2025

Robosuite-Scenic Interface: XML Injection + Mesh Extraction #391

Open

PR review changes sahil-tgs/Scenic#3

Merged

	top_cube = new Box at (0.6, 0, 0.89)
	top_cube = new Box at (0.6, 0, 0.89),

	top_cube = new Box at (0.6, 0, 0.89),
	top_cube = new Box on bottom_cube,

Robosuite-Scenic Integration [Custom only] #383

Are you sure you want to change the base?

Robosuite-Scenic Integration [Custom only] #383

Uh oh!

Conversation

sahil-tgs commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

RoboSuite-Scenic Interface

Setup

Main Files

Overview

Architecture

Examples

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sahil-tgs Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sahil-tgs commented Sep 8, 2025 •

edited

Loading

sahil-tgs Sep 28, 2025 •

edited

Loading