the latency exibited in cartesian_position

[myendless1]

When projecting the cartesian position onto camera panel, it seems the cartesian is not perfectly aligned with the movement of gripper. The latency from target_cartesian_position is even more severe. Is there a way to relieve this(i tried to use velocity to fix but the result is not satisfying)? Following is a visualized demo.

world_coord_projection.mp4

[XuWuLingYu]

The same issue occurs in my program: the provided wrist pose differs from the ground truth.

[leeibo]

@myendless1 Hello, i'm trying to project the cartesian position onto camera panel too, but i can't find the gripper position, how did you obtain the coordinates of the gripper?

[myendless1]

@myendless1 Hello, i'm trying to project the cartesian position onto camera panel too, but i can't find the gripper position, how did you obtain the coordinates of the gripper?

The gripper position is myself-estimated, by shifting certain value from cartesian_position in xyz directions respectively (as these values are fixed, it's not hard to approximate).

I followed this convert script to read the full dataset gs://gresearch/robotics/droid_raw and used the cartesian_position as robotcoordinate

[leeibo]

@myendless1 Apologies for the interruption. May I kindly ask if the xyz offsets applied to each episode are consistent? Would it be possible to use a fixed xyz offset for the entire dataset? Additionally, could you please share some specific xyz values? Thank you so much for your assistance!
I have tried some parameters and found that the position of the end effector is not consistent. For example, regarding the position at droid_raw/1.0.1/TRI/success/2023-08-07/Mon_Aug__7_13:07:49_2023, as shown in the figure.

However, for droid_raw/1.0.1/AUTOLab/success/2023-07-08/Sat_Jul__8_08:57:28_2023, the cartesian_position is not on the robotic arm.

[myendless1]

@myendless1 Apologies for the interruption. May I kindly ask if the xyz offsets applied to each episode are consistent? Would it be possible to use a fixed xyz offset for the entire dataset? Additionally, could you please share some specific xyz values? Thank you so much for your assistance! I have tried some parameters and found that the position of the end effector is not consistent. For example, regarding the position at droid_raw/1.0.1/TRI/success/2023-08-07/Mon_Aug__7_13:07:49_2023, as shown in the figure. However, for droid_raw/1.0.1/AUTOLab/success/2023-07-08/Sat_Jul__8_08:57:28_2023, the cartesian_position is not on the robotic arm.

the following is my guessed values, i didn't do much visualization on this so i didn't notice the not-on-the-robotic-arm problem (maybe it's the error of that single episode?), i gave up this method once i observed the latency problem of the cartesian position (which is why this issue is opened), therefore these values maybe not that helpful.
self.extra_point_gripper_coords_open = np.array([
np.array([0.0, 0.045, 0.161]), # finger 1 tip
np.array([0.0, -0.045, 0.161]), # finger 2 tip
np.array([0.0, 0.045, 0.13]), # finger 1 end
np.array([0.0, -0.045, 0.13]), # finger 2 end
np.array([0.0, 0.0, 0.13]), # gripper center front
np.array([0.0, 0.0, 0.0]) # gripper center back
])
self.extra_point_gripper_coords_close = np.array([
np.array([0.0, 0.0, 0.167]), # finger 1 tip
np.array([0.0, 0.0, 0.167]), # finger 2 tip
np.array([0.0, 0.0, 0.136]), # finger 1 end
np.array([0.0, 0.0, 0.136]), # finger 2 end
np.array([0.0, 0.0, 0.136]), # gripper center front
np.array([0.0, 0.0, 0.0]) # gripper center back
])