Enhance reward shaping and add tail/arm actuators for T-Rex and Raptor#75
Merged
Enhance reward shaping and add tail/arm actuators for T-Rex and Raptor#75
Conversation
Velociraptor: - Replace passive ball-joint forelimbs with actuated hinge joints (shoulder pitch + roll, kp=15) for future grappling/balance use - Add tail_3_pitch actuator (kp=20) for active distal tail control - Update keyframe qpos (ball→hinge reduces arm DOF from 4 to 2 each) and ctrl for the 5 new actuators (22 total, was 17) T-Rex: - Add tail actuators (tail_1_pitch kp=100, tail_1_yaw kp=80, tail_2_pitch kp=80) to counterbalance the 16.7kg head assembly - Normalize all rewards to [-1, 1] matching the raptor's pattern: forward_vel /8.0, approach_delta /max_delta, tail /10.0, energy /n_act - Port 6 missing reward components from raptor: posture penalty, nosedive penalty, gait symmetry, action smoothness, heading alignment, and lateral velocity penalty (21 actuators, was 18) - Add nosedive termination (forward_z threshold matching raptor logic) Tests updated for new action/observation space dimensions. https://claude.ai/code/session_01CK5BXeF1FvQbRJjjRvZnvM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant enhancements to the T-Rex and Velociraptor simulation environments, focusing on biomechanical realism and reward shaping for reinforcement learning. The most important changes include new actuators for tail and forelimb control, expanded and normalized reward components, and updates to tests to reflect these improvements.
T-Rex Environment Improvements
tail_1_pitch_act,tail_1_yaw_act,tail_2_pitch_act) for active balance control, increasing the action space from 18 to 21. [1] [2]Velociraptor Environment Improvements
Simulation Consistency
These changes collectively improve biomechanical fidelity and reward shaping, enabling more robust and realistic reinforcement learning experiments.