Control humanoid avatar hands with VR controllers

[FlorianLance]

Thank you! The script functions perfectly for simple VR objects (for plane and cube at least).

How to adapt the script for "humanoïd avatar" ?
The components expects a str as input and I guess we have to deal with the targets.

Thanks in advance.

Originally posted by @makselro in #3 (comment)

[makselro]

I would like to inspect this file: HumanoidAvatarComponent.cs but I cannot find it in the github.

I would like to try learning myself how to move the avatar (up/down translation, not animate) and deal with targets by inspecting the file.

[FlorianLance]

HumanoidAvatarComponent is not available from the github as it's using closed source code from a thirdparty unity library.
I'm still working on my answer for the avatar hand tracking using VR controllers as it's kind of tricky. It'll be ready soon I guess.

[FlorianLance]

So by using this code, you can move and rotate the hands of a HumanoidAvatar using the VR controllers, however depending the device you may have to tweak a little bit the rotation:

// retrieve controllers  transforms
var controllersTR = ExVR.Display().cameras().cameraRig.transform.Find("Controllers");
var leftC  = controllersTR.Find("Controller (left)");
var rightC = controllersTR.Find("Controller (right)");

// retrieve humanoid avatar component
var hC = (HumanoidAvatarComponent)ExVR.Components().componentPerName["Humanoid avatar"];

// retrieve humanoid avatar arms transforms
var rightArmTr = hC.targets["right_arm"].transform;
var leftArmTr  = hC.targets["left_arm"].transform;

// apply controllers position and rotation to the targets
rightArmTr.position = rightC.position;
rightArmTr.rotation = rightC.rotation * Quaternion.Euler(90, 0, 0);
leftArmTr.position = leftC.position;
leftArmTr.rotation = leftC.rotation * Quaternion.Euler(90, 0, 0);

[FlorianLance]

For initializing correctly a Humanoid Avatar component, you need to define the Asset Bundle containing the avatar:

You can use any AssetBundle for this, the only restriction is the necessity to have a Skin Mesh Rendered with defined bones:

When I was building VR experiments I was mostly using Mixamo avatars and animations (https://www.mixamo.com/#/), just drop them into a Unity project for importing (sometimes you may have to fix missing textures or shaders, but it is not a big deal most of the time), and create an Asset bundle from it.

You can use this code for that: https://github.com/BlankeLab/ExVR/blob/main/unity-projects/exp-launcher/Assets/Scripts/UI/Menu/AssetBundleBuilder.cs

Once this done, you can also create First Perspective Avatar Camera component for adding the head tracking by using VR HMD position:

For fitting correctly it's important to tune the offset:

The code above only work if you don't move the avatar as I was using global positions for keeping the example simple.
We may have to do some maths if you want to keep the hand tracking while moving the Humanoid avatar using its current config transform.