There are two main sample scripts (SRanipal_AvatarEyeSample_v2, SRanipal_AvatarLipSample_v2) that control the facial tracking in the sample scene.
More information about supported blendshape and API can be found in VIVE official SDK, which can be downloaded from. The “Vive-SRanipal-Unity-Plugin.unitypackage” in VIVE Eye and Facial Tracking SDK has been imported and modified in this demo project.Īccording to the blendshape document provided by VIVE, all the Microsoft Rocketbox Avatars include 42 blendshapes (SR_01 to SR_42). Note: If the project is reporting errors, it is most likely because you need to reimport Steam VR Plugin into your local Unity project via asset store. Then, Steam VR should start automatically when you hit play. This Demo’s VR setting is based on Steam VR, please make sure your Steam VR is working and OpenVR Loader is selected in the project setting. ② Orange: The face tracking device is in idle mode. ① Black: HMD does not support face tracking. There are 3 status modes for launched SR_Runtime： Launch SR_Runtime until the status icon appears in the notification tray. SRanipal Runtime(SR_Runtime) can be downloaded from the VIVE developer portal.
#ROCKETBOX 15S PRO#
A setup guide for VIVE Pro Eye HMD can be found here Set up the lighthouse base stations and headset like a normal VIVE Pro and make sure the lip tracker is plugged in the headset. VIVE Pro Eye HMD and Lip Tracker Installation.VIVE Unity Demo Project Get Started with VIVEīefore opening the Unity demo project, please set up your VIVE devices and Launch SRanipal Runtime first: The mapping file is a json file where one can set a maximum on the threshold or weight of the blenshapes to tune a bit the effects on the animation. You need to set the avatar and head of the target avatar in this component. This file will assign the Action Units from Openface to the blendshapes in the avatar. The Headbox_Openface object in the Hierarchy contains the ZeroMQ receiver to retrieve the data from the Openface Executable and the FaceAnimator component that has the Blendshape Mapping File. If you change the target avatar then you need to modify the Skinned Mesh renderer inside the OVR Lip Synch Context Morph Target component. In all the Microsoft Rocketbox Avatars the first 15 blenshapes are the visemes compatible with Oculus Lipsync, which can be defined in the OculusLipSync object in the Hierarchy.
#ROCKETBOX 15S LICENSE#
This part of the Unity demo needs to be used according to the Oculus SDK License Agreement Using the demo Latest version and documentation is available here Oculus Lipsync is an add-on plugin and set of scripts which can be used to sync mouth movements of a game or other digital asset to speech sounds from pre-recorded audio or live microphone input in real-time. Please check the license in the repo for more details on usage. Openface license states: ACADEMIC OR NON-PROFIT ORGANIZATION NONCOMMERCIAL RESEARCH USE ONLY There are two files from openface training that are too big for githubĪnd place them in OpenFaceRelease\Release\model\patch_experts Openface is only included as a release (no source code) with a direct output via ZeroMQ to the unity project. OpenFace is a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation. This unity demo has dependencies from Openface and Oculus Lipsync therefore we can't provide any particular license and it has to be looked up to the original sources. If you use this library for research or academic purposes, please also cite the aforementioned paper. Presented in the Open Access VR tools and libraries Workshop. Volonte M, Ofek E, Jakubzak K, Bruner S, and Gonzalez-Franco M (2022) HeadBox: A Facial Blendshape Animation Toolkit for the Microsoft Rocketbox Library. The following paper was published at IEEE VR 2022 to coincide with the release of this toolbox, and gives more details of the features included: In this repo you will find a Unity Demo of the Headbox tool for blendshape creation to do facial animation on the Microsoft Rocketbox.Īnd the Python script for Maya to create new blendshapes. An additional Unity demo shows the use these tools with Openface and Oculus Lipsync. These blendshapes have been released with the original library. We have created a total of 15 visemes, 48 FACS, 30 for the Vive facial tracker and 52 ARKit blendshapes.
It includes a tool to create blendshapes out of the facial bones inside Maya and transfer the new blendshapes to the other avatars in the library.
#ROCKETBOX 15S SERIES#
HeadBox Toolkit is a series of opensource tools to do facial animation on the Microsoft Rocketbox avatar library (available here ).
0 kommentar(er)
