Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend HandPoseController from HandPoseDetector. #20

Merged
merged 1 commit into from
Jul 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 19 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,30 @@ Ensure the existing project is configured with XR hand tracking. The demo projec
The addon files need to be copied to the `/addons/hand_pose_detector` folder of the Godot project.


### Add Hand Pose Detectors
### Add Hand Pose Controllers

Add Hand Pose Detector nodes into the scene - one for each hand.
Add Hand Pose Controller nodes into the scene - one for each hand. These will detect hand poses and drive [XRController3D](https://docs.godotengine.org/en/latest/classes/class_xrcontroller3d.html) nodes.

![Add Hand Pose Detectors](/docs/add_hand_pose_detectors.png)
![Add Hand Pose Controller](/docs/add_hand_pose_controllers.png)

Configure the hand pose detectors with the pose-set to detect, and the hand tracker to monitor.
Add [XRController3D](https://docs.godotengine.org/en/latest/classes/class_xrcontroller3d.html) nodes for the virtual hand-driven controllers - one for each hand.

![Hand Pose Detector Settings](/docs/hand_pose_detector_settings.png)
![Add XRController3D](/docs/add_hand_pose_virtual_controllers.png)

Connect the hand pose detector signals as desired.
Configure the Hand Pose Controller nodes with:
- The [XRControllerTracker](https://docs.godotengine.org/en/latest/classes/class_xrcontrollertracker.html) name for the virtual controller
- The type of pose to drive (`Aim` is most widely supported)
- The hand pose action map for actions to trigger on the virtual controllers
- The [XRHandTracker](https://docs.godotengine.org/en/latest/classes/class_xrhandtracker.html) name for the hand
- The set of hand-poses to detect

![Hand Pose Controller Settings](/docs/hand_pose_controller_settings.png)

Configure the [XRController3D](https://docs.godotengine.org/en/latest/classes/class_xrcontroller3d.html) Virtual Controller nodes with the name of the [XRControllerTracker](https://docs.godotengine.org/en/latest/classes/class_xrcontrollertracker.html) for each hand.

![Virtual Controller Settings](/docs/virtual_controller_settings.png)

If needed, connect the hand pose detector signals. The preferred approach is to generate actions using the hand pose action map, and then detect those actions as if generated by a standing XR controller.

![Hand Pose Detector Signals](/docs/hand_pose_detector_signals.png)

Expand Down Expand Up @@ -119,12 +132,6 @@ The inspect scene provided in the demo project can be used to inspect the flexio
![Inspect Scene](/docs/inspect_scene.png)


## Pose Driven XR Controllers

The Hand Pose Detector addon includes a HandPoseController node which creates an XRControllerTracker capable of generating XR Input Actions in response to hand poses.



## Licensing

Code in this repository is licensed under the MIT license.
Expand Down
1 change: 1 addition & 0 deletions VERSIONS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# 2.0.0 (in progress)
- Rename Action-Set to Action-Map to better match OpenXR naming
- Modify HandPoseController to extend from HandPoseDetector (breaking change)

# 1.3.0
- Fine-tune aim pose
Expand Down
91 changes: 47 additions & 44 deletions addons/hand_pose_detector/hand_pose_controller.gd
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@tool
class_name HandPoseController
extends Node
extends HandPoseDetector


## Hand Pose Controller Node
Expand Down Expand Up @@ -57,75 +57,81 @@ const _POSE_TRANSFORMS_RIGHT : Array[Transform3D] = [
]


@export_group("Controller", "controller_")

## Name for the virtual controller tracker
@export var tracker_name : String = "/user/hand_pose_controller/left"
@export var controller_tracker_name : String = "/user/hand_pose_controller/left"

## Pose type
@export var pose_type : PoseType = PoseType.SKELETON
@export var controller_pose_type : PoseType = PoseType.SKELETON

## Hand poses generating boolean values
@export var action_map : HandPoseActionMap

@export var controller_action_map : HandPoseActionMap

## Hand Pose Detector
var pose_detector : HandPoseDetector

## Controller Tracker
var tracker : XRControllerTracker
var controller_tracker : XRControllerTracker


# Called when the node enters the scene tree for the first time.
func _ready() -> void:
# Call the base
super()

# Skip if in editor
if Engine.is_editor_hint():
set_process(false)
return

# Get the pose detector
pose_detector = get_parent() as HandPoseDetector
if not pose_detector:
set_process(false)
return
# If the hand-pose-set is not specified then construct one dynamically
# from the controller action map.
if not hand_pose_set and controller_action_map:
hand_pose_set = HandPoseSet.new()
for action in controller_action_map.actions:
hand_pose_set.poses.append(action.pose)

# Subscribe to the detector events
pose_detector.pose_started.connect(_pose_started)
pose_detector.pose_ended.connect(_pose_ended)
pose_started.connect(_pose_started)
pose_ended.connect(_pose_ended)

# Create the controller tracker
tracker = XRControllerTracker.new()
tracker.name = tracker_name
XRServer.add_tracker(tracker)
controller_tracker = XRControllerTracker.new()
controller_tracker.name = controller_tracker_name
XRServer.add_tracker(controller_tracker)


# Called every frame. 'delta' is the elapsed time since the previous frame.
func _process(_delta: float) -> void:
func _process(delta: float) -> void:
# Call the base
super(delta)

# Skip if no trackers
if not pose_detector.tracker or not tracker:
if not hand_tracker or not controller_tracker:
return

# Get the hand tracker pose
var pose := pose_detector.tracker.get_pose(&"default")
var pose := hand_tracker.get_pose(&"default")
if not pose:
return

# Get the conversion transform
var hand := pose_detector.tracker.hand
var hand := hand_tracker.hand

# Get the conversion transform
var conv_xform : Transform3D
if hand == XRPositionalTracker.TrackerHand.TRACKER_HAND_LEFT:
conv_xform = _POSE_TRANSFORMS_LEFT[pose_type]
conv_xform = _POSE_TRANSFORMS_LEFT[controller_pose_type]
else:
conv_xform = _POSE_TRANSFORMS_RIGHT[pose_type]
conv_xform = _POSE_TRANSFORMS_RIGHT[controller_pose_type]

# Apply conversion to pose components
var pose_transform := pose.transform * conv_xform
var pose_linear := pose.linear_velocity * conv_xform.basis
var pose_angular := _rotate_angular_velocity(pose.angular_velocity, conv_xform.basis)

# Update the controller tracker pose
tracker.hand = hand
tracker.set_pose(
controller_tracker.hand = hand
controller_tracker.set_pose(
pose.name,
pose_transform,
pose_linear,
Expand All @@ -135,23 +141,20 @@ func _process(_delta: float) -> void:

# Customize properties
func _validate_property(property: Dictionary) -> void:
if property.name == "tracker_name":
if property.name == "controller_tracker_name":
property.hint = PROPERTY_HINT_ENUM_SUGGESTION
property.hint_string = "/user/hand_pose_controller/left,/user/hand_pose_controller/right"
else:
super(property)


# Get configuration warnings
func _get_configuration_warnings() -> PackedStringArray:
var warnings := PackedStringArray()

# Verify tracker name is set
if tracker_name == "":
warnings.append("Tracker name not set")
var warnings := super()

# Verify parent pose
var parent_pose_detector := get_parent() as HandPoseDetector
if not parent_pose_detector:
warnings.append("Must be child of HandPoseDetector node")
# Verify controller tracker name is set
if controller_tracker_name == "":
warnings.append("Controller racker name not set")

# Return the warnings
return warnings
Expand All @@ -160,37 +163,37 @@ func _get_configuration_warnings() -> PackedStringArray:
# Handle start of pose
func _pose_started(p_name : String) -> void:
# Skip if no tracker or action map
if not tracker or not action_map:
if not controller_tracker or not controller_action_map:
return

# Find the action
var action := action_map.get_action(p_name)
var action := controller_action_map.get_action(p_name)
if not action:
return

# Set the input
if action.action_type == HandPoseAction.ActionType.BOOL:
tracker.set_input(action.action_name, true)
controller_tracker.set_input(action.action_name, true)
else:
tracker.set_input(action.action_name, 1.0)
controller_tracker.set_input(action.action_name, 1.0)


# Handle end of pose
func _pose_ended(p_name : String) -> void:
# Skip if no tracker or action map
if not tracker or not action_map:
if not controller_tracker or not controller_action_map:
return

# Find the action
var action := action_map.get_action(p_name)
var action := controller_action_map.get_action(p_name)
if not action:
return

# Set the input
if action.action_type == HandPoseAction.ActionType.BOOL:
tracker.set_input(action.action_name, false)
controller_tracker.set_input(action.action_name, false)
else:
tracker.set_input(action.action_name, 0.0)
controller_tracker.set_input(action.action_name, 0.0)


# Returns an angular velocity rotated by the given basis matrix.
Expand Down
3 changes: 1 addition & 2 deletions addons/hand_pose_detector/hand_pose_controller.tscn
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,4 @@

[node name="HandPoseController" type="Node"]
script = ExtResource("1_peu0g")
tracker_name = null
pose_name = null
controller_pose_type = 1
34 changes: 25 additions & 9 deletions addons/hand_pose_detector/hand_pose_detector.gd
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,18 @@ signal pose_started(p_name : String)
signal pose_ended(p_name : String)


@export_group("Hand", "hand_")

## Name of the hand pose tracker
@export var hand_tracker_name : String = "/user/hand_tracker/left"

## Current hand pose set
@export var hand_pose_set : HandPoseSet

## Name of the hand pose tracker
@export var tracker_name : String = "/user/hand_tracker/left"

## Current hand tracker
var tracker : XRHandTracker
var hand_tracker : XRHandTracker


# Current hand pose data
var _current_data : HandPoseData = HandPoseData.new()
Expand All @@ -42,7 +46,7 @@ var _new_hold : float = 0.0

# Customize the properties
func _validate_property(property: Dictionary) -> void:
if property.name == "tracker_name":
if property.name == "hand_tracker_name":
property.hint = PROPERTY_HINT_ENUM_SUGGESTION
property.hint_string = "/user/hand_tracker/left,/user/hand_tracker/right"

Expand All @@ -62,12 +66,12 @@ func _process(delta: float) -> void:
return

# Skip if no tracker or hand pose set
if not tracker or not hand_pose_set:
if not hand_tracker or not hand_pose_set:
return

# If the palm is not tracked then skip pose detection. Any current pose will
# remain active until we see the hand again.
var flags := tracker.get_hand_joint_flags(XRHandTracker.HAND_JOINT_PALM)
var flags := hand_tracker.get_hand_joint_flags(XRHandTracker.HAND_JOINT_PALM)
if (flags & XRHandTracker.HAND_JOINT_FLAG_POSITION_TRACKED) == 0:
return;
if (flags & XRHandTracker.HAND_JOINT_FLAG_ORIENTATION_TRACKED) == 0:
Expand All @@ -77,7 +81,7 @@ func _process(delta: float) -> void:
var active_pos := _current_pose

# Find the pose
_current_data.update(tracker)
_current_data.update(hand_tracker)
var pose := hand_pose_set.find_pose(_current_data)

# Manage current pose
Expand Down Expand Up @@ -122,7 +126,19 @@ func _process(delta: float) -> void:
pose_started.emit(active_pos.pose_name)


# Get configuration warnings
func _get_configuration_warnings() -> PackedStringArray:
var warnings := PackedStringArray()

# Verify hand tracker name is set
if hand_tracker_name == "":
warnings.append("Hand tracker name not set")

# Return the warnings
return warnings


# If the tracker changed then try to get the updated handle
func _on_tracker_changed(p_name : StringName, _type) -> void:
if p_name == tracker_name:
tracker = XRServer.get_tracker(tracker_name)
if p_name == hand_tracker_name:
hand_tracker = XRServer.get_tracker(hand_tracker_name)
1 change: 0 additions & 1 deletion addons/hand_pose_detector/hand_pose_detector.tscn
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,3 @@

[node name="HandPoseDetector" type="Node"]
script = ExtResource("1_rrxmu")
tracker_name = "/user/hand_tracker/left"
2 changes: 1 addition & 1 deletion assets/inspect/pose_info.gd
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ extends Label3D


## Hand pose to diagnose
@export var pose : HandPose
@export var pose : HandPose

## Name of the hand pose tracker
@export var tracker_name : String = "/user/hand_tracker/left"
Expand Down
Loading