[WIP] Add linkages and robots

[pwab]

Hey @TwistedTwigleg. Regarding #1 I was now able to create a first structured test. I'm planning to do the following:

Linkages

Goal: Add a test scene about linkages which are used in machines (and also often seen in games with industrial background)

Scene 1: CrankRocker

• Add a working crank
• Add a crank rocker linkage with the crank as the actuator (LookAt + FABRIK TwoBoneIK)
• Add a crank rocker linkage with the rocker as the actuator (FABRIK + LookAt with constraints)

Scene 2: SliderCrank

• Add a working slider
• Add a slider crank linkage with the crank as the actuator (LookAt + Spring?)
• Add a slider crank linkage with the slider as the actuator (Spring? with constraints + TwoBoneIK)

Robots

Goal: Add a test scene about industrial robots (also often seen in games with industrial background)

Scene 3: RRComparison

• Add a 2DOF RR robot (Scara top view)
• Use FABRIK
• Use CCDIK
• Use TwoBoneIK

Scene 4: TwoDOFRobots

• Add a 2DOF TT robot (Pen plotter)
• Add a 2DOF RT and TR robot

Scene 5: MultiLinks

• Add a RRRRR parallel robot (Pantograph or sometimes 2D-Delta)
• Add a RTRTR parallel robot (Polar plotter)
• Add a TRRR serial robot (a 2d robot arm with gripper plus a driving base)
• Add a 3DOF RRR robot (base, arm and gripper) --> This should be moved to the 3D Test project

Problems right now

• Godot crashes a lot
• The stack of the first linkage isn't working (possibly the settings are wrong)
• How am I supposed to implement translational joints (like sliders)?
• I'm on holidays in the coming weeks. Will come back to this at the end of August.
• Custom Resources cannot be assigned in the editor when using "@tool" godotengine/godot#42134 is still a roadblocker for translational joints

[TwistedTwigleg]

Awesome! I think the outline for the tests should be really helpful for battle testing the 2D IK.

A few questions relating to the problem section:

---

Godot crashes a lot

Crashes related to the modification system, or just crashes in general? In my testing, the modification system seemed mostly stable, but since I know how it works, it is entirely possible (and likely even) that I overlooked something and that's causing crashes.

The stack of the first linkage isn't working

I'm confused, what is not working? The stack doesn't work with the first modifier, or the stack doesn't work when it is initially created? Or it is something else?

If the latter, you just need to reload the scene. It has to do with how Resources are handled by the Skeleton, and unfortunately there isn't really any great way to work around it that I have found. I probably should add a warning or something though on the Skeleton2D about this.
If the former, then that is likely a bug that I need to fix. Right now I'm working on polishing up the 3D, but I'll hop back to 2D soon-ish.
If it is something else, then it is probably a bug that I missed and need to fix.

How am I supposed to implement translational joints (like sliders)?

Like joints that slide on a track/rails? That should be doable, but I do not think any of the modifications I have made right now would work for that. You could make a GDScript SkeletonModification2D to handle it though, if needed.

That said, I could look into adding a track/rails/slider modification, if there is interest in having an official modification for this purpose. I'd just need to run it by my GSOC mentor first, to make sure they are cool with it.

---

Also: Thanks for making this PR and enjoy your holidays!

[pwab]

Crashes related to the modification system, or just crashes in general?

I'm not sure yet because I don't know how stable the current Godot master really is. If I encounter another one I'm going to investigate the logs so that we might get some hints. For now it only slows down my progress.

The stack of the first linkage isn't working

I'm confused, what is not working? The stack doesn't work with the first modifier, or the stack doesn't work when it is initially created? Or it is something else?

Sorry that was more a note to myself. I created the scene in one day and hadn't the time to get a grasp of stacking different kinematics. I just tried it with one linkage and it didn't work and I'm pretty sure I misconfigured something. Give me some time and if I don't get it I will come back to your questions.

How am I supposed to implement translational joints (like sliders)?

Like joints that slide on a track/rails? That should be doable, but I do not think any of the modifications I have made right now would work for that. You could make a GDScript SkeletonModification2D to handle it though, if needed.

Yes exactly. The most simplest example is a device/machine with only one slider as a joint (like a sliding door, camera slider or maybe also a train). There are also kinematics with 2 sliders (TT) like a 2d plotter device or with 3 sliders (TTT) like a 3d-printer. Combined with rotational joints the translational joints allow to create linkages like a slider crank linkage (which is common in motors or can be seen on those old trains/locomotives).

I'm not sure if there is really a need for this type of joint because its behavior can be achieved by simply adapting the bone length while keeping the tip of the bone on a specific axis. Maybe this video helps to understand what I mean - there you see how a translational joint is implemented in the kinematic chain of a slider crank linkage in Blender: https://www.youtube.com/watch?v=n27rgkp5UmQ

[TwistedTwigleg]

I'm not sure yet because I don't know how stable the current Godot master really is. If I encounter another one I'm going to investigate the logs so that we might get some hints. For now it only slows down my progress.

Ah, okay, no biggie. I was just curious because it has been awhile since I've gotten a Modification/IK related crash in 2D.

I probably should rebase the 2D IK PR to the latest version of master, as I imagine it would increase stability, but I'm hesitant to change too much since the end of the GSoC period is almost finished. I might wait until after the GSoC just to be on the safe side.

Sorry that was more a note to myself. I created the scene in one day and hadn't the time to get a grasp of stacking different kinematics. I just tried it with one linkage and it didn't work and I'm pretty sure I misconfigured something. Give me some time and if I don't get it I will come back to your questions.

Sounds good!

Yes exactly. The most simplest example is a device/machine with only one slider as a joint (like a sliding door, camera slider or maybe also a train). There are also kinematics with 2 sliders (TT) like a 2d plotter device or with 3 sliders (TTT) like a 3d-printer. Combined with rotational joints the translational joints allow to create linkages like a slider crank linkage (which is common in motors or can be seen on those old trains/locomotives).
I'm not sure if there is really a need for this type of joint because its behavior can be achieved by simply adapting the bone length while keeping the tip of the bone on a specific axis. Maybe this video helps to understand what I mean - there you see how a translational joint is implemented in the kinematic chain of a slider crank linkage in Blender: https://www.youtube.com/watch?v=n27rgkp5UmQ

It should be possible. I have seen a similar joint called a "Spring Joint" or "Spring IK", that basically does the same thing. If I am understanding correctly, the modification just needs to look at the target, calculate the distance from it to the target, and if the distance is different than the length of the bone, adjust the scale/length of the bone accordingly. In theory, it shouldn't be too hard to implement in 2D, 3D might be a tad more tricky. I'll see if I can get a GDScript version working, since I need to test GDScript powered modifications in 2D anyway and this would be a good way to test it.

[TwistedTwigleg]

Well, I got something sort of working. I have pushed an update to the repo that contains the script (test_scene_10). It uses the old GDScript, rather than the new version of GDScript in master, so it will need updating once the 2D IK PR is rebased with master.

It works fine if you are only operating on a single bone and the child bone doesn't also have a spring modification. When both have a spring modification, it doesn't work and scales strangely. I am fairly certain this is happening because the code doesn't take the current scale of the bone into account when solving, but I haven't found a solution. It does show that it should be possible however.

Additionally, it has helped me fix a bunch of little issues with the 2D IK system and GDScript, which are now currently the PR 👍

[pwab]

1

I synced my branch and looked at the test_scene_10. Seems good so far but had no time to create my own kinematic chain with this method. Will come back to this later.

2

I just tried it with one linkage and it didn't work and I'm pretty sure I misconfigured something.

Aaaand I did. One bone index was -1 and the other was 0 and after changing it (so that I now have 0 and 1 as index) this thing works like expected.

3

One important thing I encountered while working on my scene was a problem with renaming (or deleting) the target node. I wanted to change my target to another node so I thought it wouldn't be a problem to rename the current one to something else and then give the new node the targets node name. Problem was that after renaming my target node to something different I got thousands of messages like

ERROR: Target node is not in the scene tree. Cannot execute modification!
   at: SkeletonModification2DLookAt::execute (scene\resources\skeleton_modification_2d.cpp:428) - Condition "!target->is_inside_tree()" is true.

WARNING: Target cache is out of date. Updating...
   at: SkeletonModification2DLookAt::execute (scene\resources\skeleton_modification_2d.cpp:417) - Target cache is out of date. Updating...

and I was unable to rename my new node because godot almost crashed (to much to handle). It also happens when deleting the target node (so to summarize: if the reference is lost) and the error spam is that much that I'm even not able to deactivate (enabled off) the SkeletonModification. I had to delete my SkeletonModification to get godot out of that warning/error-loop.

Could this be handled differently? I'm working a lot with RigidBodies and their Joints and if the joints node_a or node_b reference is lost or renamed nothing happens. This might be due to the fact that those stuff isn't happening in the editor (tool mode). But even when playing a scene with a joint that has a node_a which is not present in the scene tree the error of a missing reference is only displayed once (if I remember correctly).

4

Another thing I'm now hanging on is the fact that somehow the nodes below the bones do not update their global position. I have a crank with a SkeletonModificationLookAt and it works fine. I added an Area with a CollisionShape to the tip of the bone to grab it when the scene is running (touch control) so that I can rotate the crank. If I drop the node the crank stays at its position and I can see that the Area with the CollisionShape also moved according to the cranks rotation. But after that I'm not able to trigger an event at the current position of the Area - instead it seems it is still at the starting position (even if the CollisionShape is drawn on the moved position). Not sure why this happens. Has this something to do with the bone_local_pose_override and its persistence?

Or do I have to call apply_rest at every bone now if I don't want the skeleton to come back to its original pose?

5

Oh and did you ever encounter such messages in the console window?

early suboptimal
suboptimal

[TwistedTwigleg]

1: ...

No problem!

2: ...

Cool, so nothing broken then. I might add a warning that gets printed though, so its a little easier to tell what is going on for new users.

3: ...

I think I can fix this! I'll look at it today and see what I can do.

4: ...

Hmm, this could be an issue and something I had not thought about...

In the scene, the Bone2D nodes all stay at the same position when IK is running. This is to emulate how the global/local pose override works in 3D, as the bones transforms themselves do not move. To do this in 2D, I am using a small hack/trick with the visual server to offset the drawn location of the Bone2D node. So visually the Bone2D nodes move when overridden using local_pose_override, but in the scene tree they stay in the same position. Again, this was done to emulate how bones work in 3D with the Skeleton3D node, and it also has the added benefit of allowing for interpolation between IK and the Bone2D pose.

However, as you have found, that means that bones attached to the Bone2D nodes will not be in the proper positions when running and instead will stay relative to the Bone2D prior to the modification running. This was something I had not considered prior to making the system, but I can see how it would be an issue as you often would want to have nodes attached to the Bone2D node that are in the correct position...

You can kinda work around this by using get_local_pose_override and offsetting the children bones using that transform, but that would require something like child_node.global_transform = get_local_pose_override(bone.index) * child_node.transform. Maybe I could do this after updating the Bone2D node in Skeleton2D? I'll take a loot at it today and see what I can do!

5: ...

I have not, and I don't think I have any code that prints suboptimal. I wonder what it goes to? My guess is that it's probably related to Vulkan or something. I really should rebase to master to fix editor issues... Will need to do that soon, shortly after GSoC ends I think...

---

Thanks for the feedback! I'll do my best to adapt the PR accordingly and fix the issues mentioned.

[pwab]

Cool, so nothing broken then. I might add a warning that gets printed though, so its a little easier to tell what is going on for new users.

This bone indexing thing is some very cool feature as I don't have to go through my scenetree and instead just push +1 and jump through my skeleton. 👍

For 3: No hurries. I learned that it is often a good idea to deactivate the SkeletonModification at first if I want to change something. And if I do this issue does not arise. But I think new users will run into this so a fix would be quite welcome.

Thanks for the explanation about 4. I almost knew that the local pose is somehow shadowed and I would have to apply the transforms manually. Would have been my next try if you didn't told me exactly that. If that could be automated it would save me a lot of code!
It wasn't an issue until input events of my child nodes where triggered at a very wrong location. Would this happen in 3D as well?

Oh and yes the suboptimal stuff is called from vulkan as can be seen here:
https://github.com/godotengine/godot/blob/a5fb4451210625702474a8fac1dd880e78a409e9/drivers/vulkan/vulkan_context.cpp#L1231

One last question for 'today':
Are angle constraints given in mathmatical positive way (counter clockwise) or the other way around? I'm not sure how other nodes handle this in Godot but I'm sure counter clockwise is always a good standard.
And what is the reference for the 0° position? The rest pose of the constrained bone or the x-axis-vector (1,0) or anything else?

---

I was able to get my first linkage working. I'll clean the visuals and add some code to let the target node move the way I would like it to move but then my first steps are done. I'm going to update this week and the next one and hopefully this PR can be merged then. If everything is fine I'll go deeper into the other modifications (like Physics) and maybe I can improve some of those test scenes a bit.

Thanks for the feedback! I'll do my best to adapt the PR accordingly and fix the issues mentioned.

I only scratched the surface now but this IK-system rocks. I never worked with Skeletons and Bones before but damn it's easy and so is the modification system. Simple but powerful. Also you already did a good job at the documentation. Keep going! :)

[TwistedTwigleg]

Alright! It took awhile, but I think I have fixed the issues in 3 and 4!

Issue 3 has a decent fix, but I'm not completely satisfied with it. It is much better though, so that's a plus. The code shouldn't flood the console with error logs anymore. I also added a few checks when getting external nodes, and for modifications that have multiple joints/bones, the bone index is now printed when giving errors. All in all, it should be better.

For issue 4, I had to rework how the local pose override functions. I had to make some low-level changes with how the modification system works in Skeleton2D and Bone2D, but now the issue of child nodes attached to Bone2D nodes shouldn't be an issue! As a bonus, it doesn't even rely on using RenderingServer as a hack to get the bone position changed. Now Bone2D nodes keep an internal Transform cache when not running IK, and this cache is used for interpolation.

From a user perspective, everything should work exactly the same as before, but the Bone2D nodes will actually rotate and move in the scene. To test it, I used an Area2D on the end of a LookAt node and it seems to be working.

Once you have a chance (and no rush!), do you mind checking to see if the issue is fixed on your end?

The only issue now is that when saving a scene in the Godot editor, the modification/IK poses are now saved rather than the cached transform. I think I can fix this, I just need to figure out how to detect when the scene is about to be saved, and the reset the Bone2D's transform to the cache prior to saving. Edit: Fixed! This is no longer an issue.

---

It wasn't an issue until input events of my child nodes where triggered at a very wrong location. Would this happen in 3D as well?

Thankfully, I expect this to be a 2D only issue. The issue occurs because of how I was "faking" moving the Bone2D node, and in 3D, there is no Bone3D node to fake move. BoneAttachment3D nodes shouldn't have this issue either, though I should double check in the future to be sure.

Now that I'm not faking the movement of Bone2D nodes, things should function in a more straightforward and expected way.

Oh and yes the suboptimal stuff is called from vulkan as can be seen here:

Good to know!

One last question for 'today':
Are angle constraints given in mathmatical positive way (counter clockwise) or the other way around? I'm not sure how other nodes handle this in Godot but I'm sure counter clockwise is always a good standard.

Rotation is handled the same way as in Godot, constraints are applied to the rotation property in the Bone2D. I believe it is counterclockwise.

And what is the reference for the 0° position? The rest pose of the constrained bone or the x-axis-vector (1,0) or anything else?

The reference for the 0° position should be the x-axis-vector for the Bone2D node. That said, the bone_angle property is used in the IK modifications to make the bone point in the right direction, but this is added on top of the rotation property of the Bone2D node. The reason behind this is that in Godot, Bone2D connections are based on positions, so you could have a Bone2D node with a child Bone2D that is directly below it (0, -1), in which case the rotation is zero but the bone angle is 90 degrees because that's the rotation from the Bone2D node to the child Bone2D node, if that makes sense.

I was able to get my first linkage working. I'll clean the visuals and add some code to let the target node move the way I would like it to move but then my first steps are done. I'm going to update this week and the next one and hopefully this PR can be merged then. If everything is fine I'll go deeper into the other modifications (like Physics) and maybe I can improve some of those test scenes a bit.

Awesome! Sounds good to me! Let me know when you are ready to merge and I'll merge it.

I only scratched the surface now but this IK-system rocks. I never worked with Skeletons and Bones before but damn it's easy and so is the modification system. Simple but powerful. Also you already did a good job at the documentation. Keep going! :)

Thanks and will do! 😃

[TwistedTwigleg]

Just an FYI - I need to update the 2D IK PR to the latest version of master to fix a merge conflict that has arisen. I was going to wait until the end of the GSOC, but because of the merge conflict, the CI tests are not running.
The conflict is really minor, thankfully, but there are some stability issues that will need fixing as well. I just wanted to give you a heads up, since the PR may be a tad unstable due to the rebase (which likely will require a rebuild on your side as well. Sorry!)

On the positive side, Godot itself should be more stable once the PR is rebased and from my testing, the overwhelming majority of the IK code works without any issues! Its really only the PhysicalBone2D that is notably affected. Sorry for the inconvenience!

Edit: The PR is now rebased!

Edit 2: Well, the PhysicalBone2D issues are because 2D physics in Godot are broken right now and not related to the modification code nor any changes made in the 2D IK PR.

[pwab]

Okay I'm back at working on this. Updated and rebuild everything. Godot sometimes doesn't want to start up at first but that's fine. I got it to work right now.

I updated my script for controlling my targets with the mouse to the new scripting language (annotations). I only had to change my lines with export. But now another problem arised which I don't have a clue about: When saving the scene (and so also when playing it because it is automatically saved) my reference to a node is lost. I'm not sure why this happens and I'm not getting any error messages. I recorded a short gif about that where you can see the code line and at the end - right before the scene starts up - the reference NodePath is lost.

Any ideas?

---

Apart from that it still runs fine here. No problems with IK stuff found right now. 👍

[TwistedTwigleg]

But now another problem arised which I don't have a clue about: When saving the scene (and so also when playing it because it is automatically saved) my reference to a node is lost. I'm not sure why this happens and I'm not getting any error messages.

Yeah, I have had this issue to. I think its something to do with the new GDScript but I am 100% sure. I think I was able to fix the issue by adding @tool to the script, if I recall correctly with how I fixed the issue when it occurred for me.

Apart from that it still runs fine here. No problems with IK stuff found right now. 👍

Awesome! That's great news.

I'll need to rebase again to master at some point soon-ish, as physics in 2D were broken due to a regression and that was causing issues with the PhysicalBone2D node, but it should be a smaller change than the latest rebase. After that, the 2D PR shouldn't (fingers crossed) need any more rebases prior to a potential merge into Godot itself.

[pwab]

Yeah, I have had this issue to. I think its something to do with the new GDScript but I am 100% sure. I think I was able to fix the issue by adding @tool to the script, if I recall correctly with how I fixed the issue when it occurred for me.

Thanks for the feedback. Haven't tested but I will keep in mind. I also had a lot of problems while working with the SceneTree inspector (while moving nodes). But I'm pretty sure these are only some regressions of the new window system.

---

I now have a first working prototype. I had to reduce the content to the crank rockers so that the scene doesn't get too complex. I'm going to add the other stuff in separate scenes. Only thing right now is the hack I had to use because of the NodePath problem. But that can be easily updated in the future.

Here is a quick preview:

---

Constraints are still hard. Not sure if I already asked but would it be possible to make them visible? This would help alot when using totally obfuscated parameters while enabling localspace and angle inverting.

[TwistedTwigleg]

Awesome! It looks great! 👍

Constraints are still hard. Not sure if I already asked but would it be possible to make them visible? This would help alot when using totally obfuscated parameters while enabling localspace and angle inverting.

It should be possible to make them visible, I just need to add a new function to the modifications called draw_gizmo or something, and then have the Skeleon2D node tell the stack to draw all the gizmos in the editor. I can take a look at implementing it soon, but right now the GSoC work is supposed to be kept in a "as is" state while the final evaluations are going on, according to the email I got from Google. The final evaluations are finished on the 8th, so I can start taking a look at adding a visible component to the modifications in 2D after that.
With the visual component, I think I can get something worked out to show the angle constraints for all the solvers that use them and I can probably get something done to show the Magnet positions as well.

(I'm not sure how I would do the same for 3D, since there isn't a _draw function and making 3D geometry is kinda complex... Will cross that bridge when I get there)

[pwab]

So after f177e90 I'm now half way ready to merge.
The two issues that hold me back are #3 and #4. In fact the last one seems to be just a little visuals bug or something. Nothing critical - but #3 really is a deal breaker for me now.

If you find some time just give me a ping and I'll gladly help solving those problems.

[TwistedTwigleg]

Hi pwab!

So after f177e90 I'm now half way ready to merge.

Great!

The two issues that hold me back are #3 and #4. In fact the last one seems to be just a little visuals bug or something. Nothing critical - but #3 really is a deal breaker for me now.

I think that #3 is actually a GDScript and Godot exported classes problem, rather than something with the 2D IK system. I haven't gone down the rabbit hole to debug the issue just yet, but when I took a curious glance at where the issue was reporting, I found it didn't appear to be anything I touched for influenced. What is strange, at least in my testing, is removing @tool allows the GDScript-powered resources to be added to the modification stack but none of its values are saved, but adding @tool after the resource has been added allows the values to be saved but you cannot add another new GDScript-powered modifications to the stack.
(Also, sorry about not replying or commenting on #3 sooner! I intended to reply to the issue once it came in, but I didn't have time at the time and then I totally forgot. Sorry!)

#4 is a bug with the TwoBoneIK solver. I have a hunch that it occurs when the two bones in the solver are folded into each other completely. I will need to debug this and see if I can fix it.

If you find some time just give me a ping and I'll gladly help solving those problems.

Will do!

Also, I don't think I've mentioned this yet, but there is a now a gizmo-like system for the 2D IK! Now LookAt, CCDIK, and FABRIK all have a simple gizmo that shows their angle constraint bounds in the editor. There are a few minor bugs left in how the gizmo is drawn, but it should make it much easier to setup constraints. TwoBoneIK's gizmo has a simple line that shows the bend direction, and the other IK solvers don't have any gizmo drawing code because i couldn't think of anything that would need to be drawn for them.

Thanks again for all the help!

[pwab]

I think that #3 is actually a GDScript and Godot exported classes problem, rather than something with the 2D IK system.

Ah okay I almost expected that because the issue came in after the rebase. But wasn't sure. The question is if there are other resource types affected by this behavior. Let's see if I can find something in Godots issue tracker.

(Also, sorry about not replying or commenting on #3 sooner! I intended to reply to the issue once it came in, but I didn't have time at the time and then I totally forgot. Sorry!)

Don't worry about it! In the meantime I found the time to read the GSoC submission summaries and - what shall I say? You guys rock! Hope everything is getting merged after testing and tweaking. So much awesome work has been done. And I was sure you'll take a brake from coding at least a few days. 😄

Also, I don't think I've mentioned this yet, but there is a now a gizmo-like system for the 2D IK!

I read every commit description so I'm aware of that. 👍
Had no time to rebuild Godot yet but I'll give you feedback when I have tried it.

[TwistedTwigleg]

I created an issue for #3 on the Godot repository: godotengine/godot#42134. Based on my testing, it seems the issue occurs for all Resources that use @tool.

I've also fixed #4, but I haven't pushed it just yet as I'm hoping to fix up a few things in the 2D IK editor gizmo code first. I'll push the commit later today though.

[pwab]

Okay the problem with TwoBoneIK seems fixed and the angle constraints are shown quite nicely. Good job so far 😄
As long as #3 isn't fixed I cannot get any further with implementing my stuff. Also I'm not able to edit text anymore because of a commit revert that isn't included in your current state (godotengine/godot#42338).

So if you would rebase again I could edit some text, remove the undone parts (for now) and then this could be merged. Then I would like to edit some other aspects of the demos until I can come back to my list of undone linkages/robots.

[TwistedTwigleg]

Awesome! I’m glad angle constraints are easier now.

I will rebase with master tomorrow 👍

[TwistedTwigleg]

The PR is now rebased with the latest version of master, and I added the minimum and maximum drawing gizmo code for TwoBoneIK! I also increased the line thickness of the TwoBoneIK gizmo to two, as I felt it looked better than a line thickness of one.

[pwab]

Great! Tests will follow in the coming days. 🚀

[pwab]

Hey @TwistedTwigleg! "Coming days" seems to be a very elastic term. I'm sorry that I haven't give an update earlier. The end of the last year and the beginning of this one was tougher than I thought.

I saw your post in godotengine/godot#40347 (comment) and I did not want to spoil the pull request review with my comments so I'm commenting here. As I read that your constrains did not work I was surprised but it reminded me on my last attempt to implement FABRIK constraints in 2018 (godotengine/godot-demo-projects#211 (comment) - as you maybe remember). I went into a lot of deadlock situations and only tested it on a human like model - so there might be even more problematic issues.

What I want to say is that I think you shouldn't be too sad (or upset?) because of the fact that it does not work and you only discovered it now at the end of the review process of the pr. You did an amazing job so far and those damn constraints are harder than the FABRIK paper suggests. So don't worry.

As I'm getting more and more time to "reenter" the Godot development process I'd like to dig into that thing. Could you provide me the part of the codebase where you tried to implement constraints for 2d? Maybe I can get an understanding what is happening there and what not.

I don't know if this helps but here is my very dirty implementation in gdscript full of deadlock situations but working somehow (the if constrained should be the most important part apart from the deadlock treatment):

func solve():
	target_reachable = true
	var distance_to_target = (origin.position - target.position).length()
	
	if distance_to_target > total_length:
		# Target is out of reach
		for i in range(joints.size()-1):
			var residual = target.position - joints[i].position
			var factor = lines[i].length() / residual.length()
			# Find new joint position
			joints[i+1].position = (1 - factor) * joints[i].position + factor * target.position
			
			if constrained:
				# Update angle
				initial_vectors[i] = get_initial_vector(constrained_vectors[i])
				current_vectors[i] = get_current_vector(connected_joints[i])
				angles[i] = get_angle(current_vectors[i], initial_vectors[i])
			
				# If the chain is streched to full length the constraints should be considered as well
				if not within_limits(angles[i], angle_constraints[i]):
					var nearest_angle = clamp(angles[i], angle_constraints[i][0], angle_constraints[i][1])
					connected_joints[i][1].position = current_vectors[i].rotated(deg2rad(angles[i] - nearest_angle)) + connected_joints[i][0].position
	
	else:
		# Target is in reach
		computing_counter = 0
		delta = get_delta()
		var succesful = fabrik(delta)
		
		if not succesful:
			print("--- Not succesful ---")
			# It could be a deadlock situation or the chain cannot reach the target because of constraints
			# Not sure if this is working or not
			var bending_success = false
			last_delta = 0
			for i in range(joints.size()-2, -1, -1):
				print("Joint ", i)
				bending_success = bend_link(i)
				if bending_success:
					break

			# Target is unreachable
			if not bending_success:
				print("Target unreachable")
				target_reachable = false
			
	
	# Update angles
	if constrained:
		initial_vectors = get_initial_vectors()
		current_vectors = get_current_vectors()
		angles = get_angles()
		
func bend_link(i):
	for bending in range(2, 360, 2):
		#print("Bending ", bending)
		joints[i+1].position = joints[i].position + (joints[i+1].position - joints[i].position).rotated(deg2rad(-bending))
		
		backward()
		forward()
		delta = get_delta()
		
		if delta <= TOLERANCE:
			print("Bending success")
			return true
	return false

func forward():
	# Forward reaching: Set root at initial point
	#TODO: Rename variables to their meanings
	joints[0].position = origin.position
	for i in range(joints.size()-1):
		var r = joints[i+1].position - joints[i].position
		var l = lines[i].length() / r.length()
		# Find new joint position
		joints[i+1].position = (1 - l) * joints[i].position + l * joints[i+1].position
		
		if constrained:
			# Update angle
			initial_vectors[i] = get_initial_vector(constrained_vectors[i])
			current_vectors[i] = get_current_vector(connected_joints[i])
			angles[i] = get_angle(current_vectors[i], initial_vectors[i])
		
			# Check for constraints
			if not within_limits(angles[i], angle_constraints[i]):
				var nearest_angle = clamp(angles[i], angle_constraints[i][0], angle_constraints[i][1])
				connected_joints[i][1].position = current_vectors[i].rotated(deg2rad(angles[i] - nearest_angle)) + connected_joints[i][0].position

func backward():
	# Backward reaching: Set end effector as target
	#TODO: Rename variables to their meanings
	joints[joints.size()-1].position = target.position
	for i in range(joints.size()-2, 0, -1):
		var r = joints[i+1].position - joints[i].position
		var l = lines[i].length() / r.length()
		# Find new joint position
		joints[i].position = (1 - l) * joints[i+1].position + l * joints[i].position

I really hope that the 2d and 3d pr is merged in the coming days. So that you can take a vacation eventually 😄

[TwistedTwigleg]

Hey @pwab!

Thank you for the response, I really appreciate it!

Constraints are rather difficult. I'm not too upset that they are not working in the 2D PR, I'm more upset that I didn't do more thorough testing and didn't notice it was broken. I am glad I found it now though rather than after its pulled-in and users expect it to be working when in fact it doesn't.

Thank you for sharing the code you used! I'd love to share the code I have used for angle constraints if it would help!
Here's a zip containing the files mentioned below: Constrained_FABRIK_CodeFiles.zip

---

For the GSoC implementation, I have both attached a file containing just the FABRIK modification and you can also find the file with constraints implemented here in this this commit.

The key functions are the forward pass, backwards pass, and the execute function. As mentioned in the PR, it seems to work only on the first joint and only when its not rotated.

---

Additionally, I have also included the constrained FABRIK solution I wrote for Twisted IK 1. It is fully written in GDSctipt and is quite old, but maybe it will help as a reference? Twisted IK 1 uses Bone2D nodes directly and lets the Bone2D node handle the bone angle rotation offsets, but outside of that it's fairly similar to the system written for GSoC.

From what I can tell, this implementation does work with constrained bones going down the chain. The important parts for constraints are in the chain_forward and chain_backwards functions. I think the key to the implementation is in the _calculate_previous_bone_angles function, from what I remember. I'm not sure how to apply it to a system like GSoC though, but to be honest I haven't had the time to look into it deeply yet either.

The code is messy though and was not written for speed, readability, code quality, or really to be read by anyone outside of myself, so its rather rough around all the edges 😆 .
I'm not sure how helpful it will be, but because it kinda works I though I'd include it in case it can help somehow.

---

I salute you for trying to figure out constraints and please let me know if you figure anything out! Likewise, if you have any questions please do not hesitate to ask and I will try to respond when possible.

Likewise, I also hope the IK PRs will be pulled in soon, but if not, I am not too worried. I plan to continue to support Twisted IK 2, so the IK stuff will not be lost and will be usable for Godot developers regardless of the fate of the PRs 😁

[TwistedTwigleg]

Hey! I forgot to mention this here until now, but I released Twisted IK 2 version 2.1.0, which includes a semi-working constrained FABRIK in 2D. It has some issues, but it seems to be fairly stable. The biggest issue it has is that its not very good at accounting for contracting to reach the target, so sometimes it gets into situations where it could reach the target but it doesn't think it can. Additionally, the code I'm using for angle constraints seems to cause it to want to snap to the wrong extreme end of the clamped angled. I released the code under MIT in this GitHub gist so, in case it helps with implementing constrained FABRIK in the GSoC stuff.

[pwab]

I'm totally aware of your plugin and it's on my whishlist. 😄
Thanks for sharing the code. Will definitly look into it.

---

As we are talking about your plugin: Is it also possible to implement slider/translational joints (aka spring-joints) there? We talked about that earlier here and I had to stop tinkering with the PR implementation because of the resources bug (which still exists as far as I know).

[TwistedTwigleg]

I'm totally aware of your plugin and it's on my whishlist. smile
Thanks for sharing the code. Will definitly look into it.

Awesome, hopefully it will help 😄

As we are talking about your plugin: Is it also possible to implement slider/translational joints (aka spring-joints) there?

Yeah, it should be possible! All of the modifications work on the Bone2D nodes directly, so it should just primarily be a case of getting the logic sorted out for something like that. Adding a joint that slides on a rail should be doable with relative ease, and a spring joint should also be doable. I can probably get a sliding rail joint bashed out and maybe even a spring joint, I would just need to think of how to approach it.
(Yeah, I think the resource bug is still an issue currently)

Edit: It should also be doable in 3D since the plugin uses Bone3D nodes! Though the implementation is a little difficult simple because of that pesky third dimension stuff 😆

[pwab]

Hello again. After a long break I'm finally back at this lovely thingy. Congrats for getting this massive improvement merged into Godot master! 🥳

I updated my fork to the newest Godot version. Most things still worked. Just some observations:

• Somehow the UI-nodes got a refresh which destroyed half my ui stuff. Had to manually redo some of those.
• Polygon2Ds seem to disappear in-game. They also don't react in editor on some settings changes.
• There are more cache and import files now. I hesitated to add the .godot repository to the .gitignore but I'm pretty sure this folder should be excluded.
• The project file was automatically updated with some changed @ to &. I could push those changes but I saw that also one of the SkeletonModification2DSpring lines got updated which are coming from Test_10 and this stuff is not part of my PR here. Any drawbacks in pushing those changes?

After updating everything it is now working as before. Some points which I observed:

• Custom Resources cannot be assigned in the editor when using "@tool" godotengine/godot#42134 is still a roadblocker not allowing me to implement sliders or translational joints in general.
• The tool-script for SkeletonModification2DSpring has some error on line 66: Parser Error: Cannot find property "xform" on base "Transform2D". Maybe the API changed here.
• Somehow empty modifications are added to my ModificationStack. I manually deleted the empty ones. Not sure why they were added automatically in the first place but they are coming back after reloading the scene. Any ideas why this happens?
• Apart from working correctly I'm getting errors from my modifications. I deactivated both crank rockers in the Test_CrankRocker scene to only see the errors coming from the simple crank (1 modification with look_at and nothing else). I received 14 errors and 12 warnings and it seems deactivating the modifications didn't reduced this much. Some of the warnings like the outdated target cache aren't new for me. But some of them I'm not able to understand. Following the most interesting ones:

W 0:00:01:0135   SkeletonModification2DLookAt::set_bone_index: Cannot verify the bone index for this modification...
  <C++-Quellcode>scene\resources\skeleton_modification_2d_lookat.cpp:255 @ SkeletonModification2DLookAt::set_bone_index()
E 0:00:01:0138   SkeletonModification2DLookAt::update_bone2d_cache: Cannot update Bone2D cache: modification is not properly setup!
  <C++-Quellcode>scene\resources\skeleton_modification_2d_lookat.cpp:199 @ SkeletonModification2DLookAt::update_bone2d_cache()
E 0:00:01:0141   SkeletonModification2DLookAt::update_target_cache: Cannot update target cache: modification is not properly setup!
  <C++-Quellcode>scene\resources\skeleton_modification_2d_lookat.cpp:264 @ SkeletonModification2DLookAt::update_target_cache()
E 0:00:01:0186   Transform2D::affine_invert: Condition "det == 0" is true.
  <C++-Quellcode>core\math\transform_2d.cpp:49 @ Transform2D::affine_invert()

Any ideas on how to solve them?

I'm very excited that these examples now work with the official Godot master. That will make testing and exploring the IK stuff easier with alpha and beta builds in the future. 🚀

[TwistedTwigleg]

Hello again. After a long break I'm finally back at this lovely thingy. Congrats for getting this massive improvement merged into Godot master! partying_face

Thanks and welcome back! 😃

I updated my fork to the newest Godot version. Most things still worked. Just some observations:

* Somehow the UI-nodes got a refresh which destroyed half my ui stuff. Had to manually redo some of those.

Yeah, that happened to me as well. I think splitting the modifications into their own files changed how they are registered, which messes up existing projects using the modifications. Sorry it happened though, it's certainly no fun to redo a bunch of work.

* Polygon2Ds seem to disappear in-game. They also don't react in editor on some settings changes.

Strange. I do not think I touched any Polygon2D code, so it may be a bug in master. Not totally sure though.

* There are more cache and import files now. I hesitated to add the `.godot` repository to the `.gitignore` but I'm pretty sure this folder should be excluded.

I think the .godot folder can be ignored but I am not totally sure. No harm either way though, it can always be removed later if needed.

* The project file was automatically updated with some changed `@` to `&`. I could push those changes but I saw that also one of the `SkeletonModification2DSpring` lines got updated which are coming from `Test_10` and this stuff is not part of my PR here. Any drawbacks in pushing those changes?

I think it's fine and there shouldn't be any draw backs. I haven't worked on the 2D test project in quite awhile, since before it was merged in, so it probably is just changes since the last time it was opened. Feel free to change whatever you need!

After updating everything it is now working as before. Some points which I observed:

* [Custom Resources cannot be assigned in the editor when using "@tool" godotengine/godot#42134](https://github.com/godotengine/godot/issues/42134) is still a roadblocker not allowing me to implement sliders or translational joints in general.

Yeah, unfortunately until that issue is fixed, custom modifications are going to be problematic.

* The `tool`-script for `SkeletonModification2DSpring` has some error on line 66: `Parser Error: Cannot find property "xform" on base "Transform2D".` Maybe the API changed here.

It looks like the function is still there in the Transform2D header, but maybe its no longer exposed to GDScript? I'll look into it once I'm working on 2D IK again.

* Somehow empty modifications are added to my ModificationStack. I manually deleted the empty ones. Not sure why they were added automatically in the first place but they are coming back after reloading the scene. Any ideas why this happens?

Strange. I have never had this issue myself and I do not think the ModificationStack caches the resources except by storing them in a list/vector. I honestly have no idea why this happens, but its rather concerning that it occurs.

Apart from working correctly I'm getting errors from my modifications. I deactivated both crank rockers in the Test_CrankRocker scene to only see the errors coming from the simple crank (1 modification with look_at and nothing else). I received 14 errors and 12 warnings and it seems deactivating the modifications didn't reduced this much. Some of the warnings like the outdated target cache aren't new for me. But some of them I'm not able to understand. Following the most interesting ones:

W 0:00:01:0135   SkeletonModification2DLookAt::set_bone_index: Cannot verify the bone index for this modification...
  <C++-Quellcode>scene\resources\skeleton_modification_2d_lookat.cpp:255 @ SkeletonModification2DLookAt::set_bone_index()
E 0:00:01:0138   SkeletonModification2DLookAt::update_bone2d_cache: Cannot update Bone2D cache: modification is not properly setup!
  <C++-Quellcode>scene\resources\skeleton_modification_2d_lookat.cpp:199 @ SkeletonModification2DLookAt::update_bone2d_cache()
E 0:00:01:0141   SkeletonModification2DLookAt::update_target_cache: Cannot update target cache: modification is not properly setup!
  <C++-Quellcode>scene\resources\skeleton_modification_2d_lookat.cpp:264 @ SkeletonModification2DLookAt::update_target_cache()
E 0:00:01:0186   Transform2D::affine_invert: Condition "det == 0" is true.
  <C++-Quellcode>core\math\transform_2d.cpp:49 @ Transform2D::affine_invert()

Any ideas on how to solve them?

I'm not totally sure. Some of the errors/warnings are not really warnings or errors, it's just printing information for the user. I really should probably add a boolean or something that prevents errors from printing on initialization, as several errors/warnings could be completely avoided.

I believe the first error occurs when a bone index is set before the modification has a reference to a Skeleton2D. This makes it where it cannot verify if the bone exists or not.

The second and third error is probably the same thing, especially if it occurs only once at the beginning. The issue is almost certainly that the Skeleton2D is null and so it cannot execute because it cannot access the Skeleton2D to set the local pose overrides. Likely the automatic property setting is running before the modification has had it's setup function called, leading to the Skeleton2D being null.

The last error I'm not totally sure on, to be honest. Normally that type of error occurs when the target in the look_at function is equal to the position of the node look_at is being called on. Probably to fix this I need to add a check in the LookAt modification to check if the target position is equal to the current bone position.

---

Thanks for checking out the code and trying it again. I really appreciate all the help!

Right now I'm trying to get the 3D IK PR in a state where it can be merged into master, and then I'll come back to the 2D IK stuff and start fixing it all up. I'm hoping to have the 3D IK PR merge-ready soon, so hopefully it will not be too long before I can start looking at the 2D IK stuff again 🤞

[pwab]

Apart from the issues which come from master branch the examples still work as expected. Only thing is this strange ModificationStack issue. Would be interesting to see if this also occurs on the 3D side. Maybe I find the time to test this.

Right now I'm trying to get the 3D IK PR in a state where it can be merged into master, and then I'll come back to the 2D IK stuff and start fixing it all up. I'm hoping to have the 3D IK PR merge-ready soon, so hopefully it will not be too long before I can start looking at the 2D IK stuff again 🤞

Take your time. Right now the 2D side works and I really need to find the time to just use it in some more sample projects. But I keep my fingers crossed for your 3D IK PR. 🍀

[pwab]

Woah I havn't seen that godotengine/godot#42134 was closed. 😮
Will come back to this as soon as I can! 🚀