Switch to using MeshCat.MeshFileGeometry and MechanismGeometries.MeshFile. #32

tkoolen · 2019-05-15T04:30:54Z

On top of #31.

Part of the plan from rdeits/MeshCat.jl#92 (comment). Ref JuliaRobotics/MechanismGeometries.jl#17.

Requires version tags for MeshCat, ValkyrieRobot, and MechanismGeometries.

Seems to be working though!

It does seem to be a lower-poly model than before though... Is there a way we can control that?

Also turned some really long short-form functions into long-form functions to improve readability; sorry, should have split that out.

update lower bounds in Project.toml once tags are in.

tkoolen · 2019-05-15T05:07:37Z

I tried this with https://github.com/tkoolen/universal-robot-dae-jl.

For tag="collision" (STL files), everything works perfectly. However for the tag="visual" DAE files, only the base shows for some reason. If I change all the links in the URDF to point to the base DAE file, then I do get multiple base links at different locations. I thought it might be a file size issue, as the base file is smaller than most, but the wrist3 file is even smaller so maybe that's not it. Any ideas?

rdeits · 2019-05-15T12:02:04Z

Hm, interesting. Does anything show up in the browser's console when you try to load the bad meshes? I can also look into this after work tonight.

tkoolen · 2019-05-15T13:31:10Z

Looks like there are indeed some issues:

tkoolen · 2019-05-17T20:36:16Z

Chrome on Linux seems to give a more helpful error message:

Uncaught TypeError: Cannot set property 'uuid' of undefined
    at handle_special_geometry (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:37)
    at ExtensibleObjectLoader.delegate (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:37)
    at ExtensibleObjectLoader.parseGeometries (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:37)
    at ExtensibleObjectLoader.parse (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:3)
    at Viewer.set_object_from_json (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:37)
    at Viewer.handle_command (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:37)
    at Viewer.handle_command_message (73edcf236a5dc82f052d45ffa75c1c3d7774fe94-main.min.js:37)
    at Object.eval (eval at <anonymous> (bundle.js:3059), <anonymous>:2:353)
    at Object.dispatch (bundle.js:177)
    at WebSocket.ws.onmessage (mux_setup.js:25)

tkoolen · 2019-05-17T20:41:18Z

So I guess the issue is here:

https://github.com/rdeits/meshcat/blob/4fc7279bfd3557d437ebe6210f5e2d70c54b7611/src/index.js#L58

tkoolen · 2019-05-17T21:45:37Z

I think I'm starting to understand what's going on. The meshcat code assumes that obj.scene.children[0] is the object we care about, but for these meshes it's just some random camera:

Indeed, for the .dae that does render, there's only one child.

rdeits · 2019-05-19T03:21:25Z

Oh, wow, good catch. Hmm, I'm not sure what the right way to hand this is--general mesh loading is weirdly complicated precisely because each file tries to be an entire scene rather than just a single geometry. I think I just need to adjust the loader in meshcat to try to add all of the geometries in the file.

tkoolen · 2019-05-19T16:03:00Z

Yeah, that sounds good. Most likely all the lamps and cameras left in the .dae file for a robot part aren't really meant to be in there anyway.

rdeits · 2019-05-19T18:53:31Z

Oh man, this is compilcated. I think I need to re-think the way mesh files are handled--it's currently assumed internally that each mesh file produces some kind of three.js Geometry, but a .DAE file is really more like its own entire scene, which we'd need to represent as a three.js Group. A Group is an Object (a class containing its own geometries and materials), not a Geometry in three.js-land, so this will take some doing.

A related question is: what exactly do we do with the materials inside a mesh file? It seems reasonable to provide an STL mesh geometry and a separate material specification, but a DAE file includes all of its material specifications internally. Should we let a user provide a material in that case? And what object would we apply that material to?

I think the most sensible answer is probably to change MeshFileGeometry <: MeshCat.AbstractGeometry into MeshFileObject <: MeshCat.AbstractObject and then load it on the javascript side as a Group (with potentially many geometries and materials). That means we'd be taking whatever material is specified in the file, without letting the user override that material in Julia.

Then, this gets even more complicated because OBJ files store their material as a reference to an .mtl file, so having the full file contents isn't sufficient to render the mesh and its materials.

rdeits · 2019-05-19T19:31:45Z

Ok, with meshcat-dev/meshcat#51 and this PR and all its friends, the UR10 now loads properly:

It looks a little goofy, though, because the DAE files include their own light sources for no apparent reason. We can either handle that by stripping them out in meshcat (on the javascript side), or just declare that we're doing the right thing and the mesh itself is silly.

…File.

rdeits · 2019-07-12T03:35:27Z

whoops, still need a tag for MechanismGeometries 0.4.0: JuliaRegistries/General#1966

codecov-io · 2019-07-12T12:20:30Z

Codecov Report

Merging #32 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #32      +/-   ##
==========================================
+ Coverage   96.69%   96.72%   +0.02%     
==========================================
  Files           5        5              
  Lines         121      122       +1     
==========================================
+ Hits          117      118       +1     
  Misses          4        4

Impacted Files	Coverage Δ
src/MeshCatMechanisms.jl	`100% <ø> (ø)`	⬆️
src/visualizer.jl	`94.11% <100%> (+0.11%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dfbbc2d...fa1ef92. Read the comment docs.

tkoolen · 2019-07-12T12:34:54Z

OK, build is passing. I'm going to quickly merge and tag a new version, since the latest release is now incompatible with the new MechanismGeometries version.

tkoolen · 2019-07-12T14:51:12Z

Hmm, so with the new versions of everything I tried the UR10 again (https://github.com/tkoolen/universal-robot-dae-jl), and ran into #32 (comment) again.

We're currently creating MeshFileGeometrys in this package. So we end up over here in meshcat:

https://github.com/rdeits/meshcat/blob/3122cecd5da022ad96bb0c8dc0f811a1bc492350/src/index.js#L63-L66

But the changes to those lines in meshcat-dev/meshcat#51 are not present in master.

Should we be creating MeshFileObjects here instead? Or does the handling of meshfile_geometry need to be fixed in meshcat?

tkoolen · 2019-07-12T15:12:36Z

With https://github.com/JuliaRobotics/MeshCatMechanisms.jl/compare/tk/meshfileobject-hack?expand=1, I the .daes at least load without error, but the poses are all wrong:

And I was under the impression that the idea was to use MeshFileGeometry.

tkoolen · 2019-07-12T16:47:36Z

Another thing I just noticed is some strange behavior with .obj files and transparency:

Maybe a face orientation issue?

tkoolen · 2019-07-13T03:22:00Z

We can either handle that by stripping them out in meshcat (on the javascript side), or just declare that we're doing the right thing and the mesh itself is silly.

I think we should just do the latter (as we are currently).

I do wonder why you got the right link-relative poses in #32 (comment) but I didn't in #32 (comment).

tkoolen · 2019-07-13T04:00:37Z

Maybe it has to do with https://github.com/rdeits/meshcat/pull/51/files#diff-1fdf421c05c1140f6d71444ea2b27638R687?

tkoolen · 2019-07-15T17:10:10Z

#32 (comment) is fixed by meshcat-dev/meshcat#58.

rdeits mentioned this pull request May 19, 2019

Slightly hack-y workaround for .DAE loading meshcat-dev/meshcat#51

Closed

tkoolen added 3 commits June 17, 2019 00:03

Dependency version requirement updates.

cf937cb

Temporarily undo MeshCat version bound changes.

7038a97

Switch to using MeshCat.MeshFileGeometry and MechanismGeometries.Mesh…

71d85d0

…File.

tkoolen force-pushed the tk/meshfile branch from ba8599d to 71d85d0 Compare June 17, 2019 04:03

Bump MeshCat lower bound.

788a3f4

rdeits closed this Jul 12, 2019

rdeits reopened this Jul 12, 2019

Add missing Random test dependency.

fa1ef92

tkoolen merged commit c06238b into master Jul 12, 2019

tkoolen deleted the tk/meshfile branch July 12, 2019 12:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to using MeshCat.MeshFileGeometry and MechanismGeometries.MeshFile. #32

Switch to using MeshCat.MeshFileGeometry and MechanismGeometries.MeshFile. #32

tkoolen commented May 15, 2019 •

edited

Loading

tkoolen commented May 15, 2019

rdeits commented May 15, 2019

tkoolen commented May 15, 2019

tkoolen commented May 17, 2019

tkoolen commented May 17, 2019 •

edited

Loading

tkoolen commented May 17, 2019 •

edited

Loading

rdeits commented May 19, 2019

tkoolen commented May 19, 2019

rdeits commented May 19, 2019

rdeits commented May 19, 2019

rdeits commented Jul 12, 2019

codecov-io commented Jul 12, 2019 •

edited

Loading

tkoolen commented Jul 12, 2019

tkoolen commented Jul 12, 2019

tkoolen commented Jul 12, 2019

tkoolen commented Jul 12, 2019

tkoolen commented Jul 13, 2019

tkoolen commented Jul 13, 2019

tkoolen commented Jul 15, 2019

Switch to using MeshCat.MeshFileGeometry and MechanismGeometries.MeshFile. #32

Switch to using MeshCat.MeshFileGeometry and MechanismGeometries.MeshFile. #32

Conversation

tkoolen commented May 15, 2019 • edited Loading

tkoolen commented May 15, 2019

rdeits commented May 15, 2019

tkoolen commented May 15, 2019

tkoolen commented May 17, 2019

tkoolen commented May 17, 2019 • edited Loading

tkoolen commented May 17, 2019 • edited Loading

rdeits commented May 19, 2019

tkoolen commented May 19, 2019

rdeits commented May 19, 2019

rdeits commented May 19, 2019

rdeits commented Jul 12, 2019

codecov-io commented Jul 12, 2019 • edited Loading

Codecov Report

tkoolen commented Jul 12, 2019

tkoolen commented Jul 12, 2019

tkoolen commented Jul 12, 2019

tkoolen commented Jul 12, 2019

tkoolen commented Jul 13, 2019

tkoolen commented Jul 13, 2019

tkoolen commented Jul 15, 2019

tkoolen commented May 15, 2019 •

edited

Loading

tkoolen commented May 17, 2019 •

edited

Loading

tkoolen commented May 17, 2019 •

edited

Loading

codecov-io commented Jul 12, 2019 •

edited

Loading