Improve project-to-plane functionality

[rbeyer]

Description
There is currently a planar map projection, allow the user to define an arbitrary plane instead of one centered at the body's center of mass and aligned in an equatorial manner.

The planar map projection was created to facilitate map projection onto a ring plane. However, for small irregular bodies, or even exploring atmospheric layers visible on a planetary limb, the ability to specify an arbitrary plane (in some cases perpendicular to the look vector) for map projection has value.

[jlaura]

Improve project-to-plane functionality

[krlberry]

Hi @rbeyer,

Is what you're looking for here the ability to project the pixels of a target to an arbitrary plane located anywhere in 3D space? (Where this plane could be located literally anywhere, including inside, outside, or tangent to the body, and have any orientation?)

If so, how would you want to specify the plane to project on to?

Also, what kind of projection would you like to do onto this arbitrary plane? Point-perspective, or something else?

If the questions I've asked above seem like they're leading down the wrong path, how do you see this working?

[jessemapel]

I would advocate for the mathematical definition, point on the plane and a normal vector (not necessarily a unit vector).

[rbeyer]

@kberryUSGS,

Is what you're looking for here the ability to project the pixels of a target to an arbitrary plane located anywhere in 3D space? (Where this plane could be located literally anywhere, including inside, outside, or tangent to the body, and have any orientation?)

Yes.

If so, how would you want to specify the plane to project on to?

Great question. Not sure. :)

This might be the hardest part, how to specify this. Ideally, you'd want to be able to express this in a completely arbitrary way, but also providing some reasonable shortcuts. The use case that this came up in was for a plane centered on the target center, perpendicular to the view vector, so this should be possible without the user having to do a bunch of vector math, because there is theoretically always a "solution" for this situation that you can derive from the position and pose data you have. Just doing that would satisfy the use case I was thinking of.

But then, I was thinking, if you're going to enable this, then you might as well provide the ability to designate an arbitrary plane. How to do that? Not sure. Maybe require a SPICE state vector or something for the purely arbitrary? So for someone to give you a completely arbitrary plane, they're gonna have to use cspice or Spiceypy to get something that can be fed to this program.

Also, what kind of projection would you like to do onto this arbitrary plane? Point-perspective, or something else?

Yes, I think this should just be point-perspective: take each ray that corresponds to a pixel, trace it out of the camera and intersect it with the arbitrary plane.

[krlberry]

Thanks for the information! We're not going to be able to actually tackle the work for this in the current sprint, but now we have enough information to scope the work and make a plan for moving forward when we pick this up again.

[rbeyer]

Sounds good.

[jlaura]

Requirements:

• The original goal is to be able to do atmospheric work.
• This is a general purpose tool.
• Specification of the plane is discussed above.
• A user's life is easier if they have convenience capability to define the plane using SPICE. This is really hard. So the MVPs are building towards being able to do that.

MVP:

• Make a plane perpendicular to the look vector of the observation.
• Project the image to that plane.

MVP (n+1):

• Allow the user to specify an arbitrary plane. As a step 1, how can the user do this?
• Need to understand how to establish the plane in the mapping group.

Example Image Data:

• @rbeyer is going to post some paths to examples if/when his computer is back alive. If not, we will grab some Monday AM of kickoff. LORRI images just before the closest approach of Pluto where you can see the disk.

Initial steps:

• Need to look at the projection code to understand how much if it might need to be modified.

[scsides]

NOTE: We are working on this issue. If anyone has input please post here.

Preliminary questions:

• Our understanding is that this projection would project data beyond the limb; that is, not stop projecting data once the pixel did not have a lat/lon, (e.g., atmosphere). Is this what is being asked for?

• Would Point Perspective work if it were to continued projecting data beyond the limb?

• Atmosphere data beyond the limb doesn't have corresponding lat/lons in ISIS. Once projected, what data is needed? (Example from planetary body data: qview -> show phase angle of a specific pixel)

• Which ISIS programs will be used on the projected data?

• What are some use cases for the projected data?

• If we were to only allow a plane perpendicular to the center instrument look vector, each image from a New Horizons instrument would have a slightly different plane associated with it, and therefore could not be mosaiced. Would that still be useful?

[rbeyer]

• Our understanding is that this projection would project data beyond the limb; that is, not stop projecting data once the pixel did not have a lat/lon, (e.g., atmosphere). Is this what is being asked for?

Yes, that is the primary motivator, but there is also the possibility that this could be useful for an object before a detailed shape model is available.

• Would Point Perspective work if it were to continued projecting data beyond the limb?

I would think so, but if you have concerns about this, let's talk.

• Atmosphere data beyond the limb doesn't have corresponding lat/lons in ISIS. Once projected, what data is needed? (Example from planetary body data: qview -> show phase angle of a specific pixel)

Not really sure that any meta-data of this nature off the limb is possible. The original intent was to be able to measure the elevation of atmospheric layers from the limb, so really just about projecting the pixels to the right place.

• Which ISIS programs will be used on the projected data?

I suspect not many, maybe qview and then possibly an isis2something. It is possible that pixel-based processing might be used, but I don't think you'd work with the resulting image and consider it any kind of true map projection.

• What are some use cases for the projected data?

1. Measuring the elevation of atmospheric hazes from the limb.
2. Making projections in which the "pixels are square" to facilitate distance measurements over raw camera images during an encounter before complete SPICE information is available.

• If we were to only allow a plane perpendicular to the center instrument look vector, each image from a New Horizons instrument would have a slightly different plane associated with it, and therefore could not be mosaiced. Would that still be useful?

Yes.

[krlberry]

We had some additional questions after reading your responses:

• What is your definition of square pixels? Is it:

  • All edges of a pixel are all approximately the same length
  • All pixels in the image are the same size
  • Something else?

• What is the connection between needing square pixels and to measure the distance between the limb of the planet and the atmosphere and the ability to project to a plane?

• Just out of curiosity, what tool will you use to measure the distance?

• One concern with Point Perspective projection is that it will result in pixels that are approximately square but of varying resolutions as you move away from the point of true scale, and as the limb is the furthest from the point of true scale, the distortion will be the worst at the limb. Will this work with your requirement to measure the atmosphere’s height from the limb?

• The current Point Perspective projection in ISIS projects to a plane tangent to the center latitude, center longitude on the body's surface. It would be possible to project onto a plane perpendicular to the look direction by selecting an appropriate center latitude, center longitude. Does this work for your use-case?

[rbeyer]

• What is your definition of square pixels? Is it:

  • All edges of a pixel are all approximately the same length

  • All pixels in the image are the same size

  • Something else?

I go into more detail in answers below, but I think the answers are "yes" to both.

• What is the connection between needing square pixels and to measure the distance between the limb of the planet and the atmosphere and the ability to project to a plane?

If there is a cloud layer floating at some altitude, that will be expressed as a brighter line (curve) of pixels that is parallel to the limb, and a scientist wants to be able to measure its "elevation" from the surface. In order to do that, they need the pixels in the projected image to enable reliable measurement.

• Just out of curiosity, what tool will you use to measure the distance?

Given that the "usual" pixels-on-the-ground measurement capabilities of qview (or any other GIS application) wouldn't know what to do with this projected image, my thought was that just using qview to measure the number of pixels from the limb to the cloud layer would do it, and then if the "size" of the pixels was dependable, then you could just multiply. This is effectively what is done now with the raw camera images, but as you can imagine, a scientist needs to do a lot of geometry and accounting to get a good value, and the hope is that this project-to-plane would make that easier.

• One concern with Point Perspective projection is that it will result in pixels that are approximately square but of varying resolutions as you move away from the point of true scale, and as the limb is the furthest from the point of true scale, the distortion will be the worst at the limb. Will this work with your requirement to measure the atmosphere’s height from the limb?

That's a good point, maybe we don't want a point projection, but more of an ortho projection onto the plane to enable measurement?

• The current Point Perspective projection in ISIS projects to a plane tangent to the center latitude, center longitude on the body's surface. It would be possible to project onto a plane perpendicular to the look direction by selecting an appropriate center latitude, center longitude. Does this work for your use-case?

Aside from the point/ortho projection issues you brought up above, even if point projection were still a good idea, this wouldn't work, as we want to measure distances off the limb. At the very least the "plane" in 3D space needs to include the center of the object, so that the distances measured at the limb are perpendicular to the surface there. Attaching this plane to a point on the surface closest to the observer, would not enable good distance measurements off the limb.

[jlaura]

@scsides @kberryUSGS @rbeyer I think we still need to get together to chat about this in real time. Sounds like we do not have a particularly good way to accomplish the desired behavior?

[jlaura]

@kberryUSGS @scsides Can we close?

[scsides]

The changes have been merged, and there are no outstanding comments. The author can reopen after a release comes out with the new feature.