Datasource vcd_catalog not working for shared catalog with tenant users

[NilsBusche]

Terraform Version

Terraform v1.3.5
on linux_amd64
+ provider registry.terraform.io/vmware/vcd v3.8.0

Affected Resource(s)

Please list the resources as a list, for example:

• Datasource vcd_catalog

Terraform Configuration Files

data "vcd_catalog" "catalog" {
  org = "org-1234"
  name = "catalog-abcd"
}

output "catalog_id" {
  value = data.vcd_catalog.catalog.id
}

Expected Behavior

Catalog ID should be returned as output, even if a tenant specific user is used.

Actual Behavior

Catalog cannot be found:

data.vcd_catalog.catalog: Reading...
╷
│ Error: unable to retrieve catalog record - [ENF] entity not found
│
│   with data.vcd_catalog.catalog,
│   on main.tf line 11, in data "vcd_catalog" "catalog":
│   11: data "vcd_catalog" "catalog" {
│
╵

Steps to Reproduce

1. terraform apply

User Access rights

Used user is a user which has only access to one specific tenant, not a system/provider user

Important Factoids

• The used catalog is a catalog shared by one organization to others organizations
• The given org value is the name of the org where the original catalog is located.
• Using a user with system/provider access rights the code above is working as expected
• Changing the org value to an org the catalog is shared to does not work, even with system/provider access, but this might be the expected behavior
• The datasource is very important for us to use the new attribute vapp_template_id in the vapp_vm

[dataclouder]

Hello,
Thanks for submitting this issue. I think I know the reason for the failure, but I will need to run a few tests to make sure. We will fix this, as it is important for most users.

[dataclouder]

The reason for the failure is this:

• when we search for the catalog, I try to retrieve the org. As a tenant, I don't have access to the sharing org, and the search fails (org not found)
• If I put the current org, the search again fails because the catalog parent is not the current org. (catalog not found)

A possible solution, which, BTW, would also solve the case where we have the catalog shared with the same name from multiple organizations, is to add a field sharing_org to use instead of org in the data source.

data "vcd_catalog" "catalog" {
  sharing_org = "org-1234"
  name        = "catalog-abcd"
}

An alternative would be to use the catalog ID as disambiguation. In both cases, the information should come from the catalog owner, as the web interface does not provide it.

Would this solution be acceptable?

[dataclouder]

The way I see it working is this:
When you make a data source with org it will fail like it does now, but the error message will contain information about the catalog belonging to another org, and that you should use sharing_org instead.
I think the message could also be able to list which orgs are sharing a catalog with the same name, so that you could pick the one that is needed.

[NilsBusche]

Thanks for your ideas. This should already be better than the current functionality. We thought about following further improvement:

First idea:
Maybe the data source should work even with shared catalogs without explictly setting a sharing_org as long as there is only one catalog with the given name. So in most cases this would be sufficient for tenant users as they don't have to worry about the sharing_org. If there are mutiple catalogs with same name available, the data source could fail with a behaviour like you mentioned in your last comment.
We thought about this because as you described the tenant user cannot see the sharing ORG in the WebUI.

Second idea:
What about adding an additional data source (or replace the current data source) which returns multiple catalog ids matching the search criteria like vcd_catalogs? This should work if there is only one matching catalog as well if there are multiple catalogs with the same name from different sharing ORGs. In this case the user has to decide how to handle this. Maybe it would be good then to add additional search criteria on top of the catalog name, so you can refine your search with this. We thought about parameters like if the catalog is shared, metadata or additional attributes. I did not have the time to look in detail into the API responses which attributes are available for catalogs as a tenant user.

[dataclouder]

Thanks. I will try to make it work in the simpler way, and then make the more complex ways possible. I will share a prototype soon.

[dataclouder]

I managed to find a solution with a simpler interface.
Users should be able to get the data source by using the origin organization in the org field.
If they don't know the right organization name, they can attempt with their own org, and receive an informational error message with the names of the orgs that contain a catalog with that name.

See PR #949 for details.
It would be helpful if someone could build the provider from that branch and try it out

[NilsBusche]

Thanks for the solution. Unfortunately I have not the time at the moment to test it out. But looking at the PR and the ideas considered in it, I think, it should solve the problems we had.