Creating image from some Azure Marketplace Images requires additional items in the ARM template.

[russellseymour]

When creating a new image from some Azure Marketplace Images the following JSON snippet is required in the root of the 'virtualMachine' resource:

"plan": {
    "name": "[parameters('imageSKU')]",
    "publisher": "[variables('imagePublisher')]",
    "product": "[variables('imageOffer')]"
}

Although this seems a duplication of the information being passed to Azure it is required otherwise the following error will be seen:

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'.",
    "details": [
      {
        "code": "VMMarketplaceInvalidInput",
        "message": "Creating a virtual machine from Marketplace image requires Plan information in the request. OS disk name is osdisk."
      }
    ]
  }
}

Would it be possible to add a new parameter in the builder configuration that states this plan information needs to be added to the template.

This is the case with creating a custom image of a Chef Server for example however Windows machines work without this information.

[cbednarski]

Would it be possible to add a new parameter in the builder configuration that states this plan information needs to be added to the template.

Thanks for the report. I'm a bit confused -- is this a bug / UX problem or something we need to fix in the documentation?

[russellseymour]

I think it is more of a feature request rather than a bug (although it can be justified both ways I think). It is not something that can be fixed in the documentation as Packer does not support this yet.

We need to be able to add the plan information from a the builder configuration when using AzureRM for some marketplace images.

[russellseymour]

I work for Chef and I am trying to build up a custom image in an Azure subscription using Packer to do it. I am using our AMP for the Chef server which needs this plan information.

[boumenot]

I do not think this is a builder/azure issue, but rather a Chef issue? I do not understand this well enough to offer any help.

Here's information that would help me.

1. More details about how you got into this state. It isn't clear what builder you used.
2. What's the JSON snippet you used to get this failure.

I can offer general help on the ARM, but I am not sure where to start.

[cbednarski]

@russellseymour Thanks for following up. I think I understand now.

@boumenot It sounds to me like we should provide a way to add arbitrary values to the JSON payload that packer sends to the Azure API when creating a machine. This allows us to support extra options that packer doesn't have logic for right now.

We support a similar feature for VMX files which allows users to specify their own key=value pairs in the config file that packer generates, even if there is not a documented packer option for it. Also, many of the hard-coded commands in e.g. provisioners can be overridden in case the user wants to specify options that packer doesn't support.

This provides the benefit of extra flexibility for advanced users but means we don't have to have a first-class workflow and extra logic for each use-case. If this is a common need we can make it a first-class feature.

[boumenot]

Makes sense. Thanks for the pointing me at the VMX work. This will be a good way to learn more.

My plan is rework the Azure builder's use of JSON documents, so we do not have hard coded strings. A member of the community stepped up to do some this work, so I am waiting to see what he comes up with, and will iterate from there.

[russellseymour]

I have got some more information from Microsoft about what this plan information is. It is for images in the Marketplace that are not first party which explains why it is not needed for vanilla Windows or Linux machines that are built from the Marketplace.

As the Chef image is a modified Ubuntu it requires the plan information. This tallies with another custom AMP image I was working with which was the Bitnami Jenkins offering.

A complete Virtual Machine resource with the plan information (just after the dependsOn) in it looks like this:

{
      "apiVersion": "[variables('apiVersion')]",
      "type": "Microsoft.Compute/virtualMachines",
      "name": "[parameters('vmName')]",
      "location": "[variables('location')]",
      "dependsOn": [
        "Microsoft.Resources/deployments/StorageAccountSetup",
        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
      ],
      "plan": {
        "name": "[parameters('chefServerSku')]",
        "publisher": "[variables('imagePublisher')]",
        "product": "[variables('imageOffer')]"
      },
      "properties": {
        "hardwareProfile": {
          "vmSize": "[variables('vmSize')]"
        },
        "osProfile": {
          "computerName": "[parameters('vmName')]",
          "adminUsername": "[parameters('adminUsername')]",
          "adminPassword": "[parameters('adminPassword')]",
          "customData": "[base64(concat(variables('cloudHead'), 'fqdn: ',reference(variables('publicIPAddressName')).dnsSettings.fqdn,'\n' ))]"
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "[variables('imagePublisher')]",
            "offer": "[variables('imageOffer')]",
            "sku": "[parameters('chefServerSku')]",
            "version": "latest"
          },
          "osDisk": {
            "name": "osdisk",
            "vhd": {
              "uri": "[concat('http://',variables('storageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/',variables('OSDiskName'),'.vhd')]"
            },
            "caching": "ReadWrite",
            "createOption": "FromImage"
          },
          "dataDisks": [
            {
              "name": "datadisk1",
              "diskSizeGB": "100",
              "lun": 0,
              "vhd": {
                "uri": "[concat('http://',variables('storageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/',variables('dataDisk1VhdName'),'.vhd')]"
              },
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
            }
          ]
        },
        "diagnosticsProfile": {
          "bootDiagnostics": {
            "enabled": "true",
            "storageUri": "[concat('http://',variables('storageAccountName'),'.blob.core.windows.net')]"
          }
        }
      }
    }

I was not able to find any documentation on this. The way I found the correct syntax and layout was to use the https://resources.azure.com page and select a machine that I had created through the portal - this gave me the JSON definition of the machine.

[cbednarski]

@russellseymour Thanks again for the detailed followup.

@boumenot It sounds like this is a fairly common use case for vendors and MS partners so we might want to make the UX nice here; I'll defer to you on the best way to proceed since you're more familiar with this part of the code, but I can hop in and help if you need.

[boumenot]

Update: We have added this to our TODO. Our plan is to allow users to specify a list of JSON patches that can be applied to the ARM template before it is submitted for processing. My plan is to use this library for the patching.

The RFCs have good information: RFC6902 JSON patch and RFC7396 JSON Merge Patch.

[boumenot]

@russellseymour - I created a PR to address your issue. Are you able to build this PR, and validate if it corrects your issue? As I dug into this more I realized that it was much more specialized than I originally thought. I would appreciate your help if knowing if the fix actually works for you.

[boumenot]

I have not seen a response, so I am closing this for now. Please re-open as appropriate.

I'll leave the PR around, but I would like confirmation before proceeding with it.

[russellseymour]

@boumenot Sorry been busy with things. Will have a look and let you know. Thanks.

[boumenot]

No pressure.

[monix16]

I am having the same issue. I want to use the plan information in my builder to build the image. I tried building this PR but failed to do so.

Cherry picking the commit id says "fatal : bad object "

Please guide me how can I test this.

[boumenot]

The PR has become stale. I'll need to update it.

The other alternative is you build directly from my clone of the Packer repo where the PR was created. This should give you a working build.

[njappboy]

Has anyone from the packer team going to revisit this issue? I'm facing this issue with an nginx plus marketplace image.

[boumenot]

I was not willing to commit the patch without confirmation that it would work. I can restart the work if you're willing to validate it.

[rtruxal]

I'm not near my C2 client and can't test this. Anyone care to validate it?:
https://github.com/rtruxal/ansible/blob/devel/lib/ansible/modules/cloud/azure/azure_rm_virtualmachine.py

[rbramwell]

Came across this issue after trying to build from an Azure marketplace image as well. I think this issue should be reopened. It's still a missing feature IMHO. @boumenot I'm willing to test/validate if needed.

[sstent]

Saw the same issue trying to use a CIS marketplace image.

[rbramwell]

@sstent Same in my case.

[mwhooker]

I'll reopen this, but we need someone to step up and test the patch. Anyone willing to do that? @sstent @rbramwell @njappboy

If you have problems building off of @boumenot's repo, I can help out, but please let me know.

[rbramwell]

@mwhooker I'm willing to test the patch. Please point me in the right direction. I checked to URL given in #3506 (comment) but it returns a 404

[rbramwell]

Found the PR #3665 Took a look at the code. I'll build the branch and give it a go.

[boumenot]

Thanks for stepping up @rbramwell! If you get stuck let me know. I should be able to give this attention next week.

[TechnicalMercenary]

I ran into this today. Any chance that I can help out and get this resolved ? Is there a work around at the moment ?

[sylus]

Running into this as well, which unfortunately makes Packer unusable for me when working with Azure.

[TechnicalMercenary]

Any information on how this needs to be tested so the issue can be resolved ? @sstent @rbramwell @njappboy @boumenot

I'm happy to help, as this is blocking me. Just send me the directions

[boumenot]

/cc @harijayms

[boumenot]

Take II. I posted a new PR.

I did a private build with this change for Windows, Linux, and MacOS (all are amd64). If you are able to try it, please do so and report back. I do not have the ability to test it end-to-end unfortunately.

[gbonk]

bash-3.2$ packer --version
1.2.2

Unfortunately, It failed for me. What else do you need from me? Logs?

==> azure-arm: ERROR: -> DeploymentFailed : At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.
==> azure-arm: ERROR:   -> Conflict
==> azure-arm: ERROR:   -> ResourceDeploymentFailure : The resource operation completed with terminal provisioning state 'Failed'.
==> azure-arm: ERROR:     -> VMMarketplaceInvalidInput : Creating a virtual machine from Marketplace image requires Plan information in the request. OS disk name is 'osdisk'.
==> azure-arm:
==> azure-arm: resources.DeploymentsClient#CreateOrUpdate: Failure sending request: StatusCode=200 -- Original Error: Long running operation terminated with status 'Failed': Code="DeploymentFailed" Message="At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details."
==> azure-arm:
==> azure-arm: Cleanup requested, deleting resource group ...

[boumenot]

I can't tell if you are using an official release, or the version with the fix that I posted in my previous updated. If it was the version I posted, I would expect a version with a Git hash. Please retry with the version I posted if you used the official version.

[gbonk]

no, definitely used the version I downloaded from OneDrive. Went as far as deleting packer from my path ( brew )

I'll double check though. Edit, done. ✔️✔️

https://gist.github.com/gbonk/135c5b04853f98b2d0ab9f4048ca999e

[boumenot]

Perfect, you included a gist. The version in the gist is much more precise, and what I was expecting to see.

2018/03/05 16:26:50 packer: 2018/03/05 16:26:50 PlanName=
2018/03/05 16:26:50 packer: 2018/03/05 16:26:50 PlanProduct=
2018/03/05 16:26:50 packer: 2018/03/05 16:26:50 PlanPublisher=

From the logs, it looks like you forgot to set plan_name, plan_product, and plan_publisher in your .json file. Of course, I could be wrong about that too. If you can, please include your .json file.

[gbonk]

You are correct. I do not set plan_name, plan_product, and plan_publisher in my .json.

If possible, please provide ( and update the) documentation on what to set those fields too. I wasn't before and this was all working fine without setting those fields.

Edit: Tomorrow at work I'll give it a try following the example above from the others.

[boumenot]

I documented them in my PR. Once the change is accepted, the PR will update the live documentation on packer.io. Until then, you will need to use the PR's edits.

[gbonk]

Fantastic... The link in your documentation was very helpful, and helped to clarify to me why this plan stuff was necessary when my previous work didn't require plan settings.

As per the link, I got the plan details...

az vm image show --location eastus2 --publisher cloudera --offer cloudera-centos-os --sku 6_8 --version 2.0.

{
  ...
  "plan": {
    "additionalProperties": {},
    "name": "6_8",
    "product": "cloudera-centos-os",
    "publisher": "cloudera"
  },
  "tags": null
}

Then passing via environment variables into my builder.json

"plan_name": "{{user `plan-name`}}",
"plan_product": "{{user `plan-product`}}",
"plan_publisher": "{{user `plan-publisher`}}",

I have achieved sweet success. 👍

And my $0.02 suggestion the documentation

• plan_name (string) The plan name. This setting, including (plan_product, plan_publisher, and plan_promotion_code) are only needed for Marketplace images. Please refer to Deploy an image with Marketplace terms for more details.
• plan_product (string) The plan product. See plan_name for more information.
• plan_publisher (string) The organization that created the image in the Marketplace. This value is the same value as Offer (image_offer). See plan_name for more information.
• plan_promotion_code (string) Some Marketplace images use a promotion code. See plan_name for more information.

[kitho-pwc]

Saw the same issue trying to use a CIS marketplace image.

@sstent @rbramwell How can we capture an image from VM created by CIS Images in Azure Marketplace?

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.