Resources in nested template not shown if reference is used in the parameter values

[J0F3]

Describe the bug
Probably related to #83 but not sure so I though it might worth it to post it here.

To Reproduce
An ARM Template with a nested deployment. The parameter value of the nested deployment uses the reference function.
When deploying the template with Whatif it shows only the vNet created in the parent template and nothing about the Subnet created in the nested template.
On a re-deployment it gets even more weird. Whatif shows that the subnet gets deleted which is actually not what appends. So very confusing.

When the parameter value is replaced by a literal string the Whatif output is shown the changes correctly. So assume it is also related to the use of the reference function.

ARM Template:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "functions": [],
  "resources": [
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2019-10-01",
      "name": "vnetmodule",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "location": {
            "value": "[resourceGroup().location]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
            "location": {
              "type": "string"
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/virtualNetworks",
              "apiVersion": "2020-05-01",
              "name": "vNet1",
              "location": "[parameters('location')]",
              "properties": {
                "addressSpace": {
                  "addressPrefixes": [
                    "10.0.0.0/16"
                  ]
                }
              }
            }
          ],
          "outputs": {
            "resourceName": {
              "type": "string",
              "value": "vNet1"
            },
            "resourceId": {
              "type": "string",
              "value": "[resourceId('Microsoft.Network/virtualNetworks', 'vNet1')]"
            }
          }
        }
      }
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2019-10-01",
      "name": "subnetModule",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "vNetName": {
            "value": "[reference(extensionResourceId(resourceGroup().id, 'Microsoft.Resources/deployments', 'vnetmodule'), '2019-10-01').outputs.resourceName.value]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
            "vNetName": {
              "type": "string"
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/virtualNetworks/subnets",
              "apiVersion": "2020-05-01",
              "name": "[format('{0}/subnet1', parameters('vNetName'))]",
              "properties": {
                "addressPrefix": "10.0.0.0/24"
              }
            }
          ]
        }
      },
      "dependsOn": [
        "[extensionResourceId(resourceGroup().id, 'Microsoft.Resources/deployments', 'vnetmodule')]"
      ]
    }
  ]
}

Expected behavior
I would except that the vnet and subnet is shown as 'create' on the first deployment and as 'no change' in subsequent deployment.

Screenshots
First deployment (with reference in parameter value). Subnet is missing in the output:

Re-deployment (with the same template). Subnet is shown as 'delete':

First deployment when the value of the 'vNetName' parameter is stet to a literal string. Both the vnet and the subnet is shown correctly in the output:

Re-deployment (with the same template) show both resource correctly (beside except some noise probably not related to this issue):

Client
PowerShell

Additional context
I noticed quite some of such strange outputs of Whatif during playing with Biceps and Biceps modules as this results in templates with nested deployment where often also the reference function is used. So I thought it will be worth to place it here. But maybe your are already very aver of that? 😊

[alex-frankel]

Basically, what-if short circuits too aggressively when we see a reference() function, but we should still be able to calculate all of the resource IDs required for what-if without evaluating the reference() call. So this should be doable.

Thanks for reporting!

[JustinGrote]

@alex-frankel I ran into this one when using modules in bicep. Anything in a module (which translates to a nested deployment) doesn't show up in my whatif result.

[JFolberth]

One comment to expand on running with cli 2.2 and bicep the plan actually ignores the child resources rather than delete like the original issue described:

[alex-frankel]

We discussed this today and the root cause of the issue is essentially that as soon as there is one parameter that uses a runtime function in a nested template, we have no good way of ignoring it/throwing it away and evaluating the rest of the template. The reason it is difficult for us because the way we evaluate expressions in an ARM Template is...fragile :) We have some work ongoing to refactor this code to make this easier for us, but no good ETA I can provide.

@J0F3 - for your case in particular, you can probably refactor the code to avoid the reference() call entirely. You can set the vnet name to be a variable in the parent template, then pass that variable to the nested template as a parameter.

[dani3lheidemann]

Hi @alex-frankel,
I recently opened #3237 and can confirm that it's the same issue.

I also can confirm that when I'm leaving out any references, that validation shows up all resources I actually want to deploy. And of course, I'm able to set additional variables to "convert" my input parameters, but I also have to waive using ARM/Bicep module outputs as input parameters for other modules, because using outputs is also defined as a reference in ARM.

May you have an update when this is gonna be fixed?

Thank you very much!

[alex-frankel]

We are working on this now with the hopes of having it resolved by the end of the calendar year. This is a bit of a gnarly, low level issue so please be patient :)

[alex-frankel]

FYI - the fix for this has been checked in. Should be deployed sometime in December.

[samhughes91]

Very happy to hear this news. Thanks for the update Alex

[J0F3]

Awesome news! Thank you @alex-frankel!

[tjementum]

can everyone help confirm that I am not missing something here?

I guess we will have to wait and see. As described in a previous post, I found a workaround to actually have --what-if work.

BUT there is a big problem even if what-if shows a plan. Resources that are “calculated” will show up as being potentially changed on every single deployment. See the following screenshot of a deployment that has NO changes:

Because all my environment variables are “calculated,” they show up with the current value in orange and the formula in green. I know the green formular will result in the same value as the orange, so nothing will be changed. But it pollutes the output and makes it next to impossible to see what actual changes will be done. If I was using Terraform and a state file, Terraform would show what changes I did.

Have you solved this, so you able to “suppress” a formular that will result in the same output?

Also, the screenshot shows that, e.g., properties.logAnalyticsDestinationType: "AzureDiagnostics" will be deleted. BUT the fact is it will not. This is because ARM does things behind the scenes.

So, I fear that when you “fix” this issue, you have an equally big task to get ALL the different Azure ARM Providers updated to behave correctly. But since this issue has been hiding this problem, the different teams do not even know that there is a problem.

[cathei]

Can't believe this major issue is open for 3+ years and no warnings are built in, people had to search to get here. I did found working workaround for me after several attempt. Where myConnectionString is mod1's computed output and we want to pipe it to mod2:

module mod1 'mod1.bicep' = {
	name: 'mod1'
}

module mod2 'mod2.bicep' = {
	name: 'mod2'
	params: {
		myConnectionString: mod1.outputs.?myConnectionString ?? '(known after created)'
	}
}

Combining safe-dereference .? and coalesce ?? default value like above template worked for me. At least what-if doesn't hide all the resources to be created with mod2. After deploying the template, the value was set properly on actual resource.

Hope this helps.

[alex-frankel]

Have you solved this, so you able to “suppress” a formular that will result in the same output?

Also, the screenshot shows that, e.g., properties.logAnalyticsDestinationType: "AzureDiagnostics" will be deleted. BUT the fact is it will not. This is because ARM does things behind the scenes.

So, I fear that when you “fix” this issue, you have an equally big task to get ALL the different Azure ARM Providers updated to behave correctly. But since this issue has been hiding this problem, the different teams do not even know that there is a problem.

@tjementum this are examples of the noise problem I was describing and resolving this issue (#157) is not going to solve that problem. We are currently investigating options for making meaningful progress on the noise issue that does not require RP/service team involvement.

cc @azcloudfarmer / @dantedallag / @snarkywolverine

[Ramu3l]

Hi @alex-frankel ,
since we are so close to October 1, are you confident we can release this feature on time?
Thank you for your support!

[daviddavis]

Hi @alex-frankel. It's October. Was just wondering if you had any updates to share. Thanks.

[azcloudfarmer]

Hello Everyone - we are excited to share the deployment of this fix is in progress and we will begin onboarding tenants gradually as soon as the rollout is complete. For those of you interested on getting your tenant onboarded to be able to receive this fix sooner, please fill this survey. We thank you all for the patience! 📎

[jikuja]

Hello Everyone - we are excited to share the deployment of this fix is in progress and we will begin onboarding tenants gradually as soon as the rollout is complete. For those of you interested on getting your tenant onboarded to be able to receive this fix sooner, please fill this survey. We thank you all for the patience! 📎

Already full?

You don't have permission to view or respond to this form

Technical details
Session id: cca0d6c4-cb08-4503-93ac-e1347b677bf1, Correlation id: 19aad79e-875d-4904-9dc4-b6abd26c53fa

[azcloudfarmer]

Hello Everyone - we are excited to share the deployment of this fix is in progress and we will begin onboarding tenants gradually as soon as the rollout is complete. For those of you interested on getting your tenant onboarded to be able to receive this fix sooner, please fill this survey. We thank you all for the patience! 📎

Already full?

You don't have permission to view or respond to this form

Technical details
Session id: cca0d6c4-cb08-4503-93ac-e1347b677bf1, Correlation id: 19aad79e-875d-4904-9dc4-b6abd26c53fa

@jikuja - thank you for catching this. The form is now open to all.

[MarkFarkas]

Hello @azcloudfarmer, can you please provide an update on the status of the rollout? When can we expect all tenants to be onboarded?

[alex-frankel]

We are blocked on rollout at the moment due to a planned end-of-year deployment freeze for specific high-profile events. Most likely we will only be able to onboard the new tenants in specific regions during November, then all regions in early December.

Apologies, as we were not as plugged into the deployment freeze timeline as we should have been. We very much want to get this out ASAP :(

[alexrecuenco]

Is this rolled out already? How can I verify if my tenant already has this upgrade rolled out?

I filled the survey to try to get it sooner, but I am not sure that ever happened, I am still getting mostly resource ignored and "going blind" :')

[alex-frankel]

Sadly, the Azure-wide deployment freeze got extended through all of January. More than likely we won't be able to roll this out until February. Sorry about the delay, but this one is totally out of our control. Thanks everyone for the collective patience.

[yangchengs]

Hello, can anybody help check if the issue #388 is related with this one?

[fbinotto]

Hi @alex-frankel , what is the ETA for the roll out?

[jamiecon]

Would be fantastic if we could get an ETA for the complete rollout. Thanks in advance.

[alex-frankel]

Hey folks - anyone who signed up via the survey link that @azcloudfarmer shared will be onboarded this week. We will send out an email as soon as your tenant has been onboarded.

If you haven't signed up, but would like to opt-in, please fill out this survey.

Assuming this initial rollout goes well, we will then share a timeline for rolling this out as the default what-if behavior.

[jamiecon]

Hi @alex-frankel we completed the survey form mid-last-week but have not had an onboarding notification and appear to still be affected by the same bug. Could you share an update when you have a moment? Would be appreciated. Thanks.