✅ PLEASE VOTE: Creating Required Module Resource Groups

[rjygraham]

Today all of the modules that deploy resources into resource groups expect the resource groups exist or are created outside of the module.

We are investigating the inclusion of resource group creation within the modules themselves; however, there are technical constraints in how the solution will work and we'd like to hear from the community to understand if including resource group creation within the modules is worth the effort.

** WE WANT YOU TO HELP US DECIDE THE FUTURE BY VOTING (SEE BELOW) *

Please read the remainder of this issue for context and then have your voice heard by voting at the following link:
VOTE HERE

We'd ❤️ to hear from you, so if you have any other thoughts/ideas/questions, please leave a comment.

Technical Constraints

• Bicep templates deploying resource groups target the subscription scope
• Bicep templates deploying resources into resource groups target the resourceGroup scope
• Since deploying the required resource groups and then resources target different deployment scopes, it is not possible to combine everything into a single Bicep template.

Proposed "Module Inclusion" Solution

The proposed solution to deal with the above technical constraints are as follows:

• Each module requiring a resource group creation would have a "wrapper" Bicep template added.
• The "wrapper" Bicep template would be the entry point for modules requiring resource groups and you would change your deployment to target the subscription scope instead of resourceGroup scope
• The "wrapper template will invoke two module deployments:
  1. Resource group at subscription scope
  2. Resource deployment targeting the just deployed resource Group as the deployment scope
• With this "wrapper" approach, the existing Bicep files deploying module resources will remain intact which means you could still deploy the modules resources only. However, file locations may move to keep things organized.

A preview of what this change for looks like for the hubNetworking module can be viewed here:
https://github.com/rjygraham/ALZ-Bicep/pull/1/files

In this example, the "wrapper" template is infra-as-code/bicep/modules/hubNetworking/resourceGroup/hubNetworkingResourceGroup.bicep and the resource template is moved from the module root to the resources folder.

Pros of "Module Inclusion" approach

• Aligns closely with the vision of ALZ-Bicep having modules that are independent deployable units
• (Hypothesis) This approach is more user friendly to those less familiar with Azure
• Provides flexibility for ALZ-Bicep consumer to choose whether to deploy a module resources with or without the resource group as part of the template
• Simplified CI/CD pipelines:
  • Removes any steps to pre-create the required resource group
  • If you were using AZ CLI or Azure PowerShell steps to deploy ALZ-Bicep based on the deployment snippets in the module README, you could simplify and use the native Azure ARM tasks/steps within Azure DevOps/GitHub Actions.

Cons of "Module Inclusion" approach

• Bicep templates don't have the notion of inheritance and therefore any parameters or outputs in the Bicep template deploying resources will need to be duplicated in the "wrapper" template
  • This adds maintenance overhead as ALZ-Bicep and individual modules evolve
  • May add friction to contributions from community members as it would not be immediately apparent that parameters and outputs from the resource Bicep template need to be copied to the "wrapper" template
• Documenting the two deployment modalities is challenging:
  • Consolidating documentation for deploying the module resources or module resource group and resources in a single README is a bit messy
  • Splitting documentation into two separate documents would require duplicating large markdown tables detailing the parameters and outputs - these would need to be kept sync
• Have to consider how we maintain organization and consistency across modules targeting the subscription scope vs modules targeting the resourceGroup scope

An "Orchestrator" alternative

An alternative to including module resource group creation in the modules themselves would be to include resource group creation in an "orchestrator" type template. The "orchestrator" template would deploy some (or all) modules sequenced in the correct order and ensure required resource groups are created before deploying modules.

Pros of the "orchestrator" approach

• Most pros of the "Module Inclusion" approach
• Module documentation remains straight forward
• No longer issues with maintaining organization and consistency across modules targeting the subscription scope vs modules targeting the resourceGroup scope

Cons of the "orchestrator" approach

• Still have the issue of copying parameters and outputs from the module Bicep template to the "orchestrator" template; however, this will need to be done if we decide to implement an "orchestrator" template anyway.
• Deviates from the vision of ALZ-Bicep having modules that are independent deployable units

[cloudchristoph]

@rjygraham We cannot vote right now - the Form is locked down - Message "You don't have permission to view or respond to this form"

[cloudchristoph]

A few thoughts about the options:

Module Inclusion Approach

This is how we've implemented this today, tbh.

It has the benefit of separating all modules within the Pipeline / Deployment Script - each deployment call has all the parameters needed.

But we've created "wrapper" modules for almost all modules. Mostly to deploy the resource group if necessary (thus implementing this idea).

To name a few problems:

1. Naming:

   We are using a PS script and a Bicep Module for naming - not ideal, because you have to maintain it at two different locations.

   (no worries - different problem - see: Reference resource group that is created via a module bicep#4992)

2. Inheritance:

   Yes, parameters you get on the "wrapper" template have to be looped to the child template. This is annoying.

   Answer: Nothing the consumer of ALZ-Bicep should have to worry about.

   Goal should be: Call hubNetworking.bicep or call hubNetworkingWithRg.bicep - provide the same parameters, get the same results.

3. Dependencies:

   To be honest, some of our modules depend on calling other modules. At least for the outputs. So it makes little difference whether we call a "belonging" or an "orchestrating" module.

   E.g.: By calling our policy-assigment module, you have to provide the topManagementGroupName, which you'll get by deploying management-groups and s.o.

4. Separation for deployments.

   In Theroy: Each Deployment Step within the Pipeline just have to provide parameters for this particular resources, so you can remove "steps" from this deployment.

   In Practice: Some of the deployment steps are using output from other deployments. No real separation possible here.

Orchestrator Approach

1. You cannot skip modules

   Because some of the modules have dependencies on other modules, your deployments are quite static.

2. Play the "input"/"output" game within the Orchestrator module rather than within the Pipeline or Script

   1. Deploy "Management Groups"
   2. Deploy "Logging and Security"
   3. Use Output of "Management Groups" and "Logging and Security" for "Policy Assignments"

   This doesn't allow indepentend deployments, but makes the whole process simpler.

   But for us: no difference to the other approach.

Vote

Using the Module Inclusion Approach allows greater flexibility for deployments. But...

In the end I think we should provide a combination:

• Use something like "master-deployment.bicep" to deploy all modules with Management Groups, Subscriptions, Resource Groups, Resources
• Use module-specific resourcegroup-aware deployment files to get your resources within a Resource Group
• Use module-specific deployment file to get your resources to the provided and already deployed resource group (as today)

[rjygraham]

@cloudchristoph thanks for the input. The form should be open for voting now if you can give it a try and let me know.

If I'm reading your comment correctly, you're suggesting we should include resource group creation within the module as well as "orchestrator" template should we go down that path?

Having lived through the pain of spiking the solution in the hubModule it'll be interesting to see if the community feels like the effort is worth it. 🤔

[jtracey93]

As agreed in ALZ bicep team call. Closing this out as no activity and gone stale.

If someone feels strongly about this please reopen or create a new issue and we will re-triage.

Thanks

Jack