Pulumi hangs when creating route 53 resolver endpoint with more than 6 ip/subnets

[ckoning]

Describe what happened

When you create an inbound or outbound route 53 resolver endpoint with more than 6 ip/subnets - pulumi will hang, sometimes for up to an hour, and not provide any error message. Sometimes pulumi will even report the resource creation as successful even when it hasn't created anything.

AWS has service quotas for IP addresses for these resolver endpoints. See docs. These limits are:

Target IP addresses per resolver rule: 6 per region (hard limit, not adjustable)
IP addresses per resolver endpoint: 6 per region (soft limit, adjustable)

The Terraform SDK does not have any validation implemented for target IP addresses:
https://github.com/hashicorp/terraform-provider-aws/blob/982985456da51701cc7dc748c25fd78a82d3dec5/internal/service/route53resolver/rule.go#L88

The Terraform SDK implements a hard limit of 10 IP addresses per endpoint:
https://github.com/hashicorp/terraform-provider-aws/blob/982985456da51701cc7dc748c25fd78a82d3dec5/internal/service/route53resolver/endpoint.go#L66

The Terraform SDK is exhibiting similar behavior:
hashicorp/terraform-provider-aws#40480

The limit exceeded error does not appear to be handled correctly.

Pulumi is inheriting this issue.

Sample program

N/A

Log output

No response

Affected Resource(s)

No response

Output of pulumi about

pulumi cli v3.124.0
python package pulumi-aws 6.67.0

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

I understand this bug may be difficult to hit reproducibly but could you attach a minimal example program? Pulumi hangs at pulumi up?

Ah upsream has an example:

resource "aws_route53_resolver_endpoint" "test" {
  name      = "test"
  direction = "OUTBOUND"
  resolver_endpoint_type = "IPV4"
  security_group_ids     = var.security_group_ids
  dynamic "ip_address" {
    for_each = var.subnet_ids
    content {
      subnet_id = ip_address.value
    }
  }
  protocols = ["Do53","DoH"]
}

Yes thanks so much for reporting and listing this issue. The ideal fix for Pulumi would be having the upstream issue fixed and then Pulumi provider inheriting the fix in a routine update.