Dedicated/Standardized Exports Unwrapper

[ambsw-technology]

Assume I want to create a module like this:

Parameters:
  Environment:
    Description: 'Name of client Environment.'
    Type: String
Resources
  PeeringConnection:
    Type: 'AWS::EC2::VPCPeeringConnection'
    Properties:
      VpcId: {'Fn::ImportValue': !Sub '${VpcModule}-Id'}
      ...

BUT the VpcModule has been exported at ${Environment}-VpcModule. How do I get to VpcModule from Environment?

• My first solution was to move the PeeringConnection into a nested stack. I can use an import to populate the VpcModule parameter and, within the stack, do the secondary import.
• I've since realized that you can get these exports to the outer layer using a dedicated nested stack. For example...

# cfn-modules/vpc/outputs.yaml

Parameters:
  VpcModule:
    Type: String
Conditions:
  Never: !Equals ['true', 'false']
Resources:
  NullResource:
    Condition: Never
    Type: 'Custom::Null'
Outputs:
  Id:
    Value: {'Fn::ImportValue': !Sub '${VpcModule}-Id'}
    Export:
      Name: !Sub '${AWS::StackName}-Id'

This lets me do everything on the outer level:

Parameters:
  Environment:
    Description: 'Name of client Environment.'
    Type: String
Resources
  VpcModuleOutputs:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        VpcModule:  {'Fn::ImportValue': !Sub '${Environment}-VpcModule'}
      TemplateURL: './cfn-modules/vpc/outputs.yml'
  PeeringConnection:
    Type: 'AWS::EC2::VPCPeeringConnection'
    Properties:
      VpcId: !GetAtt 'VpcModuleOutputs.Outputs.Id'
      ...

I'd like to suggest including a standardized outputs (or exports) YAML file in each package that wraps a module's stackname and provides all of its exports.

[michaelwittig]

Not sure if I get you right. Wouldn't the following work?

Parameters:
  Environment:
    Description: 'Name of client Environment.'
    Type: String
Resources
  PeeringConnection:
    Type: 'AWS::EC2::VPCPeeringConnection'
    Properties:
      VpcId: {'Fn::ImportValue': !Sub '${Environment}-VpcModule-Id'}
      ...

[ambsw-technology]

In theory yes, but that requires me to either (1) export all of the nested parameters in the wrapper or (2) continually modify the wrapper to expose values as-needed. I'd like to avoid polluting my exports in both cases and definitely don't want to be regularly updating the stack to facilitate the second.

My applications are broken into a bunch of pieces. About half of the modules are separated for practical reasons (e.g. reuse) and will stay that way. The other half are separated for debugging reasons e.g. I don't want to redeploy DB/Redis every time I want to test/troubleshoot adjustments to service templates that require me to deploy and undeploy (e.g. due to eports).

So I have e.g. a base template that creates and exports a VPC:

# app-base.yaml
Parameters:
  Environment:
    Description: 'Name of client to which Application is dedicated.'
    Type: String
Resources:
  Vpc:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
         ...
      TemplateURL: 'cfn-modules/vpc/modules.yml'
 ...
Outputs:
  Vpc:
    Description: 'Environment VPC module.'
    Value: !GetAtt 'Vpc.Outputs.StackName'
    Export:
      Name: !Sub 'APEX-${Environment}-VPC'

In most places where I'm using it, I've been passing this value to a nested stack so it doesn't matter that I can't double-import.

# app-service-1.yaml
Parameters:
  Environment:
    Description: 'Name of client to which DB is dedicated.'
    Type: String
Resources:
  Task:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        ParentVPCStack: {'Fn::ImportValue': !Sub 'APEX-${Environment}-VPC'}
        ...
      # in reality, a custom implementation
      TemplateURL: './aws-cf-templates/fargate/service.yaml'

In theory, I can create a new module for everything so I can double-import again. In a lot of cases I do. In some cases I'm not really reusing the feature so it doesn't make as much sense. I can still create the wrapper but it ends up being pretty trivial:

# vpc-peer.yaml
Parameters:
  Environment:
    Description: 'Name of client Environment.'
    Type: String
Resources:
  VpcPeer:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        RequesterVpcModule: {'Fn::ImportValue': !Sub 'APEX-${Environment}-VpcModule'}
      TemplateURL: './ambsw-cfn-modules/vpc-peer/module.yml'

I am trying to propose a way to unwrap module exports without the need for the nested module. The trivial wrapper is certainly a workaround. Maybe trivial wrappers are even the best practice. But I wanted to demonstrate a solution I had found that does not require extra wrappers.

[michaelwittig]

ok. So how would that "standardized outputs (or exports) YAML file in each package" look like?

[ambsw-technology]

From my original post, you can use the following pattern to convert all of a module's exports into outputs...

# cfn-modules/vpc/outputs.yaml

Parameters:
  VpcModule:
    Type: String
Conditions:
  Never: !Equals ['true', 'false']
Resources:
  # since a resource is required, this is a trick to make sure nothing happens
  NullResource:
    Condition: Never
    Type: 'Custom::Null'
Outputs:
  Id:
    Value: {'Fn::ImportValue': !Sub '${VpcModule}-Id'}
    Export:
      Name: !Sub '${AWS::StackName}-Id'
  # ... all of the rest of the outputs

For the actual VPC module (and similar), you may need a second parameter e.g. NumberAZs to know whether to export the C values, but you can provide the minimal default (e.g. 2). Then the actual user uses them in this way:

Parameters:
  Environment:
    Description: 'Name of client Environment.'
    Type: String
Resources
  VpcModuleOutputs:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        VpcModule:  {'Fn::ImportValue': !Sub '${Environment}-VpcModule'}
      TemplateURL: './cfn-modules/vpc/outputs.yml'
  PeeringConnection:
    Type: 'AWS::EC2::VPCPeeringConnection'
    Properties:
      VpcId: !GetAtt 'VpcModuleOutputs.Outputs.Id'
      ...

You don't need to pollute any export spaces. The outputs.yml has the exact same outputs as the module.yml so it's a drop-in replacement that uses existing/exported resources. It's a really simple pattern that mirrors module.yml so it's easy to standardize and offer as a feature.

[ambsw-technology]

It occurs to me that this strategy could help fully modularize complex modules like VPC (and anything else affected by the discussion in #36).

For example, I want/need the CIDR range for the two Public Subnets. Why? AWS Endpoint Services are attached to an NLB and consumer traffic looks like it's coming from the NLB's IP Address. Since the NLB IP addresses can't be obtained easily, I need to add the entire public (really DMZ in my world) range to my ALB Ingress to permit traffic. To do this today, I have to add the CIDR block output to both vpc-subnet and vpc.

In the extreme version, it would be possible to instead:

• Export the stack name for Public from VPC
• Use an outputs resource to expose the Public interface
• From the Public outputs, pick the ASubnet stack name
• Use an outputs resource to extract the entire Subnet interface
• From the Subnet outputs, pick the CIDR value

In this approach, only the Subnet interface needs modified (on module and outputs). This ensures that the Subnet interface can be arbitrarily complex without filling the parent outputs with objects. The key exception are lists-of-outputs. For example AvailabilityZones and SubnetIdsPublic (under their interface-based names) would still need to be aggregated and exported by VPC and Public respectively.