Optimizing the Stack Template

My working stack, that is able to connect to the instance, has the following design template:

Overview of Design Template -No adjustments

My first focus in refining this stack is to remove any objects that are useless due to not being connected in the system.

This includes two EC2-route tables that aren’t attached within the VPCs, and two of the three elastic IPs, as they were also not necessary.

The updated template is as follows:
Updated Design template

I had originally thought that that the unattached security groups could also be removed from the VPCs, but when I attempted to rebuild the stack, there were dependency errors that forced me to put the security groups back into the JSON script.

My next step was to personalize the stack. This has been done by renaming the stack’s instance. I renamed it to “StackInstance” for easy identification.
EC2 renamed-Stack

My final attempt in optimizing the template, was to organize the stack instance to automatically connect with the elastic IP. This was done by using the following JSON script from: AWS::EC2::EIPAssociation

{
“Type”: “AWS::EC2::EIPAssociation”,
“Properties”: {
“AllocationId”: String,
“EIP”: String,
“InstanceId”: String,
“NetworkInterfaceId”: String,
“PrivateIpAddress”: String
}
}

This however, was unsuccessful as I couldn’t determine what was required for AllocationId, and so I have left it as a manual set-up.

That was every further adjustment done upon the CloudFormer template.

Connecting to the Stack’s Instance

Now that the stack’s instance is at a lower charge rate, I will try to connect to the t2.micro instance.

My first attempt resolved in the following error:

003 Still not able to access _Error

The first thing that I noticed, was that I didn’t have a public IP attached to the instance.
003 CFTemplate EC2 Instance Info

When I looked into the elastic IP section of the EC2 service, there were none there. As these are important in allowing me to connect to the RDGW instance, I added the JSON script for elastic IPs into the CloudFormer Template that I was using. The reason that they weren’t there, was because I had accidentally left them out during the CloudFormer template stack creation process.

When I ran the CloudFormer stack again, the RDGW instance still didn’t have a public IP, however, the elastic IPs had been created, so it was only a matter of manually associating the IP to the instance.

I then tried to connect again, but the connection still failed, with the same error response showing.

I considered then, that it may be an issue with the security groups attached to the stack’s VPC. My initial response was to adjust the JSON script as set all of the security groups’ ingress IPs to 0.0.0.0/0. This action was taken because I wanted to make everything open as a means of determining whether or not the failure to connect was due to the security groups. My next attempt to connect was still unsuccessful, which determined that it was not a security issue. Because this was ruled out, I replaced the security group IPs back to their original addresses for best practice purposes.

My next consideration was to utilize my other CloudFormer template that only had the single VPC in its design. This was to determine whether there may have been a CloudFormer template construction issue that was resulting in the connection failure. This however, was not the case as the single VPC template also failed to connect.

My final attempt, was to change the WiFi connection that I was using. This is because NMIT has to potential networks, both with different firewall settings, and the connection that I use has been known to not allow a remote connection to occur. This also, was unsuccessful, as was my attempt with my home network.

With all of these potential connection errors having been ruled out, I sought help from my classmates as to the design template of their successful stacks, as this would enable me to compare my stack’s design template and see the difference in design that was causing my connection error. While helping me with providing their design template, one of the classmates suggested that I try create a new CloudFormer template from the Microsoft’s Quick Start, Scenario 3, and create the stack to be completely open to any RDP IP addresses.

I did as he suggested, and during the creation of the initial stacks from that are based from scenario 3, and set the Network Configuration for ‘Allowed Remote Desktop Gateway External Access CIDR’ to 0.0.0.0/0.
005 Specify Details NS AZ_Options

Once the stack results were organized through the CloudFormer, I ran the new template, removing any of the errors in the JSON script that were causing the stack to rollback. Once the stack was complete, I attached an elastic IP to the instance, and attempted to connect to it. The result was successful.
01 ADDS RDP Success

The previous failures were due to a discrepancy from the remote desktop gateway external access CIDR that had been set-up with the creation of the stacks prior to CloudFormer. Once that had been resolved, the connection was available.

Running the CloudFormer Template

This blog post follows on from the previous post: Adjusting the CloudFormer Template

After the first stack creation of the CloudFormer template, the rollback errors provide information on what needs to be adjusted in the JSON script.

The following list contains the ‘CREATE_FAIL’ events, and my method in resolving these errors.

Adjustments to the CF Template

  • FAIL: dbsubnetdefaultvpc91a918f5; Some input subnets (subnet-1ab50b43, subnet-93532cf7) are invalid.
    • Attempting to remove from the JSON script:
      “SubnetIds”: [
      “subnet-1ab50b43”,
      “subnet-93532cf7”
      ]
    • The first attempt was unsuccessful, so attempting to delete the entire subnet:
      “dbsubnetdefaultvpc91a918f5”: {
      “Type”: “AWS::RDS::DBSubnetGroup”,
      “Properties”: {
      “SubnetIds”: [
      “subnet-1ab50b43”,
      “subnet-93532cf7”
      ]
      “DBSubnetGroupDescription”: “Created from the RDS Management Console”,
      } },
    • This was successful

 

  • FAIL: Route4, Route1; Exactly one of [GatewayId, NatGatewayId, InstanceId, VpcPeeringConnectionId, NetworkInterfaceId, EgressOnlyInternetGatewayId] must be specified and not empty.
    • Attempting to remove from the JSON script, routes 4 and 1:
      “route4”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “0.0.0.0/0”,
      “RouteTableId”: {
      “Ref”: “rtbdbb390bf”
      }}},

      “route1”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “0.0.0.0/0”,
      “RouteTableId”: {
      “Ref”: “rtb40ad8e24”
      }}},

    • This was successful

 

  • FAIL: Route 5, Route 2; Exactly one of DestinationCidrBlock and DestinationIpv6CidrBlock must be specified and not empty.
    • Inserting “DestinationCidrBlock”: “0.0.0.0/0”, into Properties for Route2 in the JSON script.
    • Inserting “DestinationCidrBlock”: “10.0.0.0/19”, into properties for Route 5 in the JSON script (This relates to subnet cidr 1A)
    • This was successful

 

  • FAIL: Route 5, Route 2; The Gateway ID (vpce-20e01049) does not exist.
    • Attempting to delete both routes from the JSON script:
      “route2”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “0.0.0.0/0”,
      “RouteTableId”: {
      “Ref”: “rtb40ad8e24”
      },
      “GatewayId”: “vpce-20e01049”
      }},

      “route5”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “10.0.0.0/19”,
      “RouteTableId”: {
      “Ref”: “rtbdbb390bf”
      },
      “GatewayId”: “vpce-20e01049”
      }},

    • This was successful

 

  • FAIL: lcADDSScenario3RDGWStack1USE0PZ69GKRQRDGWLaunchConfiguration1QJ9NVFDQSTXX;
    Invalid IamInstanceProfile: AD-DS-Scenario-3-RDGWStack-1USE0PZ69GKRQ-RDGWHostProfile-CLZHHC4VKEC1

    • Attempting to delete section from JSON script (line 269) … LaunchConfiguration… object:
      “IamInstanceProfile”: “AD-DS-Scenario-3-RDGWStack-1USE0PZ69GKRQ-RDGWHostProfile-CLZHHC4VKEC1”,
    • This was successful

 

After resolving all of these errors, my CloudFormer template was able to create a stack without any rollbacks.

037 CFTemplate Complete

Although I am pleased that I managed to enable the stack to reach the status of ‘CREATE_COMPLETE’, due to the large amount of script deleted, I am uncertain as to whether my script still runs as it was originally designed.

When I run my template though the AWS template designer, the following diagram is displayed.
038 CFTemplate DesignerTemplate

This diagram does not look the same as the sample diagram found in the Microsoft Quick Start guide for ‘Scenario 3’, shown below.
Figure for Scenario 3

My next step then, is to compare the two diagrams to determine the discrepancies between my stack template and the sample template.

Adjusting the CloudFormer Template

Once the AWS CloudFormer template has been created, it needs to be run through the AWS CloudFormation ‘Create Stack’ option. This has been done by copying the CloudFormer JSON script into the Notebook++ program, and then uploading the Notebook++ file during the ‘Select Template’ portion of the Stack creation settings.

000 Upload File

Once, the template has been chosen, it requires a name, and the option of a tag. For my stacks, they have been numbered in regards to how many iterations of the script I have run through ‘Create Stack’.

010 NewReview

During the creation process of the stack, the created events can be viewed.

005 Create_In_Progress

The initial template stack contains errors that cause the creation process to rollback, and fail to complete the stack’s creation.

006 Rollback Error

The method used in removing the errors, was to find the first ‘CREATE_FAILED’ event for the stack, and attempt to solve this event failure based upon the information provided in the right-hand column of the event.

For my first CloudFormer script, came across the following problems, which I attempted to solve.

Adjusting the CF Script

  • DestinationCidrBlock Empty
    • For each event fail that referenced this error, I sourced the IPs and their respective Cidr from the initial Microsoft ‘Scenario 3’ stack creation settings, and placed them into their corresponding Routes based upon the route description.
  • Missing NetworkInterfaceID
  • Unresolved Dependencies
    • The unresolved dependencies were specific to two different routes. As I couldn’t determine a resolution to this error, I saved a copy of each of the routes, then deleted them.

These errors comprised the bulk of my event failures, and throughout each iteration of the stack, more errors would evolve. Most of these errors were derived from the network interface JSON script that I had added to the CloudFormer script. I eventually came to conclusion, with the help of others, that I should re-attempt the Microsoft Quick-Start ‘Scenario 3’ template through CloudFormation and Cloudformer, which would provide me with a clean template to work upon again.

The persuasive reason behind this action was the knowledge that this project was not intended to focus on the intricacies of the JSON script, which I had been doing in my attempts to successfully fix my first CloudFormer script.

Budget Update: 28/05/2017

This past week has also been spent on adjusting a CloudFormer template. As such, my current expectation for my billing list is that it will be quite low. The only expenses that should have occurred, are the AD-DS CloudFormation template (description link here) that I ran again, and the charge from the successful build of the CloudFormer template’s stack.

When looking at my billing list for my account, only two services have been used; EC2, and KMS. Upon further inspection, I have determined that the KMS charge is from an encryption key set in US-East region, which is not related to the AD-DS build.

The EC2 billing report is as follows:
EC2

 

I input this data, as well as last weeks data, in order to grasp a greater understanding of what charges have been incurred.

AD DS Budget Sheet

It appears that this past week has been less expensive than the one prior. This reduced charge is found within the EBS service specifics, which still contains the carry-over charges from the DinoStore volumes that I recently deleted. Hence, my first EBS budget isn’t exclusive to the AD-DS project charges and needs to be adjusted.

The adjusted AD-DS Budget Sheet is below, where the fee discrepancy has been resolved using the information from the ‘Budget Update: 14/05/2017‘, which contains my last reference to the DinoStore EBS volumes.

Budget Sheet Adjusted

I am unsure of the $0.19 difference in the General Purpose SSD (gp2) provisioned storage expense for the two dates, but may be able to verify whether this expense difference could be considered outlying with my next budget report.

 

AD in CloudFormation: JSON files

When starting up scenario 3 (mentioned in my previous blog), the CloudFormation service offers the option of ‘View/Edit template in Designer’. This provides me the JSON script for the template, and a physical diagram.

CF JSON and PhysMod

My first attempt in understanding and manipulating the template was by playing with the JSON scripts. I was playing around with the template, which contained links between different stacks. This made it harder to adjust the code as I needed to constantly determine where the script was pointing.

In the end, I was unable to modify the code and keep it in working order. However, this endeavor did help me gain an understanding of the code, for its format, and what AWS services it contains.

This information should provide me greater understanding in how to adjust the template that I will gain through my next method of using CloudFormer.