Optimizing the Stack Template

My working stack, that is able to connect to the instance, has the following design template:

Overview of Design Template -No adjustments

My first focus in refining this stack is to remove any objects that are useless due to not being connected in the system.

This includes two EC2-route tables that aren’t attached within the VPCs, and two of the three elastic IPs, as they were also not necessary.

The updated template is as follows:
Updated Design template

I had originally thought that that the unattached security groups could also be removed from the VPCs, but when I attempted to rebuild the stack, there were dependency errors that forced me to put the security groups back into the JSON script.

My next step was to personalize the stack. This has been done by renaming the stack’s instance. I renamed it to “StackInstance” for easy identification.
EC2 renamed-Stack

My final attempt in optimizing the template, was to organize the stack instance to automatically connect with the elastic IP. This was done by using the following JSON script from: AWS::EC2::EIPAssociation

{
“Type”: “AWS::EC2::EIPAssociation”,
“Properties”: {
“AllocationId”: String,
“EIP”: String,
“InstanceId”: String,
“NetworkInterfaceId”: String,
“PrivateIpAddress”: String
}
}

This however, was unsuccessful as I couldn’t determine what was required for AllocationId, and so I have left it as a manual set-up.

That was every further adjustment done upon the CloudFormer template.

Connecting to the Stack’s Instance

Now that the stack’s instance is at a lower charge rate, I will try to connect to the t2.micro instance.

My first attempt resolved in the following error:

003 Still not able to access _Error

The first thing that I noticed, was that I didn’t have a public IP attached to the instance.
003 CFTemplate EC2 Instance Info

When I looked into the elastic IP section of the EC2 service, there were none there. As these are important in allowing me to connect to the RDGW instance, I added the JSON script for elastic IPs into the CloudFormer Template that I was using. The reason that they weren’t there, was because I had accidentally left them out during the CloudFormer template stack creation process.

When I ran the CloudFormer stack again, the RDGW instance still didn’t have a public IP, however, the elastic IPs had been created, so it was only a matter of manually associating the IP to the instance.

I then tried to connect again, but the connection still failed, with the same error response showing.

I considered then, that it may be an issue with the security groups attached to the stack’s VPC. My initial response was to adjust the JSON script as set all of the security groups’ ingress IPs to 0.0.0.0/0. This action was taken because I wanted to make everything open as a means of determining whether or not the failure to connect was due to the security groups. My next attempt to connect was still unsuccessful, which determined that it was not a security issue. Because this was ruled out, I replaced the security group IPs back to their original addresses for best practice purposes.

My next consideration was to utilize my other CloudFormer template that only had the single VPC in its design. This was to determine whether there may have been a CloudFormer template construction issue that was resulting in the connection failure. This however, was not the case as the single VPC template also failed to connect.

My final attempt, was to change the WiFi connection that I was using. This is because NMIT has to potential networks, both with different firewall settings, and the connection that I use has been known to not allow a remote connection to occur. This also, was unsuccessful, as was my attempt with my home network.

With all of these potential connection errors having been ruled out, I sought help from my classmates as to the design template of their successful stacks, as this would enable me to compare my stack’s design template and see the difference in design that was causing my connection error. While helping me with providing their design template, one of the classmates suggested that I try create a new CloudFormer template from the Microsoft’s Quick Start, Scenario 3, and create the stack to be completely open to any RDP IP addresses.

I did as he suggested, and during the creation of the initial stacks from that are based from scenario 3, and set the Network Configuration for ‘Allowed Remote Desktop Gateway External Access CIDR’ to 0.0.0.0/0.
005 Specify Details NS AZ_Options

Once the stack results were organized through the CloudFormer, I ran the new template, removing any of the errors in the JSON script that were causing the stack to rollback. Once the stack was complete, I attached an elastic IP to the instance, and attempted to connect to it. The result was successful.
01 ADDS RDP Success

The previous failures were due to a discrepancy from the remote desktop gateway external access CIDR that had been set-up with the creation of the stacks prior to CloudFormer. Once that had been resolved, the connection was available.

Budget Update: 09/06/2017

This budget update is the final update for AD-DS. As my instance has been changed to a t2.micro, there has been very little expense involved with this project.

The table below shows the billing costs for this project.
ADDSBudgetSheet

In order to determine how this budget update compares to the prior ones, I have organized certain service actions into the spreadsheet below.
ADDS Billing
This shows that there aren’t any outlying costs within my budget, which is expected as I haven’t done anything different from the previous weeks.

 

In conclusion, I originally expected the budget for the Active Directory project to be higher than what has resulted. This is good from a financial perspective, but does also imply that I still need to gain the knowledge and experience required to accurately predict the financial scope of this type of project.

 

 

 

Changing Instance Size in Stack Template

The Microsoft Quick Start template provides an enterprise instance of t2.large in its formatting, which costs more than what is needed for this project. This has been adjusted in the CloudFormer template by replacing the “t2.large” with “t2.micro”.

003 Instance size change

The template was able to run with this smaller instance, which is beneficial for me as I now don’t need to be as concerned with the costs involved for the instance, especially as it charges the same, per hour or part hour.

 

 

CloudFormer Template: Virtual Private Clouds

After having successfully created the stack from the cloud template, I have been unable to adjust it to that of the Quick Start diagram, I have however, managed to adjust my first template so that it successfully creates a stack also.

This provides me with two different templates, one that creates a single VPC and one that creates two VPCs.

001 1VPC

Stack with Single VPC

002-2vpcs.png

Stack with Two VPCs

My original purpose was to have multiple VPCs, so I will be using that specific template for further design adjustments.

 

Running the CloudFormer Template

This blog post follows on from the previous post: Adjusting the CloudFormer Template

After the first stack creation of the CloudFormer template, the rollback errors provide information on what needs to be adjusted in the JSON script.

The following list contains the ‘CREATE_FAIL’ events, and my method in resolving these errors.

Adjustments to the CF Template

  • FAIL: dbsubnetdefaultvpc91a918f5; Some input subnets (subnet-1ab50b43, subnet-93532cf7) are invalid.
    • Attempting to remove from the JSON script:
      “SubnetIds”: [
      “subnet-1ab50b43”,
      “subnet-93532cf7”
      ]
    • The first attempt was unsuccessful, so attempting to delete the entire subnet:
      “dbsubnetdefaultvpc91a918f5”: {
      “Type”: “AWS::RDS::DBSubnetGroup”,
      “Properties”: {
      “SubnetIds”: [
      “subnet-1ab50b43”,
      “subnet-93532cf7”
      ]
      “DBSubnetGroupDescription”: “Created from the RDS Management Console”,
      } },
    • This was successful

 

  • FAIL: Route4, Route1; Exactly one of [GatewayId, NatGatewayId, InstanceId, VpcPeeringConnectionId, NetworkInterfaceId, EgressOnlyInternetGatewayId] must be specified and not empty.
    • Attempting to remove from the JSON script, routes 4 and 1:
      “route4”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “0.0.0.0/0”,
      “RouteTableId”: {
      “Ref”: “rtbdbb390bf”
      }}},

      “route1”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “0.0.0.0/0”,
      “RouteTableId”: {
      “Ref”: “rtb40ad8e24”
      }}},

    • This was successful

 

  • FAIL: Route 5, Route 2; Exactly one of DestinationCidrBlock and DestinationIpv6CidrBlock must be specified and not empty.
    • Inserting “DestinationCidrBlock”: “0.0.0.0/0”, into Properties for Route2 in the JSON script.
    • Inserting “DestinationCidrBlock”: “10.0.0.0/19”, into properties for Route 5 in the JSON script (This relates to subnet cidr 1A)
    • This was successful

 

  • FAIL: Route 5, Route 2; The Gateway ID (vpce-20e01049) does not exist.
    • Attempting to delete both routes from the JSON script:
      “route2”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “0.0.0.0/0”,
      “RouteTableId”: {
      “Ref”: “rtb40ad8e24”
      },
      “GatewayId”: “vpce-20e01049”
      }},

      “route5”: {
      “Type”: “AWS::EC2::Route”,
      “Properties”: {
      “DestinationCidrBlock”: “10.0.0.0/19”,
      “RouteTableId”: {
      “Ref”: “rtbdbb390bf”
      },
      “GatewayId”: “vpce-20e01049”
      }},

    • This was successful

 

  • FAIL: lcADDSScenario3RDGWStack1USE0PZ69GKRQRDGWLaunchConfiguration1QJ9NVFDQSTXX;
    Invalid IamInstanceProfile: AD-DS-Scenario-3-RDGWStack-1USE0PZ69GKRQ-RDGWHostProfile-CLZHHC4VKEC1

    • Attempting to delete section from JSON script (line 269) … LaunchConfiguration… object:
      “IamInstanceProfile”: “AD-DS-Scenario-3-RDGWStack-1USE0PZ69GKRQ-RDGWHostProfile-CLZHHC4VKEC1”,
    • This was successful

 

After resolving all of these errors, my CloudFormer template was able to create a stack without any rollbacks.

037 CFTemplate Complete

Although I am pleased that I managed to enable the stack to reach the status of ‘CREATE_COMPLETE’, due to the large amount of script deleted, I am uncertain as to whether my script still runs as it was originally designed.

When I run my template though the AWS template designer, the following diagram is displayed.
038 CFTemplate DesignerTemplate

This diagram does not look the same as the sample diagram found in the Microsoft Quick Start guide for ‘Scenario 3’, shown below.
Figure for Scenario 3

My next step then, is to compare the two diagrams to determine the discrepancies between my stack template and the sample template.

Adjusting the CloudFormer Template

Once the AWS CloudFormer template has been created, it needs to be run through the AWS CloudFormation ‘Create Stack’ option. This has been done by copying the CloudFormer JSON script into the Notebook++ program, and then uploading the Notebook++ file during the ‘Select Template’ portion of the Stack creation settings.

000 Upload File

Once, the template has been chosen, it requires a name, and the option of a tag. For my stacks, they have been numbered in regards to how many iterations of the script I have run through ‘Create Stack’.

010 NewReview

During the creation process of the stack, the created events can be viewed.

005 Create_In_Progress

The initial template stack contains errors that cause the creation process to rollback, and fail to complete the stack’s creation.

006 Rollback Error

The method used in removing the errors, was to find the first ‘CREATE_FAILED’ event for the stack, and attempt to solve this event failure based upon the information provided in the right-hand column of the event.

For my first CloudFormer script, came across the following problems, which I attempted to solve.

Adjusting the CF Script

  • DestinationCidrBlock Empty
    • For each event fail that referenced this error, I sourced the IPs and their respective Cidr from the initial Microsoft ‘Scenario 3’ stack creation settings, and placed them into their corresponding Routes based upon the route description.
  • Missing NetworkInterfaceID
  • Unresolved Dependencies
    • The unresolved dependencies were specific to two different routes. As I couldn’t determine a resolution to this error, I saved a copy of each of the routes, then deleted them.

These errors comprised the bulk of my event failures, and throughout each iteration of the stack, more errors would evolve. Most of these errors were derived from the network interface JSON script that I had added to the CloudFormer script. I eventually came to conclusion, with the help of others, that I should re-attempt the Microsoft Quick-Start ‘Scenario 3’ template through CloudFormation and Cloudformer, which would provide me with a clean template to work upon again.

The persuasive reason behind this action was the knowledge that this project was not intended to focus on the intricacies of the JSON script, which I had been doing in my attempts to successfully fix my first CloudFormer script.

Microsoft Active Directory Quick Start: Scenario 3

In order to fully determine how the ‘Microsoft Active Directory Quick Start: AD DS with AWS Directory Service on the AWS Cloud’ template works, I implemented the template into AWS CloudFormation’s ‘Create Stack’ option.
001 New Stack_LI

The creation of this stack involved many specific details that I hadn’t previously considered would be of importance. However, as I went through the details, I recalled seeing them within the JSON scripts that I had been altering.
003 Specify Details NS AZ_Options004 Specify Details NS EC2_Config N_D005 Specify Details NS RDGW N_D

There were a few things that had me momentarily confused. One of these was with my availability zones, as I had not realized that my account region was set to Oregon. As I live in New Zealand, the Sydney region is the most optimal for reducing latency. To change this, I needed to quit this stack creation and start it again within the Sydney region. Also, because of initial lack of realization of setting up the template within a different region, I did not understand why I could access my key pair. The revelation of being in the wrong region helped my realize that key pairs are region specific, something which  I did not know earlier.

The next step in setting up the stack was the ‘Options’, in which I created a tag for the stack, but did not change any of the permission settings as I currently don’t want to create more potential complication than necessary. This is something that I could consider adjusting once I manage to successfully recreate the CloudFormer version of this stack.
006 Options NS

Upon reviewing the reviewing the stack template to be created, and pressing ‘create’, the site responded with the following error:
010 Error from CIDR

In my ignorance of what information was required for the stack creation, I had left the Remote Desktop Gateway (RDGW) CIDR blank in the ‘Network Configuration’ section. I had also left the ‘Microsoft Active Directory’ section blank, as I didn’t fully understand why it was needed, and so had left it clear.

My first attempt to solve to the RDGW CIDR was to input the VPC CIDR into it. However, this did not work. One of my classmates suggested using the CIDR from the first public subnet, which ended up working, as it appears that the gateway required a larger mask than the one that is supplied with the VPC.
012 RDGW IP config NS

I also filled in the Microsoft Active Directory section once I realized that it required input.
011 AD Config for NS

 

Once these problems were resolved, I was able to create the stack.
018 AD_DS Stacks
This particular template created four stacks; an AD stack, a VPC stack, an RDGW stack, and a general stack. Each of which, implemented certain AWS services.

The RDGW stack had the following outputs:
022 RDGW Stack Outputs

 

The AD stack had the following outputs:
023 AD Stack Overview

 

The VPC stack had the following outputs:
024 VPC Stack Outputs

 

The general stack had no outputs:
025 3S Stack Outputs

 

The outputs from the stacks are important to know as they can be selected in CloudFormer, but don’t necessarily have a tag attached, which can make them hard to distinguish from any other service objects that still exist within the AWS system. It is also important because not all of them exist on the free tier, which means that if the stacks are left running, a large fee can quickly accrue. The simplest way to stop the fees is to delete the stacks as that deletes all of the service objects involved. This is not always the optimal choice, but it is the best one for me once I’ve mapped the stacks through CloudFormer.

AD in CloudFormation: JSON files

When starting up scenario 3 (mentioned in my previous blog), the CloudFormation service offers the option of ‘View/Edit template in Designer’. This provides me the JSON script for the template, and a physical diagram.

CF JSON and PhysMod

My first attempt in understanding and manipulating the template was by playing with the JSON scripts. I was playing around with the template, which contained links between different stacks. This made it harder to adjust the code as I needed to constantly determine where the script was pointing.

In the end, I was unable to modify the code and keep it in working order. However, this endeavor did help me gain an understanding of the code, for its format, and what AWS services it contains.

This information should provide me greater understanding in how to adjust the template that I will gain through my next method of using CloudFormer.

Introduction to Active Directory Domain Services

This next project involves using AWS’s CloudFormation service in building an active directory (AD), and also deploying and automating it.

Amazon CloudFormation service is described as “a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications”.
Hence, it is an applicable service for deploying and automating an AD. –What is AWS CloudFormation

The specific CloudFormation template that will be used for this project is the ‘Microsoft Active Directory Quick Start: AD DS with AWS Directory Service on the AWS Cloud’, which is scenario 3.
Quick Start Options

The template is designed to provide the following architecture:
Figure for Scenario 3

The purpose for this project is to redesign the template so that it is simplified, improved, optimized, and personalized.