Final Budget Report: 09/06/2017

As the semester comes to a close, so does my budgeting for this course.

Of the $100 USD credits for Amazon Web Services, I have $79.35.

Looking over the past months, my expenditure has been:
Monthly Report Graph

This can be represented as proportions of the credits, as follows:
Visual Monthly Report

 

In terms of projects, the QwikLabs assignment was for the most of March, Dinostore was during April, and AD-DS has been from the start of May. This is stated in the table below.
Project Billing

The discrepancy between my expenditure and my credits occurs because my expenditure is based upon my billing list which updates more frequently than the credit amount in my account.

The reason for the large cost involved with the Dinostore project is due to my lack of experience with AWS’ RDS, which I accidentally left running for multiple hours and hence the increased charge. This was not the only reason however, as I did often purposefully run RDS during the course of the project.

 

Conclusion
This assignment of budgeting has been beneficial for myself in gaining knowledge and habits for keeping account of my money on the Amazon Web Services platform. These skills will be able to be transferred and utilized within other aspects of my study, future career, and personal life for goals that involve financial oversight.

 

Advertisements

Budget Update: 09/06/2017

This budget update is the final update for AD-DS. As my instance has been changed to a t2.micro, there has been very little expense involved with this project.

The table below shows the billing costs for this project.
ADDSBudgetSheet

In order to determine how this budget update compares to the prior ones, I have organized certain service actions into the spreadsheet below.
ADDS Billing
This shows that there aren’t any outlying costs within my budget, which is expected as I haven’t done anything different from the previous weeks.

 

In conclusion, I originally expected the budget for the Active Directory project to be higher than what has resulted. This is good from a financial perspective, but does also imply that I still need to gain the knowledge and experience required to accurately predict the financial scope of this type of project.

 

 

 

Comparison of Cloud Computing Service Vendors

The purpose of this analysis is to identify vendors who are offering IaaS public cloud computing services that could be used by New Zealand organizations.

The three vendors being compared are: IBM, Google Cloud Platform, and Amazon Web Services (AWS).

Comparing Technologies:

What are three technologies that these vendors offer?

1.) Storage:
Storage servers are designed to hold data volumes from a company, while keeping the data enabled to be transferred. Storage quality can be considered with five different factors; flexibility, scalability, reliability, availability, and data integrity.

Flexibility of storage is based upon whether the deployment is public, private, of hybrid, and whether the deployment solution is held within a region, or across many regions. [1]

Scalabiltiy of storage is the ability for the server or servers to handle large changes in data volume as the required by the organization.
This can be done by scaling out, which is the process of increasing server number in response to increased data volumes, or scaling out, which is the process of adding more to a single server in response to increased data volumes. [4]

Reliability is a percentage measurement based upon the cloud provider’s promised accuracy of data, and guarantee of transfer delivery of the data. [5]

Availability is a percentage measurement based upon the cloud provider’s promised up-time, with maintenance outages such as updates, and unplanned outages kept in mind. [6]

Data integrity is the ability on the provider to ensure that any data does not become corrupted, and if such a case occurs, the data is able to be fixed.

IBM offer storage services of object storage, file storage, block storage, and mass storage. [2]
Google Cloud Platform offers object storage, file storage, and big data storage. [9] 
AWS offers object storage, file storage, and block storage. However, it contains multiple services of object storage, each designed for different purposes. [10]

2.) Load Balancing
Load balancing is designed to assist with the uptime of a server and the response time of the traffic involved. This is done having the processing and communications of a system shared across multiple servers, thereby reducing the strain from any single server. This is effective in reducing latency, and easing the load on the servers.[7]

IBM offers local load balancing, local and global load balancing, and high availability dedicated load balancers. [7]
Google Cloud Platform offers global load balancing, regional internal load balancing, and regional network load balancing. [12]
AWS
 offers a ‘Classic Load Balancer’ which applies to application and network level information. It also offers an ‘Application Load Balancer’ which applies to application level information that is more advanced than what is required for the Classic Load Balancer. The Classic Load Balancer is able to route traffic across AWS’s ‘EC2 instances’, whereas the Application Load Balancer is able to route traffic and load balance on a single instance.[11]

3.) Containers
Containers are a form of virtualization technology that are able to run applications, and storage in an isolated environment. Containers store one application and its relevant data that is required for it to run. This form of application packaging is designed to increase infrastructure efficiency, with each container only using the least amount of required resources, and multiple containers able to be run upon a single instance. [13][14][15]

IBM provides a container service that utilizes the open-source ‘Kubernetes’, which assist in deployment, scaling, and management of the container. [16]
Google Cloud Platform provides cluster set-up and management for containers, which are built upon the Kubernetes system. [17]
AWS provides a container service for the management of Docker containers. The containers are designed to use a cluster of EC2 instances, and utilize many other AWS services. [18]

 

Comparing Technology Charges:

What are the charges involved for these technologies?

1.) Storage
Storage pricing is considered for regional storing.

IBM storage is located in the US, which would imply latency issues. The website currently provides pricing information for the US, but not for Australia or New Zealand. However, this doesn’t necessarily imply that Australia doesn’t have an IBM server centre, it simply requires further research and contact to find it.

Object Storage [19]  Per GB per Month
(US Pricing) Regional Resiliency
Storage  0-499.99TB $0.022
Storage  500TB+ $0.020
Data Retrieval No Charge
Class A Operations: PUT, COPY, POST, and LIST Requests ($ per 1,000 Requests) $0.006
Class B Operations: GET and all other Requests ($ per 10,000 Requests) $0.005
Delete Requests No Charge

Google Cloud Platform is set in Singapore, which is the closest location to New Zealand, as Google doesn’t have an Australian region yet. [20]

Object Storage [21] Per GB per Month
 (US Pricing) Regional
Storage $0.020
Data Transfer Free
Class A Operations (per 10,000 operations) $0.050
Class B Operations (per 10,000 operations) $0.004
Free Operations Free

AWS do have a region in Sydney, Australia, and as such, these prices are taken from that location. However, the pricing is still in USD.

Object Storage [22] Per GB per Month
 (US Pricing) Regional
Storage:First 50 TB 0.025
Storage: Next 450 TB 0.024
Storage: Over 500 TB 0.023
Data Transfer  FREE
Class A Operations (per 10,000 operations) $0.0055
Class B Operations (per 10,000 operations) $0.0044
Delete Requests FREE

Conclusion
In terms of regional storage pricing, Google Cloud Platform offer the lowest flat-rate pricing, but are more expensive for their combined costs class A and B operations than AWS. IBM offers the next lowest tiered-pricing for storage, but their class A operations cost almost ten times the amount of the AWS class A operations. In conclusion, I would consider AWS to offer the lowest price for storage overall.

 

2.) Load Balancing

IBM

Load Balancing [23]
Local Load Balancing (Per month) W SSL
250 Connections 49.99 99.99
500 Connections 99.99 199.99
1000 Connections 199.99 139.99
2500 Connections 499.99 999.99
5000 Connections 999.99 1999.99

Google Cloud Platform

Load Balancing [24]
Item Price per Unit (USD) Pricing Unit
First 5 forwarding rules $0.025 Per Hour
Per additional forwarding rule $0.010 Per Hour
Data processed $0.008 Per GB

AWS

Load Balancing [25]
0.0252 per Application Load Balancer-hour (or partial hour)
0.008 per LCU*-hour (or partial hour)
*LCU contains: [25]
25 new connections per second
3000 active connections per minute
2.22MBps (=1 GB per hour)
1000 rule evaluations per second

Conclusion
Comparing load balancing between the different providers is harder than comparing storage as each provider measures load balancing differently. In this instance, the specifications of an organization requiring load-balancing would hold more information in regards to which provider would be most suitable. For a smaller sized business, I would consider the AWS load balancing option to be the most suitable as it is cheaper than Google Cloud Platform’s service, and may not require the same amount of connections that is offered by IBM’s service.

 

3.) Containers

IBM
IBM utilizes the open-source Kubernetes and does not display a charge allocated with this service. [16]

Google Cloud Platform

Container [26]
Item Cost
First 120 build-minutes per day per billing account No charge *
Additional build minutes** $0.0034 / build-minute
* Promotional free tier of 120 free build-minutes per day is subject to change.

**The Google Cloud Platform defines build minute charges as such: “A build-minute is incurred for every minute that a build initiated by Container Builder is in process. Build-minutes are not incurred for the time that a build is queued. Charges are accrued to the billing account associated with the Google Cloud Platform Console project that initiated the build.” [26]

AWS
Amazon Web Services states the following in regards to container pricing: “There is no additional charge for Amazon EC2 Container Service. You pay for AWS resources (e.g. EC2 instances or EBS volumes) you create to store and run your application. You only pay for what you use, as you use it; there are no minimum fees and no upfront commitments.” [27]

Conclusion
In terms of container service options, IBM is an option but does require a separate site which creates more hassle then what is optimal. The Google Cloud Platform charges for the same type of storage offered free by IBM, and hence, would be my last choice. AWS only charges for the resources used by the container, which is something that neither other provider mentioned. However it uses a different form of container than what is offered by IBM and Google Cloud Platform, which may cause difficulties for a business, but that is something that would require further study.

 

Comparing Security Measures:

What security measures are used to ensure the safety of operations and client systems/data?

IBM
IBM’s security measures deal with security, privacy, and compliance. In regards to compliance, their website provides a compliance list with respect to their Bluemix products. [28] In terms of security and privacy from IBM’s position, they provide partnership options for ‘Intrusion Protection Systems’ and ‘Intrusion Detection and Prevention Systems’ with security software, scanning and logging capabilities, and regular updates.[29]  In terms of security and privacy from an organization’s position, security measures such as key generation, privileges and roles, passwords, and session inactivity lockout. [30]

Google Cloud Platform
Google Cloud Platform also has security measures that involve security, privacy, and compliance. Google Cloud Platform’s compliance list involves independent audits of infrastructure, services and operations. [31] In terms of security and privacy from Google Cloud Platform’s position, they hold security and privacy events for raising awareness, have a security team, a privacy team, an internal audit and compliance team. They also assist their customers in vulnerability management, malware prevention, as well as constantly monitoring network traffic for security issues, and processing any incident management. [32] In terms of security and privacy from an organization’s position, Google Cloud Platform provides key encryption and management, [33] Identity and Access Management (IAM) permissions, roles, requests, user management, network firewall maintenance, logging, and penetration testing. [34]

AWS
AWS also contain security measures for security, privacy, and compliance. In terms of AWS’s responsibility, they provide a whitepaper detailing compliance and risk management, [35] they provide detail and best practices for optimal cloud security such as data encryption, monitoring and logging, identity and access control, and available security partners. [36] In terms of an organization’s responsibility, the customer has complete control over their data so it lies within their responsibility to ensure its security and privacy. [37] This can be achieved through key management, roles, permissions, user management, logging, encryption, incident response protocols established, multi-factor authentication, and boundary protection for both network and host levels. [37]

Conclusion
Each provider is concerned about security, privacy, and compliance. The main difference in each provider, is the level of control and responsibility that customer obtains. Google Cloud Platform has most provider-based security measures, which involve task-specific teams and network monitoring, whereas AWS provides the customer with the security solutions but places the responsibility of best practices implementation upon the customer. IBM is a smaller provider and supplies basic security processes such as key management and user passwords, but supply optional delegation of the more complex security issues to security partners.

———————————————————————————————————————————
References:

  1. (n.d.) IBM Cloud, IBM. https://www.ibm.com/cloud-computing/products/storage/object-storage/ (Last accessed: 31 May 2017)
  2. (n.d.) (July 2016) IBM Cloud Object Storage System features and benefits. https://public.dhe.ibm.com/common/ssi/ecm/ts/en/tss03183usen/TSS03183USEN.PDF (Last accessed: 31 May 2017)
  3. (n.d.)  Storage Insights: Slicestor Nodes, IBM: IBM Knowledge Center. https://www.ibm.com/support/knowledgecenter/SSQRB8/com.ibm.spectrum.si.doc/mgr_storagesystem_object_slicestor_nodes.html (Last accessed: 31 May 2017)
  4.  Graf, Brian. (17 May 2013). Scalability : Scale-up or Scale-out, What it is and Why You Should Care. https://www.brianjgraf.com/2013/05/17/scalability-scale-up-scale-out-care/ (Last accessed: 2 June 2017)
  5.  Hardiman, Nick. (2 April 2012). Service reliability: Understanding what it means and how to acheive it. http://www.techrepublic.com/blog/the-enterprise-cloud/service-reliability-understanding-what-it-means-and-how-to-achieve-it/ (Last accessed 2 June 2017)
  6.  Carlson, Lauren. (7 June 2011). The Downtime Dilemma: Reliability in the Cloud. http://blog.softwareadvice.com/articles/crm/reliability-in-the-cloud-1060611/ (Last accessed 2 June 2017)
  7. (n.d.) Load Balancing, IBM: IBM Cloud: Bluemix. https://www.ibm.com/cloud-computing/bluemix/load-balancing (Last accessed: 1 June 2017)
  8. (n.d.) Internet of Things, IBM: IBM Cloud: Bluemix. https://www.ibm.com/cloud-computing/bluemix/internet-of-things (Last accessed: 2 June 2017)
  9. (n.d) Choosing a storage option, Google Cloud Platform. https://cloud.google.com/storage-options/ (Last accessed: 3 June 2017)
  10. (n.d.) Cloud Storage with AWS, Amazon Web Services. https://aws.amazon.com/products/storage/?nc2=h_l3_db (Last accessed: 3 June 2017)
  11. (n.d.). Elastic Load Balancing, Amazon Web Services. https://aws.amazon.com/elasticloadbalancing/?nc2=h_l3_n (Last accessed: 3 June 2017)
  12. (n.d.) Load balancing, Google Cloud Platform: Compute Engine: Documentation. https://cloud.google.com/compute/docs/load-balancing/ (Last accessed: 3 June 2017)
  13. (n.d.) What are Containers?, Amazon Web Services. https://aws.amazon.com/containers/ (Last accessed: 3 June 2017)
  14. Shapland, Rob. (February 2016). Cloud Containers — What they are and how they work. http://searchcloudsecurity.techtarget.com/feature/Cloud-containers-what-they-are-and-how-they-work (Last accessed: 3 June 2017)
  15. Perlow, Jason. (21 April 2015) Containers: Fundamental to the cloud’s evolution. http://www.zdnet.com/article/containers-fundamental-to-the-evolution-of-the-cloud/ (Last accessed: 3 June 2017)
  16. (n.d.) Kubernetes. https://kubernetes.io (Last accessed: 3 June 2017)
  17. (n.d.) Container Engine, Google Cloud Platform. https://cloud.google.com/container-engine/ (Last accessed: 3 June 2017)
  18. (n.d.) Amazon EC2 Container Service, Amazon Web Services. https://aws.amazon.com/ecs/?nc2=h_l3_c (Last accessed: 3 June 2017)
  19. (n.d) IBM Cloud Object Storage: Public Services. IBM: Object Storage Public. http://www-03.ibm.com/software/products/en/object-storage-public/#othertab2 (Last accessed: 3 June 2017)
  20. (n.d.) Cloud Location: Google Cloud Platform. https://cloud.google.com/about/locations/#locations (last accessed: 3 June 2017)
  21. (n.d.). Google Cloud Storage Pricing, Google Cloud Platform. https://cloud.google.com/storage/pricing (Last accessed: 4 June 2017)
  22. (n.d.) Amazon S3 Pricing, Amazon Web Services. https://aws.amazon.com/s3/pricing/ (Last accessed: 4 June 2017)
  23. (n.d.) Load Balancing, IBM: Bluemix. https://www.ibm.com/cloud-computing/bluemix/load-balancing (Last accessed: 4 June 2017)
  24. (n,d,) Load Balancing and protocol forwarding, Google Cloud Platform. https://cloud.google.com/compute/pricing#lb (Last accessed: 4 June 2017)
  25. (n.d.) Classic Load Balancer Pricing, Amazon Web Services. https://aws.amazon.com/elasticloadbalancing/classicloadbalancer/pricing/ (Last accessed: 4 June 2017)
  26. (n.d.) Pricing and Quota, Google Cloud Platform. https://cloud.google.com/container-builder/pricing (Last accessed: 4 June 2017)
  27. (n.d.) Amazon EC2 Container Service Pricing, Amazon Web Services. https://aws.amazon.com/ecs/pricing/ (Last accessed: 4 June 2017)
  28. (n.d.) Compliance without complication, IBM Cloud: Bluemix. https://www.ibm.com/cloud-computing/bluemix/compliance (Last accessed: 4 June 2017)
  29. (n.d.) Secure your platform, IBM Cloud: Bluemix. https://www.ibm.com/cloud-computing/bluemix/security-privacy#privacy (Last accessed: 4 June 2017)
  30. (n.d.) Data Security and Privacy Principles for IBM Cloud Services, IBM. http://www-03.ibm.com/software/sla/sladb.nsf/pdf/7745WW2/$file/Z126-7745-WW-2_05-2017_en_US.pdf (Last accessed: 4 June 2017)
  31. (n.d.) Google Cloud Platform Security, Google Cloud Platform. https://cloud.google.com/security/compliance (Last accessed: 4 June 2017)
  32. (n.d.) Google Security Whitepaper, Google Cloud Platform. https://cloud.google.com/security/whitepaper (Last accessed: 4 June 2017)
  33. (n.d.) Cloud Key Management Service, Google Cloud Platform. https://cloud.google.com/kms/ (Last accessed: 4 June 2017)
  34. (n.d.) Google Cloud Platform Security, Google Cloud Platform. https://cloud.google.com/security/ (Last accessed: 4 June 2017)
  35. (May 2017) Amazon Web Services: Risk and Compliance, Amazon Web Services. https://d0.awsstatic.com/whitepapers/compliance/AWS_Risk_and_Compliance_Whitepaper.pdf (Last accessed: 5 June 2017)
  36. (n.d.) AWS Cloud Security, Amazon Web Services. https://aws.amazon.com/security/ (Last accessed: 5 June 2017)
  37. (November 2016) AWS Well-Architected Framework, Amazon Web Services.  https://d0.awsstatic.com/whitepapers/architecture/AWS_Well-Architected_Framework.pdf (Last accessed: 5 June 2017)

Budget Update: 04/06/2017

As it is the start of a new month, there will be two different items discussed in this blog. The first is my AD-DS budget for last month, and the second is my non-credit billing from the month.

Active Directory Budget.

A few days ago, I received an email indicating that one of my AD-DS alarms had been triggered:

AD DS Alarm State

This alarm is for my 10% forecast alarm, which indicates that my anticipated cost for this project was larger than the current expenditure trend. Despite this, I still looked into my budgets and billing information to determine where the expenses have occurred.

The AD DS budget is as follows:

AD DS Budget

I have provided multiple filters for this budget, so the billing information is important in understanding which service has influenced the budget forecast.

The two services that are being used are the EC2 service and the KMS service.
The EC2 Billing information is as follows:
EC2 Billing

The KMS service is as follows:
KMS Billing

As the KMS service is only being used in the North Virginia region, this is not part of the AD DS budget expenses. (This is elaborated upon in previous budget reports.)

The EC2 billing information can be processed into a spreadsheet, which more succinctly displays the costs involved for this project, and can be displayed graphically.
AWS Budget SpreadsheetAWS Budget Graph

The large spike involving the EBS-SSD provisioned storage is likely due recent activity on the AD DS cloud template, in which I was periodically creating and destroying the stack.

As of current, I am satisfied with the project’s budget progress due to its low expense despite having originally required the higher-priced t2.large instance in the stack’s creation.

 

Monthly non-credit billing
Due to it being a new month, my AWS account has sent a billing invoice for the month prior.
Billing Statement_LI

In following the available hyperlink, my billing information is brought up. It shows that these charges are from my budgets and tax.
Billing Statement Reason

Although I am unhappy with being charged, rather than the fee being removed from my available credits, I consider it worthwhile to keep in consistent knowledge of the expenses involved from each project.

Budget Update: 28/05/2017

This past week has also been spent on adjusting a CloudFormer template. As such, my current expectation for my billing list is that it will be quite low. The only expenses that should have occurred, are the AD-DS CloudFormation template (description link here) that I ran again, and the charge from the successful build of the CloudFormer template’s stack.

When looking at my billing list for my account, only two services have been used; EC2, and KMS. Upon further inspection, I have determined that the KMS charge is from an encryption key set in US-East region, which is not related to the AD-DS build.

The EC2 billing report is as follows:
EC2

 

I input this data, as well as last weeks data, in order to grasp a greater understanding of what charges have been incurred.

AD DS Budget Sheet

It appears that this past week has been less expensive than the one prior. This reduced charge is found within the EBS service specifics, which still contains the carry-over charges from the DinoStore volumes that I recently deleted. Hence, my first EBS budget isn’t exclusive to the AD-DS project charges and needs to be adjusted.

The adjusted AD-DS Budget Sheet is below, where the fee discrepancy has been resolved using the information from the ‘Budget Update: 14/05/2017‘, which contains my last reference to the DinoStore EBS volumes.

Budget Sheet Adjusted

I am unsure of the $0.19 difference in the General Purpose SSD (gp2) provisioned storage expense for the two dates, but may be able to verify whether this expense difference could be considered outlying with my next budget report.

 

Budget Update: 17/05/2017

For the past week I have been working with AWS CloudFormation to create and automate an Active Directory using scenario 3 from the following AWS manual.

This scenario involves the creation of stacks that utilize Amazon services from their enterprise level, which results in this being a potentially very expensive endeavor.

After running the stacks for a few hours, my billings showed the following EC2 charges:
EC2 Billing

My conclusion from this, was that I needed to create a budget alarm to ensure that I remain aware of the involved expenditure for this project.

AD DS Alarm

The current limit for the budget is only $10 because my intention is not to try and stay within this budget, but to keep it small enough that I remain conscious of the expenditure. The purpose for this is to provide myself with minimal cost involvement in comprehending the cost value associated with the stack.

Budget Update: 14/05/2017

At close on midnight last night, I received an AWS notification email, telling me that my EC2 budget had passed one of its limits. Since my last budget post, my activity with AWS has been within the CloudFormation services, which is free-tier.
Budget Alarm

 

To understand why I was being charged, I checked upon billing info. The results showed that I was still being billed for EBS services.
14_05_17 Bill Report

 

My conclusion was that I had not completely deleted my EBS volumes during my last attempt. In checking the EC2 service, I was proven right.
EBS Volumes

 

This time, I made sure to delete my EBS volumes, and stop the unnecessary credit charge.
Deleting EBS Volumes

 

My next budget update should only show purposeful charge, which I have initiated.