Budget Update: 24/04/2017

My expectation for this week’s AWS bill is that it would be high because I have been using services such as RDS and EC2 for the DinoStore Project.

When I looked online tonight, my bills page showed this number:

This is an increase of $7.24 since I last checked my budget.

I did receive a message from one of budgets earlier in the week, on the 19th of April. I decided that as I was still in the process of completing labs 4, 5, and 6 of the DinoStore project, that I would write up the blog once I had completed the labs. My intent in this, was to view the overall expenditure for these RDS based labs.

As the previous budget was now set too low, I changed the budget amount from $1 USD (credit) to $5 USD (credit). Because I only adjusted the budget rather than creating a new one, all of the previously logged expenses remain. In the updated budget this put the forecast percentage to 55%. This required me to create alarms for forecasts above 55%.

Percentage forecast alarm creation for budget

In order to clarify where the $11.41 of credits have been billed, I looked through the full billing information of each AWS service.

Below are billing snapshots of the three services that caused expense over the last week:

This slideshow requires JavaScript.

I compiled the billing information into my spreadsheet so that I could compare the cost amount for each service.

As expected, most of my costs can be seen to be from the RDS service. What I did not initially expect is that most of the cost would be from the ‘$0.017 per RDS db.t2.micro instance hour running MySQL’ specific service which corresponds to the MySQL CE RDS in the North Virginia region. However, this does start to make sense when I consider that although my Sydney based RDS server is free, my backup to the North Virginia region is not. Hence, whenever I’ve had my instances running this past week, both regions have been charged for every hour or partial hour of use, which has caused the high expense from the North Virginia region.

I created another spreadsheet, this time looking at the percentage increase of cost per week (in relation to the week prior). Extrapolating this information from my billing spreadsheet provides me with the same information, but from a different perspective, enabling me further understand the financial investment required for each service and its charged attributes.

 

Considering this from a business financial perspective, the question arises, is having the North Virginia backup worth its expense?
During the restoration of my RDS instance snapshots, I ensure that they are not multi-AZ deployment enabled, as this would cause a charge from the RDS Service Storage attribute.
In terms of financial capability; In my scenario, where I have $100 USD credit for free, and this project is not for long term, the North Virginia expense is not too problematic, even though it could be considered as needless expenditure. Considering expense value leads me to another question, is this North Virginia backup a setting that can be changed within my RDS instances, or is it locked into the instance properties and can’t be adjusted?

It appears that both questions have a fundamentally wrong assumption. When I created the RDS instance quite a few weeks ago, I created a read-replica of one of my instances and placed it in a different availability zone -North Virginia to be precise. My understanding was flawed, in that I thought that my instances were constantly being backed up into the North Virginia instance whenever I restored them from their snapshots. This understanding was backed up with the results that I found within my last budget report. Tonight, after being unable to found anything relating to North Virginia in my two instances, I changed the region on my AWS site, and looked at my RDS service for the North Virginia region. What I found was a running instance.

So in answer to my first question: this North Virginia read-replica appears to a needless expense, and a pricey lesson in keeping track of everything in AWS, especially that which is organized into different regions.

As far as whether storing replicas or other services within different regions is worthwhile; that depends entirely on the business goals, financial priorities, and cloud server influence for each individual business.

Resolving Bills in AWS

Over the past week, I have been keeping a close eye on my AWS billing log so as to determine which services have charging me without my knowledge.

On the 14/4/2017, I received an EC2 alarm that I was able to determine that it came from the EBS service. To my knowledge, that was the cause of credit expenditure, so I shut it down.

However, I received another alarm notification on the 17/4/2017, this time in regards to my RDS budget.

This concerned me, as I had removed my instances from being Multi-AZ enabled, which meant that I was running them from the free tier. To gain more understanding, I looked through the billing specifics for the RDS service.

What I determined, was that my instances were being charged under the North Virginia region. Due to my assumption that they were running completely within the free-tier conditions, I was not using best practices, and was keeping my instances running while not in use. I had not considered that my decision to have them backed up in another region would invoke a fee.

My next move, was to snapshot and delete my main instance. I chose not to delete the replica immediately, as in my previous experience with backing up and restoring instances, I had found the replica to be more difficult to re-establish during the restoration process.

I checked my bills the next morning to find that the running replica instance was still charging my account. I then proceeded to take a snapshot of the instance, then delete it. My concern for unnecessary charges took precedence over the increased difficulty for restoration. The following screenshot shows the billing amount for the RDS service in North Virginia.

Billing amount at 09:33am 18/04/2017

I then took another screenshot later in the day to check whether there was any increase in the billing amount.

Billing amount at 11:45am 18/04/2017

The lack of increase confirmed that it was the two running instances that were causing my credit to be charged.

Just in case there were any other expenditures that were increasing my bill, I also checked the overall billing amount during the course of the day, Once at 11:45am and once at 01:25pm. I saw no increase in the overall budget. This meant that I had managed to resolve all of the unnecessary expenditures.

Budget Report 14/04/2017

I received another billing alarm last night, this time with respect to my EC2 budget.

Upon further inspection, the source of expenditure was from AWS’ EBS service. This seemed strange to me as I haven’t used EBS during this DinoStore project.

In order to gain a greater understanding of my expenditure, I compiled my AWS bills into a spreadsheet.

EBS charge for April has been $0.64

I originally thought that it was due to the MySQL RDS instances that I created for Lab 2, and considered that it may relate to the snapshots that I had created. The dates however, for the creation of the EBS volumes were all in March, and my RDS snapshots were created in April. Upon looking into greater detail of the EBS volumes, I was able to determine that they related to previous RDS instances that I had created for my QwikLabs project.

By looking into AWS’ documentation on EBS volumes, I was able to determine their use and their cost:

Device use:

Source Link

Device Cost

Source Link

My next step was to detach the EBS volumes in order to cease any further charge.

As for my DinoStore budgeting; I have been using Free-Tier available services and so will not have incurred any other charges.

Budget Report: 09/04/2017

Having been working on the DinoStore lab project for a little over a week now, it is time for me to check how I am progressing with my S3 budget.

As it turns out, the Mulit-AZ deployment option for the RDS created is quite expensive to run for long periods of time. As I am still new to using any cloud service at this point in time, so when I created the RDS instance, I was unsure of how to close it without losing all my data. As such, I left it running overnight, which cost me for my lack of knowledge and lack of sensibility to research how to efficiently use an RDS instance.

RDS cost after 24 hours

After realizing my mistake, I figured out what a snapshot was, and how to use it in saving RDS based expenses. The use of the snapshot caused further problems for me, which will be elaborated in DinoStore: Lab 2 blog, and resulted in quite a few more hours of running the costing RDS instance with no progress to show for it.

RDS Billing on 08/04/17 at 08:34pm

In order to save short-term future costs, I have created a new RDS that runs on the free tier available for AWS. This will save me credits as I learn how to better use the RDS instance so that if I find myself needing to use the Multi-AZ one later in the project, I should have the experience to be able to set it up without causing unnecessary cost to myself.

I have also set up a new budget that focuses solely on any credit expenditure for RDS. This is because I spent at least 200% of the S3 budget, and that expense was only from the RDS service.

Email Notification for S3 Budget

RDS Budget for DinoStore

DinoStore Budget

In regards to the new project that I have set before me, I have decided to create a budget on AWS that will cover the expenditures that I am currently aware will occur during the course of this project.

As most of this project involves storing data in the cloud, I have labelled the budget as S3 Budget. My current knowledge of foreseeable expenses involve the following services: RDS, S3, and AmazonCloudWatch. Hence, they are part of the related costs for this budget.

S3 Budget

 

I will be updating the budget, and the blog, as I progress with this project.

Final Budget Report for QwikLABS

Having now completed the QwikLabs assignment, I’ve considered it beneficial to determine where all my expenses occurred.

QwikLabs expenditure

The object of foremost concern, is that my largest expenditure is from an attribute in which I cannot confidently associate the cost.

However, If I look over the previous budget logs I can see that the same service has been billed prior. This means that the large expenditure seen here for the KMS is the culmination of accumulated fees.

This appeases my concerns, but is still an area that I will keep in mind. Hence, I’ve created another budget, this one focusing solely on the KMS service. When the forecasted amount exceeds $1.00, so is doubled from now, I will receive a notification informing me of this. This should assist with any short-term future activities that may involve the KMS service.

KMS Budget Plan

AWS Budgeting: Revisited

On 16 March 2017, I received an email notification from Amazon Web Services.
The email contents looked like this:

This email highlighted two things for me. Firstly, that the budget scheme applies to my available credits, and secondly, the notification email doesn’t provide me with any information of which service has caused the expense.

Upon opening my my AWS Budgets dashboard, and scouring the service for further information, I located the source of my expenditure.

The QwikLab activity on keys is the reason I’ve been charged. However, this detailed charge list shows that I’ve only spent $0.03USD, a very minimal amount. In looking at how I organized the Monthly Credit Alarm, I had structured it so that it would alarm if the forecasted value exceed 10% of my budgeted value. As my budgeted value was set at $0.10USD, my expenditure equated to 30% of it, causing the alarm to activate.

In consideration of this new information, I have concluded that with the knowledge of the AWS Budget scheme accounting for credit usage, I should now readjust my budget alarms for more realistic expenditure conditions.

AWS Account: Budgeting

This course involves a student account with AWS. This package provides $100 USD worth of credits, as well as access to the Free Tier. Whilst most of what we do in the course will be available on the free tier, some of it won’t. Now, depending on how I budget and set expenditure alarms may be the difference between having over $95 USD at the end of the course, or having needed to spend my own money.

Although this class implies that I’m still only a student playing around and familiarizing myself with the service, I want to consider the budgeting aspect from a business viewpoint. By this, I mean that I want to forecast weekly expenditure based upon the previous week and my knowledge of what AWS services are required for the current week.

As of current, I haven’t needed to use any of the services provided by AWS, so I do not have any information to base my forecasts upon. With this in mind, I’ve created multiple alarms that will alert me of any expenditure over $0.001 (USD).

This should, by my understanding, alert me of any expenditure, which I can then use to make any appropriate changes to the current alarms or to create new ones.
However, as this is my first time using AWS, I’m not completely sure as to whether the threshold set in the alarms will respond to the expenditure of the credits, or whether it will respond once all the credits have been spent.

In an attempt to insure myself against such implications, I have also created a budget using the AWS tool available.

The budget is set to $1.00 (USD), with alerts set for when it the forecast costs for the month exceeds 1% of the budget, and for when the actual costs exceed 1% of the budget.

Utilizing both tools should help ensure that I remain in complete knowledge over my finances for my AWS account. I will however, on occasion, manually check my credit as this will enable me to gain a greater understanding on how AWS interprets the credit usage. I will then adjust my alerts and budgets accordingly.