AWS Certified

I am finally AWS certified developer now. I took the Developer Certification – Associate Exam at end of Oct.

In the following post, I may post series of tutorial post talking about AWS training & exams.

In the mean time, I will continue working on AWS Certified Solutions Architect.

Posted in AWS

Learning notes on Terraform and Terragrunt

Terraform/Terragrunt is a powerful tool to provision cloud resources. This post is to talk about some key feature and few tricks about it. Firstly, to describe them in quote,

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules

Both of them are powerful tools and I am supposed to do this post long time ago. Now finally motivated enough to pull together a few bits and pieces I have learned along the way by using terraform in practice. Here is the list.

  1. Remote State & Locking

    This can’t be emphasized enough. Remote state enables trackable resource management and alleviate teamwork conflicts. And most importantly, this feature has been extended by terragrunt so that we can do inherited structure of config setting, which means Don’t-Repeat-Yourself. Take below for example,

    This remote_config setting will support all other backends types by terraform as well.

    And when this file is included when running terragrunt, all belowing terraform settings will be updated.

     

  2. Modularity is king of re-usable code

    This is my favourite feature of terraform. Creating modules means the code can break into multiple levels of granularity. And each level of complexity VS repentance can be easily managed at no cost (except for calling init again for redeclaring the modules before calling plan). A simple setup would looks like below,

    (Note: the double slash (//) is intentional and required. It’s part of Terraform’s Git syntax for module sources. Terraform may display a “Terraform initialized in an empty directory” warning, but you can safely ignore it.)

    This is only the starting point. Some further thinking may be,

    1. Different modules could have different dependencies. Since terraform doesn’t support module level dependency. It could be done by terragrunt.

       
    2. Modules managed by terragrunt requires a lot of tfvars file. Maybe for simpler project managing one module in terraform which gathers all the pieces for other modules will be easier, which essentially is creating one master module on top of other modules. It gives better experience when transfering this project to pure terraform. And it much clearer in the way that tfvars files are defined.
  3. Loop, if-else syntax with magic “count”
    Essentially its terraform Interpolation of variables. Terraform can support both list and map types. Below are some most inspiring examples I have known so far.

    However, there is a BIG catch when using the count. DON’T DEPENDS COMPUTED RESOURCES. Since count is sent before dynamically computing any resources. So at the stage count is calculated, computed resource/numbers won’t be availabe. So you will get below error,

     

  4. Connecting gap between terraform & terragrunt
    1. Including Sequence

      Child block when including will always override the parent block settings if the setting has the same name. However, if the setting has a different name, the two sets will be merged together. And if in this case, the child setting takes higher priority. Two useful functions to mention here,

      find_in_parent_folder() : This function returns the path to the first terraform.tfvars file it finds in the parent folder above current terraform.tfvars file.

      path_relative_to_include() : This function returns the relative path between the current terraform.tfvars file and the path specified in its include block. It is typically used in root terraform.tfvars file so each child module store its state file at different key.

    2. Readable Global VARs

      Some environment variable is visible to both terragrunt and terraform. For example export TERRAGRUNT_IAM_ROLE="arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME" is equivalent to call terragrant with option --terragrunt-iam-role "arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME". Further example could be,

      TF_VAR_name

      TF_CLI_ARGS

      TF_INPUT

In summary, using terraform is so much a better experience compared to other alternatives. No never-ending JSON template file and dodgy (at least IMO) DSL syntax.

Here for FYI useful links,

https://github.com/gruntwork-io/terragrunt

https://www.terraform.io/docs/providers/aws/index.html

https://www.terraform.io/docs/configuration/interpolation.html

https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9

IAM Role learning summary

IAM Role provides the flexibility to applications which can securely make API requests from role-profiled instances without manually creating and distributing AWS credentials.

From the FAQ of IAM Role, we can see the clear difference between an IAM User and IAM Role,

An IAM user has permanent long-term credentials and is used to directly interact with AWS services. An IAM role does not have any credentials and cannot make direct requests to AWS services. IAM roles are meant to be assumed by authorized entities, such as IAM users, applications, or an AWS service such as EC2.

There are 2 points on the emphasis.

  1. IAM roles cannot make direct requests to AWS services
  2. IAM roles can be assumed by 3 different kinds,
    • IAM Users
    • applications
    • AWS EC2 (or ASG)

How to understand “IAM roles cannot make direct requests to AWS services

IMHO, IAM users who has the credential and associated access can direct make request to AWS service using public APIs. (e.g. awscli or boto3).
But for IAM roles, it has to fetch for the temporary credentials first based on the role. Then from there, the autobot or application could take use of the public api as normal user.


A typical use case for IAM role

  1. Create an IAM role.
  2. Define which accounts or AWS services can assume the role.
  3. Define which API actions and resources the application can use after assuming the role.
  4. Specify the role when you launch your instances.
  5. Have the application retrieve a set of temporary credentials and use them.

As illustrated below,


IAM role essential usage thinking

Its application focused no matter assumed by either users or EC2. Roles exists for the only purpose of serving a particular application’s need.
For example,
A role created specifically for EC2 related process:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*"
            ],
            "Resource": "*"
        },
    ]
}

Then this role is granted to the auto-bot agent as well as the an aws actual user so that both this user and bot can do the same ec2 instance related operations.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::xxxx:user/hans.wang",
          "arn:aws:iam::xxxx:role/application-running-agent"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

My static photo album hosted on S3

Hooray!

I just created my static photo album photo.hanswang.info

It is using Amazon Web Service S3, serving all content as static page.

Since its whole purpose is an photo album for family, S3 server is perfect for serving this content as CDN.

See the official reference from AWS S3 develop Guide.

Here is how I did it.

  1. Create a s3 bucket named “photo.hanswang.info”, and make it public. By setting the policy, and endpoint url. Detail see AWS S3 doc ref
  2. Change DNS record for photo.hanswang.info to be CNAME to bucket public url
  3. Build the index.html inside the bucket root dir, and all other associated resources, such as sprited icons, aggregated css, and most importantly the relative urls for photos
  4. Deploy and integrate the index.html page with jQuery Masonry. Done and Enjoy!