3. Amazon Web Services

The Services tab will allow you to search and sort services that you see on the home page. AWS also provides Additional Resources to help you get started using AWS, and a Service Health check on the right panel that lets you see the health of your services right on login. The bell in the top banner provides you with alerts given by AWS. These alerts give you any open issues, any future scheduled changes, and any other notifications.

Under your username, you can access your account, your organization, billing dashboard, organization, and sign out. Figure 3-2 shows these options and where to access them. We will look into the billing dashboard more in the Pricing section of this chapter. AWS Organizations enables you to apply policy-based controls centrally across multiple accounts in the AWS Cloud. You can consolidate all your AWS accounts into an organization, and arrange all AWS accounts into distinct organizational units.

The region to the right of your username allows you to select the Amazon Region you are working in. The Region Names are as follows:

The region that you select becomes your default region in the console. Some services are available in regions that others aren’t. So if you are navigating the console and don’t see the resources you are looking for, try selecting the region the resource was created in. US-East is the default region, so I would check there first.

AWS also provides a Support service that offers Support Center, Forums, Documentation, Training, and Other Resources. These are good for learning new services and getting hands-on experience with new services. The documentation provides a Getting Started guide with examples, SDKs and Tools, resources, and examples for AWS services. It is extremely helpful when getting started with new services.

In Chapter 1, Table 1-1 showed Lambda pricing for AWS in comparison to Azure and Google Cloud, but now you will see how to navigate the Billing Management Console under your account to help manage your costs. Under your username, select Billing and Cost Management Dashboard. This will take you to a page that looks similar to the one shown in Figure 3-3.

AWS gives you a lot of accessibility with managing costs of services, payment methods, generating reports, and viewing your bills. All of these capabilities are available through the Billing dashboard on the left panel. The Billing option lets you sort through your AWS bills by month. It provides a history of these bills as well as a CSV export option. Figure 3-4 demonstrates this capability.

Another important billing capability is Consolidated Billing, which allows you to handle multiple accounts under one master account. It works with AWS Organizations to create multiple organizations, organize accounts in these organizations, and apply policies to these organizations. A good use case for this is a large company with multiple projects. Instead of using resource groups and tags, you can keep your AWS applications and resources completely separated from one another with Organizations and Consolidated Billing.

Another solution is to set up billing alerts within the billing console. Billing alerts will send you email notifications when your account hits a certain dollar amount or resource amount set by you. I made the mistake of thinking all services in the free tier were free and was hit with a pretty hefty bill after spinning up multiple RDS instances and EC2 instances. Since then, I have set my billing alerts to notify me when I go over $1.

The Lambda console can be accessed from the Services tab across the top, under Computing. The Lambda dashboard lets you view the number of functions, the total code storage, account-level metrics over the past 24 hours, and what’s new within Lambda. Figure 3-5 gives you an overview of a Lambda dashboard.

The Functions section is where you go to access and create Lambda functions. Figure 3-6 shows the Functions UI.

We will explore the Lambda portal and all of its capabilities in more detail as we begin creating our Lambda functions. However, before we can create our functions we will need to configure our IAM (Identity and Access Management). It is important to note that setting up IAM services is not required for using functions, but we are following best practices and it is in your best interest to go ahead and follow the next couple of steps. We will dive into more advanced examples later that will require the use of IAM to use different services.

The IAM console is found under Services and Security, Identity, and Compliance. The console gives you a dashboard, groups, users, roles, policies, identity providers, account settings, a credential report, and encryption keys. From the dashboard (Figure 3-7), you can access a user’s sign-in link. This is where users who are not admin users are directed to log in. It also gives you an overview of your IAM Resources, including the number of users, groups, customer-managed policies, roles, and identity providers.

To begin, it is important to complete the five steps listed in the Security Status console: deleting your root access keys, activating MFA (multi-factor authentication) on your root account, creating individual IAM users, creating groups for permissions, and applying an IAM password policy. By following these steps, you are ensuring that your IAM settings are properly secured so you can begin creating users and roles.

Roles, Policies, and Users are your means to set permissions to people, services, and resources. Roles are created under the Roles tab and allow you to create roles with set policies. These roles can be assigned to users and services. For instance, if I have a group of developers who I want to be able to edit and access Lambda and services, but not root account information, I can create a role called Developers.

After the role is created, I can assign certain policies to it. Policies determine the amount of access a role has to a service. Figure 3-8 demonstrates the Policy console with all of the preconfigured policies. You also have the option to create your own.

Policies describe the amount of access allowed to a particular service. For instance, the AdminstratorAccess policy gives you full access to all resources for all services.

The Users window lets you add users to AWS. They are given their own login and whatever roles and policies you attach to them. To access the console, users are given Access Keys and a password that are downloaded by CSV or sent directly to them. You determine the amount of time they have with the default password and all of the password policies regarding their user login.

You also have the ability to add users to groups. Groups can be used to make permissions easier. If you have a group of users you want to all have admin access, you can add them to a group so all of the group policies are applied across the board. For the purpose of our serverless applications, we won’t be assigning users or groups, but it is good to keep these opportunities in mind as you build bigger applications with a larger group of people.

AWS requires you to assign a role to your Lambda functions . These roles can differ across Lambda functions as they require access to different AWS services. However, just to get started with our Hello World function, we are going to create an AWS Lambda role that can be assigned to our functions.

In the Roles tab, we will click the Create New Role option. We will name our role “lambda_basic_execution.” Under permissions, we will attach the AWSLambdaExecute policy. If you look into this policy, you can see the exact permissions attached to it. The policy allows full access to CloudWatch to log our function, and provides read/write access to AWS S3. Figure 3-9 shows what the role should look like after creating it.

The Role ARN at the top of the console is the Amazon Resource Name. This is what uniquely identifies the role we just created. When we create our first function, we will assign our Lambda to this role, giving it all the permissions specified within the one attached policy.

We will start by creating a new function in the Lambda console. After clicking Create a Lambda Function, you will see a list of blueprint options (Figure 3-10). These blueprints give you a Lambda skeleton that you can edit to complete the functionality you are looking for. To start off, we are just going to select a blank function.

After selecting a blank function, we next have to configure our function. This requires assigning it a name, description, runtime, and a handler and role. Names do not have to be unique universally, just within your functions. I named this hello-world and gave it a description and a runtime of Node.js 6.10. AWS also allows you to either edit your code inline or upload a zip. Since this function is going to be simple and small, we can edit it inline. Figure 3-11 shows what your configuration should look like.

The exports.handler defines this as the function handler. It takes in an event (trigger), context, and callback, which is what we will use to signify that the function is finished executing. Our callback is currently responding “Hello from Lambda.” Next we need to configure our handler and role. We will leave Environment Variables, Tags, and Advanced Settings blank for now, but feel free to look them over. Our handler is index.handler, and our role is the lambda_basic_execution role that we created previously. Once this is configured (Figure 3-12), we can move on and create our function.

Next, we will look at testing and executing our function. For now, we won’t set up a trigger, because that requires trigger configuration. We just want to see what an executing Lambda looks like, how we can create one, and how we can access the logs.

To test our Lambda function in the console, we can use the Configure Test Event action. Inside the function, if you click Actions, you will see a list of actions you can take on your Lambda:

We will configure a test event . In the input test event, AWS provides several event templates. Feel free to explore these to see what different incoming events look like. We will use the Hello World event, as shown in Figure 3-13. This event just offers a JSON of various keys and variables.

Since our Lambda is not configured to do anything specific with the event, we should be able to get our Hello World response from our test event just by triggering it. The test event works the same way a trigger would, causing the Lambda function to execute and respond to the incoming event. You are given options to Save and to Save and Test. With the Save button, the function is not run. Save and Test saves your function and tests it using the provided test case. This is event-driven architecture in action. Figure 3-14 shows the Execution result , Summary, and Log Output.

The execution result demonstrates what we specified our Lambda to do. Since we only specified a callback with a string, that is what we are receiving. The summary shows you the duration of the execution, billed duration, and amount of resources configured. This is important for further configuration of your function. If your Lambda is using only so much memory , that would be a good thing to adjust in its configuration to limit unnecessary space and charges.

We also see the output of the log. Even without explicit logging, we were given the Start of the execution with the request ID and version being run, the End of the request, and a final report. The following demonstrates the code we are running for our Hello World function:

Now that we have shown the function executing, let’s change it to repeat back the event coming in. Figure 3-15 shows our updated Lambda function.

To improve on this Hello World function, try parsing the event and responding to parts of it. Also include inline logging that you can look at in the log output and in CloudWatch .

Now, we will examine CloudWatch and see where our logging happens and all of the metrics we get out-of-the-box on our function. In the log output section, navigate to the Click Here option. This will take us to the CloudWatch portal, where our function’s logs live. Figure 3-16 shows the two executions I made earlier in this exercise.

These logs are helpful for analyzing and monitoring your function’s executions. In addition to CloudWatch, there is a Monitoring tab in the Lambda function’s console that gives you a high-level overview of your function’s executions and outputs. To look deeper into a log, click on its stream. This will give you a full detailed log for that execution. Figure 3-17 shows the executions on my Hello World function.

Now that we have become acclimated to the Lambda console, test events, and CloudWatch, we will build upon our Hello World function with environment variables.

The code for the Hello World function can be found at https://github.com/mgstigler/hello-world.git .

Environment variables are those set globally across your serverless application. They are given a key and a value . The value is that of the actual variable, while the key is the name used for that variable throughout the application. The benefit of environment variables is both security and ease of use.

Rather than leaving API keys and various access information scattered throughout your code, you can assign the actual secure variable to an environment variable to be used anonymously. In addition, if you know you are going to be using a variable repeatedly, setting it as an environment variable allows you to access it across your project without re-declaring it. It also allows you to make quick changes in one spot.

To see how environment variables are used, we are going to implement them in our Hello World application.

We are going to create an environment variable with the key provider and value AWS. This also demonstrates a best practice of separating provider logic from your code to prevent vendor lock-in. While for this example we are just using the value AWS, later it could be used to represent different services. For instance, if we knew we wanted to access a database, we could use the key DB_Host and set the value specific to the AWS database hostname. This makes it easily configurable if we choose to move to a different cloud provider. Figure 3-18 shows where and how we have configured our environment variables.

Now we can access this environment variable within our code. Figure 3-19 demonstrates how we reference environment variables and the log output for the execution of the Lambda function.

This shows how easy it is to create and access variables in your code. Now that we have completed a Hello World demonstration of Lambda, we will look at creating a new application that uses an HTTP event and responds to it by returning data from DynamoDB.

API Gateway is an AWS service that lets you easily create and access an API all through the API Gateway console. It gives you a public RESTful API interface to a wide host of AWS services. This allows you to interact easily with your databases, messaging services, and Lambda functions through a secure gateway. In addition, it is incredibly easy to set up and create endpoints for. This allows the continuation of rapid development. To begin using API Gateway, navigate to the API Gateway service under Application Services. The API Gateway console shows your APIs, usage plans, API Keys, Custom Domain Names, Client Certificates, and Settings. Figure 3-20 shows an example of the API Gateway Console.

To create an API, simply click Create API. This will take you through the process of setting up your own API. We will create a New API and name it Recipe API. After creating your API, you should be directed to a console that allows you to configure your API (Figure 3-21) . We want to add a resource to our API called “recipes.”

Resources allow you to have multiple objects for your methods. We also want a single GET method attached to our resource. Figure 3-22 shows what your API console should look like after configuring your endpoint.

We then need to configure an integration for our GET method. You can have Lambda create an API Gateway endpoint for you, but doing it this way gives you control over what the endpoint looks like and its configuration. For this, we want to select the Lambda function , specify the region, and select our GetRecipes Lambda function. This tells the API gateway when the GET method is accessed, to execute the Lambda function. Figure 3-23 shows the complete integration of the Lambda function.

Before going back to the Lambda console to set this API as our trigger, we need to deploy our API to a stage. Staging is important for version control and deployment. Since we are still in development, we are going to name our stage “Beta.” Under Actions, next to Resources, click Deploy. The pop-up shown in Figure 3-24 will ask you to create a stage if you haven’t already. Go ahead and create a new stage and name it “beta.” Stages represent a unique identifier for a version of a deployed REST API that is callable by different users and services .

After deploying, you will be directed to the Stages tab (Figure 3-25). This is where you configure settings for your stage, stage variables, SDK Generation, Swagger exports, Postman extensions, deployment history, and documentation history. Take a look at each of these tabs to see everything that is available to you through API Gateway.

At this point, we have an API Gateway with one resource and one method, deployed to a Beta stage, and integrated with our Lambda function. We are ready to set this API as our trigger for our Lambda function and begin responding to requests.

Back in the Lambda function, GetRecipes, we can now configure the API we just created as our trigger. Under Triggers in our function, click Add Trigger. Here, we select API Gateway from the drop-down menu and specify the API we just created. This will tell our Lambda function to wake up to events coming in from this specific service. Figure 3-26 demonstrates the correct configuration for our function.

Leaving the Lambda code as is, we can go into API Gateway and test our trigger. By clicking on the GET resource, we can select Test and test our API (Figure 3-27). Since it is a GET method, it requires no request body. On the right in API Gateway, you should see a response, status, and latency. You could also view this in the browser by just hitting the GET endpoint. The URI for this example is

https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:174208833299:function:GetRecipes/incovations

Now that our trigger is configured, we can develop our Lambda function to respond to this request by returning recipes from a DynamoDB table. To do this, we will first create a DynamoDB table, Recipes, and prepopulate it with some recipes. Use the Services tab to navigate to the DynamoDB service. Here we will click Create Table (Figure 3-28) and name our Table “Recipes” and assign it a Primary Key of “Id” of type “Number.” The primary keys should be unique to items. The DynamoDB table consists of items and attributes. In our table, each item is a recipe which has attributes of meal, description, and prep-time.

From here, we will create a couple of recipe items to populate our table. To do this, click Create Item and append fields Meal, Description, and PrepTime with various values (Figure 3-29).

When our table has been populated with a couple of recipes, we can build our Lambda function to return these recipes in the response to the API (Figure 3-30).

To create the Lambda portion of our serverless application, we will be using TypeScript, Node, and NPM. The next section will go over how to accomplish this.

For our Lambda function, we will be developing within Visual Studio Code, using TypeScript, Node, NPM, and the AWS SDK. First, it is important to format our project structure so we can easily zip and deploy our function. Figure 3-31 shows the structure I have chosen to create this Lambda function. Within our AWS project, I created an HTTPTrigger project with a Shared folder and a GetRecipes folder. The GetRecipes folder will hold our handler.js file, which will be triggered by the GET request. The Shared folder contains a Recipes model that defines the structure of the incoming request.

For now, we will create the function without using the Serverless Framework, so outside of the HTTPTrigger project, we will need a package.json file to specify what NPM installs, and the tsconfig.json file to configure our TypeScript builds. Your package.json file should include aws-sdk and typescript as dependencies.

The tsconfig.json should be configured to build on save and to compile on save. This will compile the JavaScript files for TypeScript files as you save. Listing 3-2 shows the tsconfig.json file.

We can now do an NPM install on our project to install all of the node modules we will need to create our Lambda function. This will create a node_modules folder in your project with all of its dependencies. We will also create a recipeModel.ts file (Listing 3-3) in the Shared folder. This model will define the structure of the recipes we created in our DynamoDB table. We can then use this in our handler.js file to format our response to the GET request. In the future, with other requests, you can use this model to format the request.

In our handler.ts file, we will create our GetRecipes module that will take in an event, context, and callback (as we have done in our Hello World example) and will utilize aws-sdk to communicate with our DynamoDB Table and respond back to our request with a list of recipes. Listing 3-4 demonstrates this handler function, followed by the steps that will let us go into further detail.

We can now compile our TypeScript files into JavaScript files. Once we have created the handler, model, node modules, and compiled our TypeScript, we can compress and upload our application. It is important to remember that the handler.js file must remain at the root of your compressed files. The compression must occur at the level shown in Figure 3-32.

After uploading the zip file to our GetRecipes Lambda, there are still a couple of configurations to take care of. First, we need to update our handler. Our handler should now be listed as handler.GetRecipes. This is because the module we are exporting is called GetRecipes and it is found in the handler.js file. We should also add our environment variable TABLE_NAME with its proper value. We also need to add a policy that gives us access to DynamoDB to our Lambda role. This can be done in AWS IAM under Roles. Finally, we can test our Lambda function using Postman and the URL given to us in the API Staging console. Figure 3-33 demonstrates a Postman request and the response we get back.

We now have a start-to-finish serverless application using API Gateway, Lambda, and DynamoDB.

In the next section, we will use the skills and tools we learned with the HTTP Trigger to create a separate Lambda function triggered by a storage event.

AWS offers many storage options ranging from Relational Databases, to NoSQL Databases, to Blob storage. In this exercise, we are going to explore using AWS S3, blob storage, as a trigger for a Lambda function. Within the S3 service, let’s create a bucket. Your current S3 console will look something like Figure 3-34.

As shown in Figure 3-35, I’m naming mine recipe-images-ms. Bucket names are universally unique so you will have to make sure your bucket name has not been used before. By “universally,” I mean across all AWS regions and accounts. Within the S3 settings while configuring your bucket, you are presented with options such as Versioning, Logging, and Tags. We will leave the default values for now. The only thing we will change is the public permissions. We will make the objects in this bucket open to the public so we can access these recipe images from the web.

After our bucket and its respective images are configured, we can move on to setting S3 as our trigger and creating our Lambda function.

From the Lambda console, create a Lambda function called UpdateRecipe . This Lambda function will receive events from S3 as an object is uploaded (PUT). It will then update the corresponding object’s recipe with an image URL. For simplicity, we will name our image uploads with the key to their corresponding recipe. For example, the recipe for Mac & Cheese has an Id of 2. To associate an image URL with that recipe, we will upload a file named 2.

Within the Lambda console, configure your trigger to be S3, and select the bucket that you created previously. Your trigger will end up looking like Figure 3-36.

Now that an S3 event is set to trigger our Lambda, we can format our function to handle the event the way we would like it to. The first step is understanding the event request that is coming in. To simulate this request, I used the Set Test Event blueprint for the S3 PUT operation. The following JSON is what it provides:

We can format our Lambda to handle this request by creating a TypeScript model for the S3 PUT request. We want to specify the object key so we can parse it and grab the associated DynamoDB item in order to place the image URL with the correct recipe. The next section will cover how our Lambda function handles this request.

The following steps summarize this process:

To test the success of our function after zipping it with the Shared folder and the node modules, we can upload an image to our S3 bucket. I found a picture of mac & cheese, one of my recipes, and uploaded it to the bucket. To upload to the bucket, simply go to the bucket location and click Upload. The prompt is shown in Figure 3-37.

Figure 3-38 shows the log results from the upload.

We can also test the upload by looking directly in the DynamoDB table. As shown in Figure 3-39, the ImageUrl is now added as an attribute with the provided image URL.

Furthermore, when you open the provided image URL, you are redirected to an image of the recipe (Figure 3-40).

As before, there are many ways to improve on this Lambda function. Separation of provider logic is strongly encouraged as well as deploying with the Serverless Framework. The code for this portion of the exercise can be found at https://github.com/mgstigler/Serverless/tree/master/AWS/aws-service/StorageTrigger