Coming out from the from the latest AWS builders day (12 December 2018 Dublin). My second time at the same event I decided to concentrate on one topic: Serverless. I so followed the full track which included various topics. Here I attempt a recap of Danilo’s first session: “Serverless: the state of the union”. Here I linked the original slides for this track.
What do we mean by Serverless application? An application where we don’t need to manage servers, storage, infrastructure, where scaling up or down is automatic and based on the load at any point in time will automatically adjust, where everything is highly available and secure.
The high availability model for Lambda can be resumed with a simple “Hello World” function be replicated across all the availability zones in the region. If we think to Dublin/Ireland region (eu-west-1) has 3 availability zones (1a, 1b, 1c) (including one or more datacenter).
AWS Lambda is very low cost that’s why you pay for value, by the number of invocation, by the hundreds of milliseconds of execution time.
Main advantage of Serverless as far as we understand the model is cost reduction, Danilo says that there are a lot of customers that are easily reducing the cost from 30% to 90%. An additional advantage of using a Serverless architecture is that enable fast implementation of innovation and new ideas (Agility).
The idea is really to focus on what is going to change on your solution rather than its infrastructure, a Serverless architecture eliminates almost all energy consumption due to manage a physical infrastructure environment. Developers will be able to concentrate on coding new features of the software without worrying about bouncing services around 😉 which is great!!!
Really impressive to see how the Serverless portfolio growing over the years. Some of those are: AWS Lambda, Amazon API Gateway, AWS Step Functions and the Serverless Application Repository.
Today there are hundreds of thousand of companies using Lambda, in this presentation was introduced the case of Capital One an American Bank that built a Serverless public API to advertise credit cards to their customers check this article A Serverless and Go Journey to know more about this use case. Additionally, large enterprises are starting to adopt Serverless. They often start with migrating simple scripts to Lambda transforming what we like to call ‘the cron jobs army’ into Lambda functions. The next step they start building microservices and designing applications with the approach of building additional independent components rather than having a one large monolithic application. Finally customer move into more critical architecture as Capital One did.
Another example of large adoption of the Serverless model is Thomson Reuters Case Study which monitors customers application usage through Amazon Kinesis producing boards of metrics in real time showing how the customers are using the services that they offer. Finra, a financial authority, another Serverless use case, is now processing every single stock transaction using Lambda functions (for more FINRA Case Study).
National Geographic has a full media platform content built on AWS Lambda. These as well others companies are moving to the Serverless model (National Geographic Speeds Innovation with Serverless Computing on AWS).
AWS priorities in the improvement of their services are based on the service fundamentals such as: availability, latency, security and scale. They also try to enable new development patterns, adding new services and more features enable developers to continuously improve and evolve in what they are doing. The ultimate goal is to let builders build and this means that developers needs to concentrate into writing code. An interesting sentence that came from Danilo is that ‘anything that is not unique to you should be something that you can take off-shelf and integrate with your own application. Finally their aim is to empower Serverless developers and operations to easily deploy code and manage the Serverless architecture.
From a service standpoint all AWS features/services are PCI-DSS complaint (this means that they can be used for payments). Last year AWS has been also strictly focusing on GDPR for all services. AWS Lambda has now also an SLA (99,95%).
AWS Lambda is a Tier 1 service, this means that is used by others AWS services and that ‘must’ be available in every region. AWS has been keeping the reduction of prices, for example API Gateway now is charging less than half of the original price (1,51/million). An additional improvement came with Step Functions that are user to build state machines, now they can get up to 1000 transaction/sec.
An interesting launch is Firecracker, which is defined as a micro virtual machine. Before this new innovative approach Lambda used to run in a Sandbox environment, a similar approach to what Docker containers are using to create a Sandbox environment but because of security concerns they had to run every single function in a different sandbox/container and this was resulting in an expensive workflow.
The idea of Firecracker (developed internally) is that now the micro-vm runs on EC2 bare-metal instances, one vm can start in less 125 milliseconds. It is possible to run 150 vm per second and they can run thousands of micron-vms per physical server. It is an OpenSource project so you can find the repo here FireCraker
SQS has been integrated as an event source for Lambda, now you can have up to 20 Lambda function consuming data from Kinesis streams.
In the past few years a lot of data analytics has been moved into the Serverless world that’s why the timeout of the function has been increased up to 15 minutes and the size of the payload of asynchronous events is now 256kb.
On the APIs side now API Gateway offers the possibility to build private APIs reachable from within a private VPC (also limits have been increased).
One of the launch that AWS did at re:Invent 2018 is related to workflows. Today managing business workflow is extremely important. Probably the easiest AWS service to use for this purpose today is AWS Step Functions, they enable the possibility to build complex workflow (state machines).
Here an interesting article on the topic : New Compute, Database, Messaging, Analytics, and Machine Learning Integration for AWS Step Functions
Another integration for Lambda that was launched at re:Invent 2018 is ALB (Application Load Balancer) integrations with Lambda, now there is the possibility to execute a Lambda function from an Application Load Balancer. At the time of this talk was mentioned that Serverless websockets were to be released soon, which happened about 20h ago (Announcing WebSocket APIs in Amazon API Gateway), the general idea is to provide a native WebSocket interface on API Gateway for bidirectional real time applications.
Normally with Lambda function when you were to update your code in the cloud you had to wrap in a zip file all the dependencies and upload the archive (zipped file) for each Lambda function, now you can create a layer (once) and share it with multiple function.
Another interesting launch is Nested Apps for the Serverless Application Repository. This allows the sharing of components, module or full applications privately or publicly with others to improve agility.
Additional languages support such as: Go, Node 8, .NET core 2.1, Powershell and Python 3.7. AWS added also additional improvements on API Gateway, now you get from github a framework for Developer portal . API Gateway has been integrated with the Web Application Firewall and X-Ray.
Now you can create API Gateway endpoint via the Lambda console, the SAM CLI has been re-written from scratch.
Custom runtimes are also a new feature that allows bringing any Linux compatible language (custom runtimes are distributed as layers).
There is now an Application View in the Lambda console that let you view and monitor all resources/components of your application. New is also the Toolkit for JetBrains’ IDEs Intellij and Pycharm – let developers use their own tools to create, test, debug and deploy serverless applications.
DynamoDB is now charges on-demand, we used to set read/write throughput and based on that we were getting charged, now AWS introduced a new capacity on-demand mode, the tables can automatically scale the throughput based on load (Amazon DynamoDB On-Demand – No Capacity Planning and Pay-PerRequest Pricing).
Another extremely interesting launch for DynamoDB is the introduction of Transactions (now it supports the ACID model Atomicity, Consistency, Integrity, Durability (Amazon DynamoDB Transactions)
A super interesting sample app from the Serverless Application Repository that Danilo presented during the talk is the AWS Serverless Twitter Event Source that can be found here (Twitter Event Source App). Which we are enjoying a lot :), Thanks!!!