Participating in AWS chatbot challenge 2017
Last year at reInvent 2016, AWS introduced Amazon Lex - a service for building conversational interfaces using voice and text. To promote this service, Amazon announced AWS chatbot challenge 2017. It was a great opportunity to learn a new service in depth and build something from scratch. Therefore, me and my friend Samtadecided to give it a try. After a few weeks of efforts, We successfully built a chatbot called SplitSmart Bot ( watch video ) which allows user to manage shared expenses using SplitWise. Bot is deployed on Facebook messenger which makes it work on seamlessly on any browser, phone, tablets etc. You can chat, discuss and split expenses in one place.
This post describes our short but exciting journey of participating in the hackathon and building SplitSmartBot.
Getting started
When we started thinking about the challenge we had two concerns:
- We didn't have any experience in NLP and deep learning.
- We didn't have much UI or front end experience to build chat interface.
However, AWS surprised us big time. We discovered that we don't need either. Lex abstracts all the complex deep learning and NLP stuff with easy to understand constructs like Bot, Intent, Slots types and utterance. AWS documentation has some really good hands on tutorials to explain these basic concepts. Moreover, Lex provides seamless integration with existing messenger platforms like Facebook and Slack. So that became our front end. Both problems solved !
Implementation Challenges
Building a chatbot has different set of challenges than a typical web or mobile application. Here are some of the challenges we faced.
- Implementing Oauth workflow inside chatbot: Our bot needs to access user's SplitWise account for making calls to SplitWise API. Therefore, user needs to authorize the bot to use his account. SplitWise supports this via Oauth which is easy to integrate with web or mobile app. However, challenge was to implement this workflow from within chatbot. We decided to go with Serverless approach and implement this AWS API gateway, Lambda and Dynamodb.
- User experience: The challenge was to make interaction with chatbot simple and intuitive. To make it happen, we need to 'train' Lex engine with different phrases or sentences ( called utterances in Lex terminology ) We interacted with Lex experts on slack channel and followed their recommendations. However, the real issues were discovered during our beta testing ( when we asked friends to try out the bot). Depending upon their feedback, we trained our Lex bot to handler variety of interactions. E.g. To add a new expense, some people preferred to type "I paid 300 Rs for drinks" while others were more comfortable with "add bill of Rs 300". Good thing about Lex is you don't have to change backend code to handle new utterances.
Accomplishments
- Serverless Oauth implementation: As mentioned above, Serverless Oauth implementation was one of the trickiest to implement. It was a good learning experience. See our submission page to learn more about architecture
- Open source contribution: To interact with SplitWise API, we used open source python SDK. However, some features we need were missing in this SDK. We are really pleased to be able to make open source contribution.
Support from AWS community
For any new service to become popular, you need solid documentation and community support. As usual, AWS documentation was up to the mark. Moreover, there was a Slack channel for participants to ask questions, discuss ideas and report issues. People shared code snippets, how-to videos, scripts etc. which helped us move faster. Overall it was a wonderful experience.
Final outcome - SplitSmartBot !
After a few weeks of reading, design and implementation ( On weekends ! ), we are proud to present our entry - SplitSmart Bot. Right now it is NOT publicly available and can be access only on invitation basis. Watch the bot in action here on my youtube channel. If you are interested to try out, please visit our Facebook page and send us a message. For more information on design and architecture, visit our submission page.
We are still waiting for the results. But the journey was indeed fulfilling.
- Atharva and Samta
Serverless 'SPA' for tired cloud architects
Architecting applications for cloud is not an easy job. A few years back when we talked about 'going cloud', we had limited options like PaaS, IaaS etc. In last few of years, containers took the industry by storm and we started talking about microservices based architectures. As if this change was not disruptive enough, serverless architecture joined the party. Serverless architecture ( sometimes called FaaS ) is a very promising alternative to traditional Paas/IaaS approach. Serverless doesn't mean there are no servers. It only means servers are provisioned on demand by cloud providers. With serverless architecture, the burden of provisioning and maintaining the resources such as VMs, volumes, IP addresses, load balancers is take away from architects. Cloud providers such AWS, Microsoft, Google, IBM are already providing support for building serverless applications.
Another interesting paradigm which is around for a while is SPA - Single Page Applications. SPAs are web apps that load single html page and dynamically update the page as user interacts with the app. SPAs use Ajax/HTML/Javascript to create fluid and responsive web pages avoiding constant page reloads.
In this post, I will discuss how Serverless and SPA together make a great pair for building cloud-native applications based on microservices architecture. I will use an example of an imaginary movies booking application called www.i-love-movies.com where users can search movies, showtimes and book a show.
Architecture
Our movie application is built using AWS cloud ( No surprises ! ). Let's take a look at how we can use different AWS services to build application without provisioning even a single VM!
A] Hosting a website in S3: Our SinglePage application is built using AngularJS/HTML/Javascript/CSS technologies. But how do we server the content to users ? As we see in the digram, there are no VMs running webservers. Instead, Our application is hosted in S3 bucket with website hosting enabled. DNS records for www.i-love-movies.com are registered with AWS DNS service Route53. Optionally, we can use AWS CDN ( content delivery Network) for better performance.
B] Building backend microservices with Lambda and API Gateway: For dynamic interactions, we use AWS Lambda for building microservices such as movies service, showtime service etc. Advantage of Lambda is resources required to execute the functions are provisioned on demand and you pay only for the execution time and memory/CPU resources. Amazon API Gateway exposes the Lambda functions as REST APIs. Lambda function uses Amazon RDS ( database service ) for persistence.
C] AWS SDK for Javascript: This is another cool feature provided by AWS where one can interact directly with AWS services from javascript page ( See details ) . As shown in the diagram, SDK interacts with API gateway to call lambda functions. SDK also interacts with AWS Cognito to authenticate users through social identity providers such as Facebook, Twitter.
D] Deploying Serverless SPA with SAM: AWS cloud formation is a great tool for deploying AWS services as a single stack. Recently, AWS introduced Serverless Application Model ( SAM) which provides simplified syntax for expressing serverless resources. SAM extends AWS CloudFormation to add support for API Gateway, AWS Lambda etc. For monitoring and logs, AWS CloudWatch and CloudTrail support is available for Lambda functions.
Note: This is definitely not a complete architecture of movies application (e.g. payment gateway integration is missing.
Advantages
- No more headache of provisioning VMs, networking topology, firewalls, load balancing etc. No need to worry about autoscaling. AWS Lambda takes all the burden
- You pay only for the execution time and resources used by Lambda function. No more hourly billing.
- SAM simplifies overall deployment of application and Devops efforts
Drawbacks
- Migrating existing/legacy applications to Serverless architecture is not straightforward. This approach works well for new applications.
- Vendor lock in !
- Lambda function has certain limits. E.g. Time limit of 300 seconds. It is not meant for long running functions.
Conclusion
To summarize, Serverless Single Page application architecture provides a new way of building cool web applications. However, just like any new technology, it is not panacea. Happy Architecting !
References
- Martin Fowler: https://martinfowler.com/articles/serverless.html
- Serverless Application Model : http://blog.couchbase.com/2017/january/microservice-aws-serverless-application-model-couchbase
- AWS Lambda: http://docs.aws.amazon.com/lambda/latest/dg/welcome.html
- Single Page Applications : https://en.wikipedia.org/wiki/Single-page_application
Restless Microservices: Part 1
Over the last few years, REST has been a de-facto standard for building platform independent web services. REST along with JSON provides a lightweight protocol for building web services which can be seamlessly consumed by browser, mobile applications and desktop clients. Whether it is a web service exposed to customer or a backend service, REST is the preferred choice of application developers. However, with the advent of microservices architecture, there are a new set of challenges in the inter-microservices communication. REST can still be the best option when we want to expose APIs to customers. However, when we have dozens of independent microservices communicating with each other, is REST/JSON the most efficient protocol for exchanging messages across services running in the same context, same network, and possibly the same machine? Are there any alternative protocols/frameworks available ? Are these options being used at very high scale by giants like Google, Facebook , Twitter and LinkedIn etc?
In this series of blog posts, we will try to answer the above questions. We will discuss and evaluate different alternative frameworks developed and/or used by internet giants Facebook, Google Twitter and Amazon .Part 1 covers the motivation behind not using REST and gives a short summary of the different alternatives such as RPC , Asynchronous Message broker communication, Serverless microservices etc. In the upcoming blog posts, we will in detail, evaluate microservices frameworks developed and open sourced by Facebook, Twitter and Google etc.
References
[1] HTTP2: https://hpbn.co/http2/
[2] Protocol Buffers vs JSON : http://blog.codeclimate.com/blog/2014/06/05/choose-protocol-buffers/
[3] REST vs JSON : https://www.linkedin.com/pulse/rest-vs-rpc-soa-showdown-joshua-hartman
In this series of blog posts, we will try to answer the above questions. We will discuss and evaluate different alternative frameworks developed and/or used by internet giants Facebook, Google Twitter and Amazon .Part 1 covers the motivation behind not using REST and gives a short summary of the different alternatives such as RPC , Asynchronous Message broker communication, Serverless microservices etc. In the upcoming blog posts, we will in detail, evaluate microservices frameworks developed and open sourced by Facebook, Twitter and Google etc.
Why RESTless
Although, REST/JSON provides a very lightweight protocol for communication, following factors motivated software industry to find better alternatives for microservices.a] Lack of schema
In microservices environment, number of services can grow rapidly with each exposing multiple APIs. Lack of schema in REST/JSON poses a big integration challenge in this case. In such environment, it is a good idea to maintain a well-defined interface for every call. Interface Definition Language ( IDL ) such as Thrift or Protobuff forces service owners to publish strict interface definitions, which streamlines the process of integrating with services. IDL based frameworks provide built-in schema validation which is another plus point over REST.
b] REST modeling can be unwieldy sometimes
Sometimes it is difficult or unwieldy to map certain APIs to the REST model of Resources and methods. In other cases the mapping may require client to make several serial calls, hurting performance in the datacenter.
c] Binary protocols are more efficient than JSON
There is a performance impact due to cost of serializing objects to a large JSON string. Keeping data in binary both in memory and on the wire by building eliminates cpu and memory required for string encoding schemes such as Base64 or JSON. In monolithic architecture this overhead was negligible as we had to perform this serialization only a few times. With dozens of microservices talking to each other, this overhead can be significant.d] Advent of HTTP2
HTTP2 protocol was published in 2015 and still in the phase of early adaption by browsers and application servers. Following new features added in HTTP2 proves to be very suitable for microservices.- Binary protocols: At the core of all performance enhancements of HTTP/2 is the new binary framing layer(, which dictates how the HTTP messages are encapsulated and transferred between the client and server.All the HTTP semantics, such as verbs, methods, and headers remain same, but the way they are encoded while in transit is what’s different. Unlike the newline delimited plaintext HTTP/1.x protocol, all HTTP/2 communication is split into smaller messages and frames, each of which is encoded in binary format.
- Bidirectional streaming and multiplexing: In HTTP/1.x, if the client wants to make multiple parallel requests to improve performance, then multiple TCP connections must be used. The new binary framing layer in HTTP/2 removes these limitations, and enables full request and response multiplexing. Using this feature multiple requests and responses can be interleaved without blocking on anyone.
- Server Push : Another powerful new feature of HTTP/2 is the ability of the server to send multiple responses for a single client request. This means, in addition to the response to the original request, the server can push additional resources to the client without the client having to request each one explicitly!
Alternatives
Over the past few years, many new approaches/frameworks built to mitigate the problems of REST discussed in the previous section. Some of the popular ones are listed below. These approaches/frameworks are used by many internet giants for developing highly scalable services.
a] RPC frameworks
RPC stands for "remote procedure call". The concept of RPC is not new . However with the advent of HTTP2 and new binary protocols, some promising frameworks have come up which provide better performance over REST. Some of the more popular open source frameworks are Thrift ( Facebook), GRPC ( Google ), Spring RPC, Finagle ( Twitter).b] Async Messaging ( AMQP , task queues)
For microservices architecture, Asynchronous messaging with tasks queues can be a very good alternative. Especially, when we have chain of calls ( MS1 -> MS2 -> MS3 ) among microservices, async messaging provides better decoupling. This approach also avoids the HTTP issues like timeout, connection reset, load balancing etc.c] Extended/Improved REST framework
In spite of some limitations, many people are not willing to abandon the REST completely. So, they came up with some extended / improved version of REST. Rest.li from LinkedIn and Netflix Ribbon are some of the examples where they tried to improve REST by adding features like type safety, client side load balancing etc.d] Server less architecture ( FaaS )
Serverless architecture or Function as a Service, is becoming a very popular alternative for building scalable microservices. The idea is to trigger a Function ( Code ) based on the event such as file upload, notification etc. Cloud providers like Amazon ( Lambda ), Microsoft ( Azure functions ) provide FaaS where we just need to upload the code and define the event to trigger the code. Rest of the server management, scaling etc is taken care by providers.Conclusion
After discussing so many approaches, the million dollar question is which one is the best ? As usual the answer is, it depends on the use case, application and scale requirements. Even internet giants like FB, LinkedIn have different experiences and opinions. FB/Google developed their own RPC frameworks, while LinkedIn started with RPC but later abandoned it to chose REST [3]. In the upcoming posts of this blog series, we will try to evaluate these options with the hope that we will be able to make an informed choice. So stay tuned !References
[1] HTTP2: https://hpbn.co/http2/
[2] Protocol Buffers vs JSON : http://blog.codeclimate.com/blog/2014/06/05/choose-protocol-buffers/
[3] REST vs JSON : https://www.linkedin.com/pulse/rest-vs-rpc-soa-showdown-joshua-hartman
Docker1.12 on AWS : 10 command(ment)s to manage Swarm cluster
Introduction
In July 2016, Docker Inc. released 1.12 version with many great features including Swarm Mode ( built-in orchestration ). Latest Swarm Mode along with docker-machine tool greatly simplifies the the job of creating/managing a cluster of docker containers on different cloud providers. In this blog post, I am going to explain how to create/manage a 3 node Swarm Cluster on AWS using docker-machine.
Note : These steps are only applicable to docker 1.12 and above !
Pre-requisite Steps
a] Install docker-machine on your Linux machine or Mac.
b] Create AWS account ( If don't have already ) and configure AWS credentials for docker machine. See details here.
c] In AWS EC2 console, create a security group called "docker-machine". Add following inbound policies to the security group. Following ports are required for swarm cluster communication. See this video for the details
b] Create AWS account ( If don't have already ) and configure AWS credentials for docker machine. See details here.
c] In AWS EC2 console, create a security group called "docker-machine". Add following inbound policies to the security group. Following ports are required for swarm cluster communication. See this video for the details
Once we have all the requisites in place, we are ready to deploy Swarm cluster on AWS using our 10 command (ment )s.
1. You shall use 'docker-machine create' to bring up new VMs
This command creates EC2 instance and installs docker 1.12 on it.
docker-machine create --driver amazonec2 swarm-manager
docker-machine create --driver amazonec2 swarm-worker-1
docker-machine create --driver amazonec2 swarm-worker-2
Verify the status of the machines using
docker-machine ls
2. You shall use 'docker swarm init' to configure Swarm manager
This command initializes Swarm manager on the VM. To use this, first get IP address of swarm manager machine followed by swarm init command.
MANAGER_IP=`docker-machine inspect --format '{{ .Driver.PrivateIPAddress}}' swarm-manager`
docker-machine ssh swarm-manager sudo docker swarm init --advertise-addr "$MANAGER_IP":2377
3. You shall use 'docker swarm join' to configure Swarm workers.
Swarm worker nodes (worker-1, worker-2) can join swarm cluster using this command.
First get the token from swarm manager.
Join worker-1
Swarm worker nodes (worker-1, worker-2) can join swarm cluster using this command.
First get the token from swarm manager.
TOKEN=`docker-machine ssh swarm-manager sudo docker swarm join-token -q worker`
Join worker-1
docker-machine ssh swarm-worker-1 sudo docker swarm join --token "$TOKEN" "$MANAGER_IP":2377
Join worker-2
docker-machine ssh swarm-worker-2 sudo docker swarm join --token "$TOKEN" "$MANAGER_IP":2377
4. You shall use 'node ls' & 'node inspect' to verify status of Swarm cluster
5. You shall use 'docker service create' to deploy service on Swarm cluster
Service create command deploys one more containers on the Swarm cluster.
docker-machine ssh swarm-manager sudo docker service create --name myredis --replicas=1 redis:3.0.6
6. You shall use 'service ps' and 'service inspect' to verify status of deployed service
7. You shall use 'docker service scale' to scale up/down the service
8. You shall use 'docker service update' for rolling updates
E.g. update redis image to version 3.0.7.
9. You shall use 'docker service rm' to undeploy service
10. You shall use 'docker-machine stop' and 'docker-machine rm' to delete VMs
Stop/Delete EC2 instances using stop and rm command.
Get status of Swarm cluster by running this command on Swarm Manager.
docker-machine ssh swarm-manager sudo docker node ls
User Inspect command to get status of both manager and worker.
docker-machine ssh swarm-manager sudo docker node inspect swarm-worker-1
docker-machine ssh swarm-manager sudo docker node inspect swarm-manager
Service create command deploys one more containers on the Swarm cluster.
docker-machine ssh swarm-manager sudo docker service create --name myredis --replicas=1 redis:3.0.6
6. You shall use 'service ps' and 'service inspect' to verify status of deployed service
docker-machine ssh swarm-manager sudo docker service ps myredis
docker-machine ssh swarm-manager sudo docker service inspect myredis
docker-machine ssh swarm-manager sudo docker service scale myredis=2
8. You shall use 'docker service update' for rolling updates
E.g. update redis image to version 3.0.7.
docker-machine ssh swarm-manager sudo docker service update --image redis:3.0.7 myredis
9. You shall use 'docker service rm' to undeploy service
docker-machine ssh swarm-manager sudo docker service rm myredis
10. You shall use 'docker-machine stop' and 'docker-machine rm' to delete VMs
Stop/Delete EC2 instances using stop and rm command.
docker-machine stop swarm-worker-1
docker-machine stop swarm-worker-2
docker-machine stop swarm-manager
docker-machine rm swarm-worker-1
docker-machine rm swarm-worker-2
docker-machine rm swarm-manager
References:
1. Docker Swarm Mode : https://docs.docker.com/engine/swarm/
2. Docker machine : https://github.com/docker/machine
Keep Calm, Use COM - A New Approach to Build Next-Gen Cloud Applications
What is COM
COM stands for Container, Open source and Microservices. In late 1990s, .com revolution transformed the way computing was done. Today, containers and microservices are transforming the way cloud computing is done. Microservices architecture is not just an IT buzzword anymore. Many companies have successfully adapted microservices architecture to develop highly scalable cloud applications. Cloud providers like Amazon, Microsoft, Google, etc. are building an ecosystem to facilitate application development using containers and microservices. No wonder open-source community played a significant role in this revolution. If you are wondering how the the next-gen scalable cloud applications will be built, COM approach is one of the most promising alternatives.
The following mind map gives an overview of COM ecosystem. This is definitely not an exhaustive list, however it could be a good starting point for exploring various available options.
To summarize, keep calm and trust COM!
Note : The acronym COM used here is meant only for the purpose of explanation. It is not a standard acronym.
Why COM
- Containers: Build, ship and Run anywhere. A docker container can run on Bare Metal, VM, AWS, Azure, Google, Digital Ocean. Docker1.2 provides native support for Mac and Windows too.
- Open source: It's open source, duh!
- Microservices: Scale, flexibility, system resilience. Monoliths are bad for health!
COM Ecosystem
Building an individual microservice might be easy. Challenge is to integrate all services to build the complete solution. Therefore, for building COM based applications, following aspects need to be considered.
- Containerizing a service: While packaging a service in the container, we must remember that container is NOT a VM. Some of the best practices while building containers are one process per container, ephemeral behavior (can be stopped and a new one can be put in place with an absolute minimum of set-up and configuration).
- Service discovery: COM based cloud application can have hundreds and thousands of containers coming up and going down continuously. Hardcoded IP addresses or traditional DNS based service discovery will not be useful in such scenarios. We need to use a dynamic and scalable service discovery solutions. Open source solutions like zookeper, consule are the suitable for such use cases.
- Service-service communication: For synchronous communication, REST/JSON has been a de-facto standard for past several years. However, other promising alternatives like grpc/protbuf, thrift have emerged. These RPC frameworks take advantage of new HTTP2 (introduced in 2015), binary protocols and bi-direction streaming. For async communication, frameworks such as AMQP, STOMP have been widely used in the industry.
- Monitoring and logging: With hundreds of containers running in the system, we need specialized solutions such as fluetd, datadog, cAdvisor for system monitoring and logging.
- Automation and CI/CD: Building and deploying microservices without proper automation and CI/CD can prove to be the worst nightmare for an engineering team. Apart from traditional deployment techniques, engineering/devops team needs to consider container orchestration and container networking mechanism.
The following mind map gives an overview of COM ecosystem. This is definitely not an exhaustive list, however it could be a good starting point for exploring various available options.
To summarize, keep calm and trust COM!
References
- Linux Foundation : Introduction to cloud infrastructure technologies
- Martin Fowler: http://martinfowler.com/articles/microservices.html
- Docker best practices : https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
Note : The acronym COM used here is meant only for the purpose of explanation. It is not a standard acronym.
Subscribe to:
Posts
(
Atom
)
