Prelude

In the past year, I worked on a project to build an e-learning platform (actually, it’s so much more!). I started using AWS Amplify, and I end up testing so many different services around it. I’ll cover the following topics:

  • Choosing the right AWS services;
  • Designing the application architecture;
  • Developing the backend;
  • Developing the frontend;
  • Deploying the application;

I hope this article will be helpful for anyone who is considering building an e-learning platform or something similar. Spoiler is: the platform is not ready yet, but I hope I will have the chance to conclude it by the end of this year 🙏 my time is super limited unfortunately :/

Choosing the right AWS services;

I have always been fascinated by aws managed services mainly because of their scalability and the ease with which they make it possible to prototype projects. So I decided to leverage AWS Amplify because I knew it was a suitable project for this scenario: prototyping.

Roughly speaking, any self-respecting application relies on three big pillars: a backend, which I knew I could create leveraging API Gateway and AWS Lambda. A frontend, which I knew I could create leveraging a framework I hold dear (Vue.js) and be able to host on AWS at almost zero cost thanks to services like S3 and Cloudfront. And a system for storing information, both as objects and as data: precisely, I knew I could rely on S3 and DynamoDB.

What makes it all complex, however, is how to make the details work: how to manage authentication within the system? how to manage user access and permissions? how to secure streaming and downloading of data? how to make payments work without blocking everything if the services involved don’t work?

To explain these points, it may be helpful to start with the final outline of what I have made so far, locate the key components, and then talk about each component as we go along.

Let’s deep dive into diagram sections.

Landing Zone

A part from Cloudflare in front of everything, we run everything behind an API Gateway created through AWS Amplify. Amplify lets you create an API Gateway able to serve your APIs and integrate them with Cognito - used both as Identify Provider and authorizer, together with IAM and aws policies. More in detail, Cognito User Pool is used as directory services of federated identities that you can exchange for AWS credentials. Identity pools generate temporary AWS credentials for the users of your app. These identities are used to let you operate with AWS services accordingly (i.e. for instance, they let you access protected content on S3, available only to specific identifies, as explained here). The cool thing is that AWS Amplify takes of this automatically, and it can do it not only with storage layers, but also with API (see doc). Everything can be done with the CLI, and integrated in your frontend by using the official npm package (see more on this).

Morever, we have a integration that is handled through a dedicated API Gateway with Stripe. Stripe provide endless ways to handle many different payments methods, and it’s shipped with an API layer and integration webhooks that let your application fluently integrate with payment system.

Video content production

The diagram starts with video content stored in Amazon S3. From here, the video is sent to AWS Elemental MediaConvert, a cloud-based video transcoding service that converts the video into the desired format and bitrate for streaming. This is done thanks to a dedicated AWS Lambda function triggered to submit the transcoding job. The Lambda function is responsible for initiating the transcoding process: the necessary parameters to MediaConvert are sent as part of the input to the jobs, available on S3 with a dedicated prefix.

As the transcoding job progresses, Amazon CloudWatch, a monitoring and observability service, tracks the status of the job and logs any relevant events. CloudWatch provides visibility into the performance and health of your AWS resources, enabling you to identify and resolve issues promptly.

Once the transcoding job is complete, Amazon EventBridge, a serverless event bus service, emits an event notification. EventBridge is used to decouple different parts of your application and enable them to communicate with each other. The event notification triggers another AWS Lambda function, which is responsible for packaging the transcoded video and storing it back in S3, and notify me through SNS.

Securing the serving content

Finally, the transcoded video is served to viewers through Amazon CloudFront, a content delivery network (CDN) that caches and delivers content to users with low latency and high performance. CloudFront is a global network of edge servers that replicates your content across different locations, ensuring that users have a fast and reliable viewing experience.

To safely serve content, I leverage HLS streaming based on Cloudfront Signed Cookies: this was possible throught a lot painful tests, and I’m going to deep dive into the way this solution can work. Practically speaking, the way you can safely stream your premium content is to provide an authenticated endpoint to your user to create and store signed cookies, and inject them accordingly in your frontend using an HLS-ready library. The combo of this steps + the fact that cookies are scoped to the host (thus it’s impossible to test this solution by serving you application locally during development) requires me months of tests to complete.

The useful thing inside amplify is that you can manage custom resources using CDK or Cloudformation templates, so that their deployment goes through the same release cycle as other Amplify resources. This is what I did for this part.

Premium content production

In this part, I again made use of a number of technologies not directly related to the world of Amplify: leveraging AWS Fargate and a chatbot, I built a pipeline capable of responding in a scheduled manner to content generation. These are produced and pushed to a temporary bucket. Again, leveraging a dedicated Event Bridge bus, the execution of a second docker image is triggered that takes this content and places it within a pull request on a Github repository. The content is not video, fear not! But I will talk about that in a dedicated article!

Conclusion

In this article I showed how high-level use can be made of many AWS managed services to instrument an application complete with a backend, a frontend, and processes to produce content in the background. In the next articles we will go over how each of these parts work taken individually, with a focus on the most critical ones.

Go to the next article about how to init the Amplify Project