Serving the static/configured API response through S3

By Rajesh krishnakumar- “Go above and beyond!”

The title idea kicked in when I saw our APM platform- a static/config api- getting hit 1200req/sec. I thought let’s move this api to the edge so the server will be free of TCP connection, memory, and computing power. This led me to explore how to serve api from edge.

The Advantage of Caching API at Edge

• Improve latency.
• Run the code closer to your customer.
• Keep the server free.
• DOS attack free.

The disadvantage of Caching API at Edge

• We don’t have control over data.
• To monitor CDN we need to pay extra costs to AWS.

Solution

After a lot of front and back, I decided to serve the static/config api response through s3.

Why s3?

• Let’s assume 1k-2k req/sec hit is coming to that api. We invalidate the distribution and suddenly cache miss-hit pops onto the server. These can overwhelm our server causing a cascading effect to increase in the response time which leads to a bad customer experience. By using s3 we can avoid this issue.
• Our existing api is in graphQL we need to write a rest api / Lamba to support the CDN distribution part. It will add up the extra cost

Hence, I decided to use S3 for distribution.

Architecture

Invalidation approach

1. Upload updated JSON data in s3.
2. Trigger the invalidation using api.

Problem we faced

I’m was ready with the CDN endpoint, but then my manager suggested we give a load test and check the stability. We saw the response was higher than we expected. Hence we were disappointed. I decided to send a mail to the AWS team as we are using CloudFront CDN to understand the problem.

AWS team comments in their own words

Traditional load testing methods don’t work well with Cloudfront. The reason is that CloudFront uses DNS to balance loads across geographically dispersed edge locations and within each edge location. When a request for content is sent to CloudFront from a client, the client receives a DNS response that includes a set of IP addresses. If you test by sending requests to just one of the IP addressed that DNS returns, you are testing only a small subset of the resources in one CloudFront edge location, which doesn’t accurately represent actual traffic patterns. The performance of the small subset of CloudFront servers may be degraded depending on the volume of data requested.

To conclude, CloudFront is designed to scale for viewers that have different client IP addresses and different DNS resolvers across multiple geographic regions.

To perform load testing that accurately assesses CloudFront performance, we recommend that you do all of the following:

• Send client requests from multiple geographic regions.
• Configure your test so each client makes an independent DNS request; each client will then receive a different set of IP addresses from DNS.
• For each client that is making requests, spread your client requests across the set of IP addresses that are returned by DNS, which ensures that the load is distributed across multiple servers in a CloudFront edge location.

Aws team helped us with the load test and we are satisfied with the response time.

Key takeaway

1. Learned caching the api response at the edge
2. Load testing for CloudFront.

Summary

To summarize, we have created CDN distribution with s3. S3 will have our response JSON. whenever we need to invalidate we upload new JSON data and invalidate CDN distribution.
Using this approach the server will be free of TCP/HTTP connection, memory, and computing power.

Meet the team!

Author
Rajesh Krishnakumar

Reviewer
Muthukumar K

Editor
Mridula Saravanan

We at CaratLane are solving some of the most intriguing challenges to make our mark in the relatively uncharted omnichannel jewellery industry. If you are interested in tackling such obstacles, feel free to drop your updated resume/CV to careers@caratlane.com!