Shopping on Amazon
Ever got frustrated while shopping on Amazon? Maybe because of their wide range of alternatives for every product, but definitely not because of their e-commerce Apps. Whether it’s your saved payment details or grocery items you wanted at the last minute, Amazon seamlessly manages your data at scale, speed and with state of the art security measures
Data is the most critical part of any application, making sure it’s secured and delivers as quickly as possible is something most customers expect as a basic functionality these days. If a shopping website doesn’t impress you within the first 30 seconds, it’s probably the last time you might visit it right? Well, that’s the approach we millennials carry on a usual weekend or a shopping sale festival. However, it takes a massive infrastructure setup to handle such kind of relentless performance and to keep the shopaholics coming back every time.
It’s quite an uber-luxury for Amazon to have a decent IT department, they call themselves Amazon Web Services, heard about them? If not, just an FYI they also back most of the infrastructure for Netflix (the mother of drag. err well the TV shows!). The very fact of having a high profile client’s finance and reputation at stake, anything short of a reliable, scalable and durable solution would have been intolerable.
One of the databases that back Amazon’s huge data collection is indeed one of the giants of the NoSQL world — AWS DynamoDB — a fast and flexible non-relational database service for all applications that need consistent, single-digit millisecond latency at any scale. It is a fully managed cloud database and supports both document and key-value store models. Its flexible data model, reliable performance, and automatic scaling of throughput capacity make it a great fit for mobile, web, gaming, ad tech, IoT, and many other modern-day applications.
Why NoSQL?
NoSQL designs deliver faster data operations and can seem more intuitive, while not necessarily adhering to the ACID (atomicity, consistency, isolation, and durability) properties of a relational database. Although, with a little more effort of having an additional DynamoDB table to capture additional values or a Global Secondary Index, etc. you can make DynamoDB work for your case as a regular SQL store would have. There are many well-known NoSQL databases available, including MongoDB, Cassandra, HBase, Redis, Amazon DynamoDB, etc. Each of those was built for a specific range of uses and will offer different features.
Our story
Here at Blazeclan, we have been utilizing AWS DynamoDB for many of our customers. One of our use cases was a complete Serverless and NoSQL solution for implementing a middleware application. AWS DynamoDB, AWS Lambda and a few more AWS services were chosen for storing data and creating APIs for implementing a microservices-based architecture.
We used DynamoDB for storing Session and Configuration data of our customers. This data was consumed on the AWS Lambda functions for creating APIs for the application. AWS DynamoDB provided us millisecond latency at scale, helping us achieve blazing performance, monitoring and integrating with other AWS services with ease. API gateway was used to expose the APIs as well as to enforce security.
DynamoDB is a key-value store and works really well if you are retrieving individual records based on key lookups. Although if you have a “hot-key” in your dataset, i.e., a particular partition key that you are accessing frequently, make sure that the provisioned write capacity unit on your table is set high enough to handle all those queries.
So if your application is using too many Read Capacity Units on a single key, you either need to over-provision all the other partitions (expensive), manage a ton of “Throughput Exceeded” errors (not ideal), or figure out how to decrease access to that key.
In our case, we were able to identify the “Hot partition” issue by logging every method’s processing time. This way, we were able to narrow down the DynamoDB calls which were taking much longer than usual to respond when a particular partition key was queried frequently within a short time span.
The DRS for DynamoDb – DAX
One of the solutions to avoid hot-keys was using Amazon DynamoDB Accelerator (DAX), which is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement, even at millions of requests per second.
DAX does all the heavy lifting required to add in-memory acceleration to your DynamoDB tables, without requiring developers to manage cache invalidation, data population, or cluster management.
Moreover, one does not need to modify application logic, since DAX is compatible with existing DynamoDB API calls and can be enabled with just a few clicks in the AWS Management Console or using the AWS SDK.
Although for cost optimization we proposed a much cheaper alternative, We created separate APIs which returned the DB response and were cached using AWS CloudFront. We further modified the code to call these cached APIs instead of the DB tables. As the data of the partition keys that was being queried was not frequently modified, caching the DB response for a few minutes did the trick, thus avoiding querying the partition key frequently.
DynamoDB thus complemented our serverless solution quite well. It’s fast, flexible and if you architect your solution properly, it can fit like a glove in your application as well! Contact us today at sales@139.59.4.14 to know how we can leverage the benefits of AWS Cloud for your business requirements.
