Customer Profile
QioTec offers software platform services for an always-on Industrial Plant, to help customers configure optimized solutions that improve productivity and lower costs. The company lean start-up methodology allows customers rapidly to learn, identify high-value use cases and build prototypes to test business models, new markets, etc. with minimum risk and investment.
QioTec allows customers to fail fast, test assumptions and build minimum viable products that can scale rapidly. They believe that their unique combination of leading-edge open source Internet of Things, Cloud, Mobile and Big Data Technologies with proprietary IP can provide SaaS (software as a service) solutions with significantly improved adaptability, scalability, and cost-to-deliver relative to incumbent software providers.
The Challenge
QioTec was looking to deploy a solution for its customers with specific customizations. The existing deployment was complex and ran in multiple clusters/ instances with complex integration with other services. This setup used to take more than 4 – 5 weeks to make one environment available. QioTec wanted to reduce this time to less than a week so that the new customers can be on-boarded faster.
Another challenge that they faced, was to build a custom solution for log and event monitoring of these deployments. This was very important for QioTec team in order to support their multiple customers with a lean structure.
They were looking for partners who could help in automating the infrastructure deployment on AWS and manage the configuration and security.
The Solution
The BlazeClan team of certified SAs conducted an exhaustive study of the system and charted out a roadmap for deploying this solution.
The team was deeply involved in building the solution for complete deployment automation and creating a unified dashboard for seamless monitoring of all the events and logs of different environments.
BlazeClan was involved in providing 2 major solutions to Qio tec:
ELK Stack:
QIO wanted to process the application logs and the OS logs. For which Blazeclan did the customised ELK stack configurations for QIO. For event logging and monitoring, since the log density/ volume was high and heterogeneous:
Elastic search was used to process the log events and the custom plugin was built to define and monitor the thresholds.
Kibana dashboards were customised to suit the requirements.
Custom ELK was deployed in highly available mode with data being collected using CollectID agents.
DevOps based infrastructure automation
QIO has a large customer base with different AWS account for each of their customer. Their infrastructure architecture was same across these customer. Due to this, whenever a new customer on boards, they required to set up same environment for them, however the number of servers could be different, depending upon the scale of these customer.
For each new instance launch, it had to pass through certain lifecycle rules, which has been defined by QIO. These rules included:
The instance should have the latest version of the application.
It should have a hostname.
It should have a DNS entry in Route53 private hosted zone.
It should update the repositories whenever the instance comes into service.
Automate reboot of instance if the repository update requires it.
DevOps in AWS
BlazeClan’s team comprehended the need to adopt a DevOps based approach for infrastructure automation. They created templates for deploying each component of the infrastructure making it more flexible and scalable.
Deployment/provisioning of QIO’s production, staging, and development environments on AWS was executed using AWS CloudFormation
Ansible scripts were utilized as part of the Continuous Integration and Continuous Deployment process for automated deployments. The instances life cycle has been automated using Ansible scripts.
The Benefits
Reduced Time to market: The company was able to onboard the customer within a weeks’ time as compared to the earlier cycle of 4-5 weeks.
High availability: Successful deployment of the solution followed by 24/7 monitoring resulted in making the platform highly available for the company.
Agility: With real-time monitoring of the platform and deep dive into log analytics, QioTec was able to perform faster root cause identification in the event of problems.
Tech Stack
BlazeClan availed a number of AWS services to execute this project successfully.
- Amazon EC2 was used for computing capacity management for their application deployment. It helped in reducing the time required to spin up new server instances to minutes, allowing them to quickly scale capacity, both up and down, as per their requirement.
- Amazon RDS service was used to set up, operate and scale relational database and was mainly used for Postgres deployments.
- ELK stack was utilised for log aggregation and analytics. It is a combination of Elasticsearch (a NoSQL database and search server), Logstash (a log shipping and parsing service), and Kibana (a web interface that connects users with the Elasticsearch database and enables visualisation and search options for system operation users). It helped in providing a centralised and searchable repository for all infrastructure logs, thereby providing a unique and holistic insight to the customer.
- Amazon S3 was used to store and retrieve any amount of data from anywhere and everywhere.
- Amazon VPC helped in setting up the private network with AWS as it provided isolation and security.
- AWS CloudFormation was used in the production/development/UAT environments for mirroring purposes for automated deployments.
- Ansible scripts were utilised as part of the Continuous Integration and Continuous Deployment process for automated deployments. The instances life cycle has been automated using Ansible scripts.
