Is your Company generating large amounts of data which is leading to increased costs, high CapEx, and complex infrastructures? Maybe, the concept itself carries lofty business hopes for unearthing nuggets of hidden data and patterns. Harnessing the power of Big Data means you have to add new technologies to the infrastructure. Moreover,digging into your requirements for business data may turn into finding the right data by filtering out the noises. Especially when business and IT professionals involved in some stage of a big data program, are hitting a wall on one aspect of implementation. Or in its expectations of disruptive tech trends for 2016, Gartner Research writes that the maturity of “strategic big data” will move enterprises toward multiple systems – content management, data marts and specialized file systems.
Now if the problem is clear that effectively combining and managing Big Data is not that simple may be an automated tool is a convenient option.
Introducing Cloudlytics 2.0
The big data framework Cloudlytics 2.0 – built in-house – is an analytics engine that addresses applications from different domains like infrastructure monitoring and IoT. It gives organizations an edge over their competition by providing real-time insights and help take the proactive approach for products and services.
Features of Cloudlytics 2.0
Cloudlytics 2.0 integrated with BlazeClan’s managed services support system automates the process of monitoring logs like Apache, Cassandra, Ruby, Java, Python, JavaScript, PHP, Apache server, Tomcat, MySQL, syslogng, rsyslog, nxlog on a unified dashboard. This also consists 24×7 scrutinizing of server management processes like utilization of CPU, RAM, disk usage etc.
Components and Flow of Cloudlytics 2.0
Cloudlytics 2.0 covers of the following components:
- Aggregation Engine– This engine is in charge of aggregating data produced by large-scale applications from various sources such as IOT Devices, servers for buffering. HTTP, TCP and MQTT protocols are supported by this engine for data ingestion. Amazon Kinesis along with custom API layer has been used to constitute the aggregation layer.
- Transformation Engine– Specific rules are fed into the engine to provide meaningful descriptions of these data by transforming raw data into information. Examples include identifying information related to IP addresses, countries with the maximum no. of hit requests, state from which the content is being downloaded etc. A combination of Amazon Elastic Map Reduce (EMR), LogStash and TalenD along with custom wrappers has been used to constitute the transformation layer.
- Querying Engine– Using AWS services like AWS Lambda and AWS Elastic Search in querying engine allows users to perform advanced queries to gain visibility into transformed data. This is graphically and visually reported on the dashboard.
- Visualization– Based on the filtered data from the advanced queries, the results are organized and presented in customized views using various tools like Kibana, Graffana and Tableau.
Benefits of Cloudlytics 2.0
- Leverages Big Data Tools and Technologies
- Integration with our Managed Services Support System
- User-friendly advanced interface
- Real Time insights
- Effective Deployment
- Personal Cloud Manager
- Scalable and Fast
Use case 1 – Analyzing IoT Weather Station’s Data in Real-Time
The organization’s key needs included the ability to stream real-time data along with complex transformations and visualization and an efficient way to measure fluctuations in the voltage and weather data. The team needed an automated IT solution to help them overcome these challenges effectively, use their resources efficiently, and perform analysis seamlessly.
BlazeClan’s solution enabled the customer with Real-Time Transformation and visualization for the data being streamed from their IOT devices.
Use case 2 – Analyzing Audit Logs
The company is an AWS Customer using AWS CloudTrail as a service for maintaining audit logs. They needed an automated mechanism to draw meaning out of these logs by analyzing and visually representing them. Cloudlytics 2.0 provided the customer the ability to search, analyze, and alert on AWS Cloud Trail log and VPC flow logs. All of the logs are automatically parsed and indexed so that customers can get answers to questions such as:
- What is happening in a user’s account over a given period of time?
- What is the source IP address for an event?
- Which user activities failed due to inadequate permissions?
- Which user changed the settings of a security group and when did the change occur?
- Which user launched or terminated an EC2 instance?