AWS Cloud\u2019s Kinesis
\n\u201cIt is a capital mistake to theorize before one has data.\u201d \u2013 Sherlock Holmes<\/p>\n
Just a few days back I overheard someone saying, \u201cYou know the world no longer revolves around the sun but DATA\u201d. Well, I couldn\u2019t disagree. Today, the whole world revolves around data. But having data alone is of no use; it is like having a gun with no bullets; you have it but cant make use of it. And here steps in, \u201cBig Data Analysis\u201d \u2013 the driving force of today\u2019s business world. As Rightly said by Geoffrey Moore,<\/p>\n
\u201cWithout big data analytics, companies are blind and deaf, wandering out onto the web like a deer on a freeway.\u201d<\/p>\n
What the Industry leaders are thinking?
\nThe \u2018Big\u2019 in \u201cBig Data Analysis\u201d implies the humongous size of data (upto Peta-bytes), that needs to be analyzed. However the beauty of big data does not lie in just collecting the data, but to make sense out of it also. A recent survey conducted by IBM across 900 businesses and IT executives from over 70 countries, following trends emerged:<\/p>\n
Industry leaders<\/p>\n
Are 166% more likely to make most decisions based on data.
\nAre 2.2 times more likely to have a formal career path for analytics.
\nCite growth as the key source of value from analytics (75%).
\nMeasure the impact of analytics investments (80%).
\nHave some form of shared analytics resources (85%).
\nThis shows the significance of data in today\u2019s dynamic business world and the impact of its proper analysis linking it directly to the success of a business. However, the processing of this data is done in terms of \u201cBatch Processing\u201d i.e. data generated over a time interval ranging from a few hours to weeks or even months are collectively processed. So if data that is \u201cold\u201d can make such a difference, imagine the impact of real time data analysis. Analyzing data as and when it arrives.<\/p>\n
In the Dawn of this Big Data onslaught, Cloud service-giant Amazon Web Services has released its new service, \u201cKinesis\u201d to tackle the Big Data Challenges.<\/p>\n
[How AWS Cloud Components are solving the Big Data Conundrums]<\/p>\n
So what Exactly is Amazon Kinesis ?
\nWell, we talked about big data, its significance & batch processing. So data was being analyzed in a few weeks or months, so would it make any difference if it could be processed in Real Time?<\/p>\n
Well before you answer instinctively, Consider this. Your favorite mobile company launching a new model is promoting it on the Internet. The company will try hard to track the related Tweets & Status Updates and collate all these \u201cresponses\u201d understanding the market\u2019s response to its proposed product. Based on which it can make changes to the product or marketing strategy. In other words, it will be able to predict better whether the product will be a success or not. But sometimes it would prove futile if the company receives this data after the product has been launched or it may be even forced to extend the launch date waiting for the arrival of such data!<\/p>\n
Yes you guessed it! With Kinesis, the company will be able to analyze the markets\u2019 response on the very same day of its promotion launch! Imagine the difference it can make to the future of that product. So data analysis at \u201creal time\u201d will be, in the near future, the fine line between failure and success.<\/p>\n
As described by Amazon, Kinesis is \u201ca fully managed service for real-time processing of streaming data at massive scale\u201d.<\/p>\n
[Learn How does AWS EMR Optimizes Hadoop in the Clouds]<\/p>\n
Nowadays the sources of data are majorly from various kinds of sensors,with no one particular source of data. Kinesis too doesn\u2019t constrain you to any one source. Kinesis can ingest data from a myriad of sources ranging from cell phones to large servers or may be as described as in AWS re:INVENT 2013 \u201cany device that is capable of making a \u201cput\u201d call can be a source of data for Kinesis\u201d. Kinesis is more about quick computations & this quickly computed data can be later stored into other AWS Services like RedShift, DynamoDB, etc. for further analysis.<\/p>\n
How does AWS Kinesis work ?
\nLumberjacks Sending Trees downstream
\nImagine the way in which lumberjacks used to transport lumber \u2013 lumber would be pushed into a stream of fast flowing water reaching all the way down to the end of the stream, fetched from there to be transported to the factories for further processing. Similarly, AWS Kinesis can be considered as a continuously flowing stream. Cell phone devices, servers (similar to lumberjacks), who push the server logs, tweets, stock market data (similar to logs) into the stream & AWS Kinesis, implementing the Kinesis Client Library (we will be discussing this in our further blogs) will be at the end the stream ready to process the data as and when it comes.<\/p>\n
As mentioned earlier, Kinesis is a managed service, therefore, the user does not have to think about how to process data simultaneously but concentrate more on the logic of processing this data. Plug-in your code for processing the data and let Kinesis handle the processing for you. A few scenarios in which Kinesis can be used:<\/p>\n
Server logs can be ingested at real time: Here dashboards can be created providing various vital statistics of the servers, at real-time, vital to many IT businesses.
\nClick streams from websites: Customers can be provided with real time analytics of their websites
\nAdvertisement campaigns: Effectiveness can be known at real time.
\nStock market firms: can make use of the real time data coming in, helping them know the trends of the industry.
\nWe will be publishing blogs that will talk about details on how Kinesis works, concepts like \u201cShards\u201d, Kinesis Client library \u2013 their implementation, examples on Kinesis applications, etc. Subscribe to Our Blog & Stay Tuned to Our Kinesis Blog Series & More!<\/p>\n","protected":false},"excerpt":{"rendered":"
AWS Cloud\u2019s Kinesis \u201cIt is a capital mistake to theorize before one has data.\u201d \u2013 Sherlock Holmes Just a few days back I overheard someone saying, \u201cYou know the world no longer revolves around the sun but DATA\u201d. Well, I couldn\u2019t disagree. Today, the whole world revolves around data. But having data alone is of […]<\/p>\n","protected":false},"author":192,"featured_media":17094,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1368],"tags":[2774,2739,2776,2778],"class_list":["post-29705","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data-on-aws-asean","tag-amazon-kinesis-asean","tag-aws-kinesis-asean","tag-aws-kinesis-real-time-big-data-analytics-tool-asean","tag-big-data-on-aws-asean"],"acf":[],"yoast_head":"\n