Today's Question:  What does your personal desk look like?        GIVE A SHOUT

 BIG DATA


  Hologres vs AWS Redshift

Hologres and Redshift are both data warehousing solutions, but they have some differences in terms of features, architecture, and target use cases.Underlying Infrastructure Hologres: Built on Alibaba Cloud's Apsara distributed computing platform, Hologres leverages the underlying infrastructure for storage, computation, and management. It benefits from Alibaba's expertise in cloud-native architecture and real-time data processing. Redshift: Amazon Redshift is based on a Massively Parallel Processing (MPP) architecture, designed to distribute and parallelize queries across multiple nodes for fa...

323 0       REDSHIFT REAL-TIME HOLOGRES AWS BIG DATA ALIBABA


  Cracking the Data Lineage Code

What is Data Lineage? Data lineage describes the life-cycle of data, from its origins to how it is manipulated over time until it reaches its present form. The lineage explains the various processes involved in the data flow of an organization and the factors that influence each process. In other words, data lineage provides data about your data. Data lineage helps organizations of all sizes handle Big Data, as finding the creation point of the data and its evolution provides valuable insights.Almost every decision can be helped by data lineage, from a software engineer choosing what...

1,152 0       BUSINESS BIG DATA DATA LINEAGE


  How Kafka achieves high throughput low latency

Kafka is a message streaming system with high throughput and low latency. It is widely adopted in lots of big companies. A well configured Kafka cluster can achieve super high throughput with millions of concurrent writes. How Kafka can achieve this? This post will try to explain some technologies used by Kafka.Page cache + Disk sequential writeEvery time when Kafka receives a record, it will write it to disk file eventually. But if it writes to disk every time it receives a record, it would not have very good performance. In fact, Kafka has a fantastic design here which is it utilizes the pag...

8,372 0       KAFKA BIG DATA


  Why Most of us Get Confuse With Data Quality Solutions and Bad Data?

How to fix this misunderstanding is what Big Data professionals will explain in this post.The C-level executives are using data collected by their BI and analytics initiatives to make strategic decisions to offer the company a competitive advantage. The case gets worse if the data is inaccurate or incorrect. It’s because the big data helps the company to make big bets, and it impacts the direction and future together. Bad Data can yield inappropriate results and losses.Some interesting facts and statistics about big data, data warehousing, and data quality-90% of US companies are applyin...

1,272 0       BIGDATA


  Data Scientists and Their Harder Skills than Big Data

The field of data science is often confused with that of big data. Data science is an aid to decision makers in a company with a logical approach. Who is a Data Scientist? A Data Scientist reviews a huge collection of data(that may extend to a couple of terabytes of disk space or thousands of excel sheets). This humongous chunk of data is not feasible for being handled, sorted and analyzed by a single person.Here we require the help of data science, and most recently, the field of Artificial Intelligence has gained considerate limelight. With the use of efficient algorithms we can so...

3,267 0       BIG DATA


  How Google Utilizes Big Data for SERP

Google is an expert when it comes to big data. This is evident in their development of various techniques and open source tools which are used by the big data industry professionals. These tools and technique allow Google to sift through millions of different websites and enormous amounts of data in order to provide users with correct answers in a matter of milliseconds. But how does Google accomplish that with such precision? To answer that, we need to focus on the complex activities that go on behind every search query.Entering the search queryGoogle has always wanted to make a search engine...

2,212 0       GOOGLE BIG DATA


  Make Big Data Collection Efficient with Hadoop Architecture and Design Tools

Hadoop architecture and design is popular to spread small array of code to large number of computers. That is why big data collection can be made more efficient with hadoop architecture and design. Hadoop is an open source system where you are free to make changes and design new tools according to your business requirement. Here we will discuss most popular tools under the category Hadoop development and how they are helpful for big projects.Ambari and Hive– When you are designing a cluster, there is plenty of repetitive tasks take lots of efforts and time. Now Hadoop architecture a...

4,928 0       HADOOP ARCHITECTURE HADOOP HIVE ARCHITECTURE HADOOP ARCHITECTURE AND DESIGN


  Spurring the Consumer Feedback Loop with Connected Devices

In a press release from earlier this year, Gartner had predicted that by the year 2018 mobile devices would account for initiating 5% of consumer services cases, registering a marginal rise of 0.02% from 2014.Research shows that most businesses lose around a whopping $83 billion owing to poor consumer services (Source: kissmetrics) in the US alone while globally, the average cost of losing a consumer is $243. It is only viable to think of automating support services as a way to lower the costs of losing a consumer and in a way contributing towards improving the revenues. Although some e-commer...

2,060 0       BIG DATA ANALYTICS SOLUTIONS LOYALTY PROGRAMS FOR CUSTOMERS