Shark, Real-time queries and analytics on big data

Shark, Real-time queries and analytics on big data

Shark is an interactive SQL system for Hadoop that claims to provide blazing fast (even real-time) performance that is comparable to MPP databases. It is highly-scalable system that works on top of  Spark that includes features for data co-partitioning, fault tolerance, and even the integration of machine learning. Shark supports many Hive data formats as well as HDFS, HBase, and Amazon S3.

Documentation for Shark is available Github. According to the project website, it takes around 5 mins to set up Shark on a single node for a quick spin, and about 20 mins on an Amazon EC2 cluster.

Advertisements
Tagged with: , , , , , ,
Posted in analytics, big data, hadoop, technology
One comment on “Shark, Real-time queries and analytics on big data
  1. svulli says:

    It would be really interesting to see how well it performs with datasets stored on S3.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: