HIVE- TO SIMPLIFY HADOOP
Hive is a component of Hortonworks Data Platform(HDP). Hive provides a SQL-like interface to data stored in HDP. In the previous tutorial we used Pig which is a scripting language with a focus on dataflows. Hive provides a database query interface to Apache Hadoop.
People often ask why do Pig and Hive exist when they seem to do much of the same thing. Hive because of its SQL like query language is often used as the interface to an Apache Hadoop based data warehouse. Hive is considered friendlier and more familiar to users who are used to using SQL for querying data. Pig fits in through its data flow strengths where it takes on the tasks of bringing data into Apache Hadoop and working with it to get it into the form for querying. A good overview of how this works is in Alan Gates posting on the Yahoo Developer blog titled Pig and Hive at Yahoo! From a technical point of view both Pig and Hive are feature complete so you can do tasks in either tool. However you will find one tool or the other will be preferred by the different groups that have to use Apache Hadoop. The good part is they have a choice and both tools work together.
This file has all the statistics from 1871–2011 and contains more than 90,000 rows.
Input file path:
Step 1 – Load input file:
We need to unzip it into a directory.We will be uploading files from the dataset in “file browser” like below
In Hue there is a button called “Hive” and inside Hive there are query options like “Query Editor”, “My Queries” and “Tables” etc.
On left there is a “query editor”. A query may span multiple lines, there are buttons to Execute the query, Explain the query, Save the query with a name and to open a new window for another query.
Step 2 – Create empty table and load data in Hive
In “Table” we need to select “Create a new table from a file”, which will lead us to the “file browser”, where we will select “batting.csv” file and we will name the new table as “temp_batting
Data has been loaded, the file (batting.csv) will be deleted by HIVE.
We execute the following command, and this will show us the first 100 rows from the table.
Step 3 – Creating a batting table and transfer data from the temporary table to batting table
Step 4 – Create a query to show the highest score per year
We will create a query to show the highest score per year by using "group by".
SELECT year, max(runs) FROM batting GROUP BY year;
Step 5- Get final result
We will execute final query which will show the player who scored the maximum runs in a year.
No comments:
Post a Comment