hql語法參考文章:http://blog.csdn.net/acmilanvanbasten/article/details/17252673
一、單表select
1、and、sort by 、limit的使用
hive> select * from weather where city='hangzhou' and weath='fine' and minTemperat=-16 sort by pmvalue DESC limit 5;
//查詢城市是hangzhou,天氣fine並且最低溫度是-16° 並且pm值最大的5條記錄,結果如下
2014-05-23|07:34:58 China hangzhou fine -16 -10 496
2014-05-23|07:34:58 China hangzhou fine -16 -6 496
2014-05-23|07:34:58 China hangzhou fine -16 14 496
2014-05-23|07:34:58 China hangzhou fine -16 0 496
2014-05-23|07:34:58 China hangzhou fine -16 -8 496
Time taken: 29.266 seconds, Fetched: 5 row(s)
2、in
hive> select * from weather where city='hangzhou' and weath='fine' and minTemperat in (-16,-17) sort by pmvalue DESC limit 10;
2014-05-23|07:34:57 China hangzhou fine -17 -12 498
2014-05-23|07:34:57 China hangzhou fine -17 10 498
2014-05-23|07:34:57 China hangzhou fine -17 -14 498
2014-05-23|07:34:58 China hangzhou fine -17 -7 496
2014-05-23|07:34:58 China hangzhou fine -16 -6 496
2014-05-23|07:34:58 China hangzhou fine -16 6 496
2014-05-23|07:34:58 China hangzhou fine -16 -12 496
2014-05-23|07:34:58 China hangzhou fine -16 1 496
2014-05-23|07:34:58 China hangzhou fine -17 2 496
2014-05-23|07:34:58 China hangzhou fine -17 11 496
Time taken: 29.277 seconds, Fetched: 10 row(s)
3、group by
select * from weather where city='hangzhou' and weath='fine'
and minTemperat in (-16,-17) group by pmvalue;
4、把select的結果輸出到文件中:
INSERT OVERWRITE LOCAL DIRECTORY '/home/hadoop/selectResult'
select * from weather where city='hangzhou' and weath='fine' and
minTemperat in (-16,-17) sort by pmvalue DESC limit 100;
二、多表聯合select
ps:準備工作,再創建兩張表,一張是城市名稱與區號的對照表,一張是pm值對應的空氣狀況表,表結構如下:
create table cityinfo
(name string, number string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ' '
STORED AS TEXTFILE;
create table pminfo
(pmvalue string, pmlevel string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ' '
STORED AS TEXTFILE;
1、3表聯合查詢,返回所有字段,前5個結果
select *
from cityinfo join weather on (cityinfo.name=weather.city)
join pminfo on (pminfo.pmvalue=weather.pmvalue)
where city='hangzhou' and weath='fine' and minTemperat=-16 limit 5
Job 1: Map: 2 Reduce: 1 Cumulative CPU: 3.34 sec HDFS Read: 229134 HDFS Write: 336 SUCCESS
Total MapReduce CPU Time Spent: 9 seconds 300 msec
OK
hangzhou 0571 2014-05-23|07:33:59 China hangzhou fine -16 11 0 0 A
hangzhou 0571 2014-05-23|07:35:44 China hangzhou fine -16 -5 0 0 A
hangzhou 0571 2014-05-23|07:35:44 China hangzhou fine -16 -14 0 0 A
hangzhou 0571 2014-05-23|07:35:44 China hangzhou fine -16 12 0 0 A
hangzhou 0571 2014-05-23|07:35:44 China hangzhou fine -16 18 0 0 A
Time taken: 34.448 seconds, Fetched: 5 row(s)
2、3表聯合查詢,使用別名,只返回部分列
select cy.number,wh.*,pm.pmlevel
from cityinfo cy join weather wh on (cy.name=wh.city)
join pminfo pm on (pm.pmvalue=wh.pmvalue)
where wh.city='hangzhou' and wh.weath='fine' and wh.minTemperat=-16 limit 5
Total MapReduce CPU Time Spent: 6 seconds 790 msec
OK
0571 2014-05-23|07:33:59 China hangzhou fine -16 11 0 A
0571 2014-05-23|07:35:44 China hangzhou fine -16 -5 0 A
0571 2014-05-23|07:35:44 China hangzhou fine -16 -14 0 A
0571 2014-05-23|07:35:44 China hangzhou fine -16 12 0 A
0571 2014-05-23|07:35:44 China hangzhou fine -16 18 0 A
3、LEFT,RIGHT 和 FULLOUTER 、聚合等高級特性
//待續
補充知識:join時mp如何工作?
join 時,每次map/reduce 任務的邏輯:
reducer 會緩存 join 序列中除了最後一個表的所有表的記錄,再通過最後一個表將結果序列化到文件系統。這一實現有助於在 reduce 端減少內存的使用量。實踐中,應該把最大的那個表寫在最後(否則會因爲緩存浪費大量內存)。例如:
SELECT a.val, b.val, c.val FROM a
JOIN b ON (a.key = b.key1)JOIN c ON (c.key = b.key1)
所有表都使用同一個 join key(使用 1 次map/reduce 任務計算)。Reduce 端會緩存 a 表和 b 表的記錄,然後每次取得一個 c 表的記錄就計算一次 join 結果,類似的還有:
SELECT a.val, b.val, c.val FROMa
JOIN b ON (a.key = b.key1)JOIN c ON (c.key = b.key2)
這裏用了 2 次 map/reduce 任務。第一次緩存 a 表,用 b 表序列化[王黎17] ;第二次緩存第一次 map/reduce 任務的結果,然後用 c 表序列化。
hive hql 語法參考網址:http://blog.csdn.net/hguisu/article/details/7256833