1、概念
窗口分析函數：窗口函數也稱為OLAP（OnlineAnalytical Processing）函數，是對一組值進行操作，不需要使用Group by子句對數據進行分組，還能在同一行返回原來行的列和使用聚合函數得到的聚合列。
官網：
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics
2、sum, avg, max, min
數據準備：cookie.txt
cookie1,2015-04-10,1
cookie1,2015-04-11,5
cookie1,2015-04-12,7
cookie1,2015-04-13,3
cookie1,2015-04-14,2
cookie1,2015-04-15,4
cookie1,2015-04-16,4
建表準備相關：
create database if not exists myhive;
use myhive;
drop table if exists cookie;
create table cookie(cookieid string, createtime string, pv int) row format
delimited fields terminated by ',';
load data local inpath "/home/bigdata/cookie.txt" into table cookie;
select * from cookie;
sum
SELECT cookieid,createtime,pv,
SUM(pv) OVER(PARTITION BY cookieid ORDER BY createtime) AS pv1, -- 默認為從起點到
當前行
SUM(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN UNBOUNDED
PRECEDING AND CURRENT ROW) AS pv2, --從起點到當前行，結果同pv1
SUM(pv) OVER(PARTITION BY cookieid) AS pv3, --分組內所有行
SUM(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3 PRECEDING
AND CURRENT ROW) AS pv4, --當前行+往前3行
SUM(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3 PRECEDING
AND 1 FOLLOWING) AS pv5, --當前行+往前3行+往后1行
SUM(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN CURRENT ROW
AND UNBOUNDED FOLLOWING) AS pv6 --當前行+往后所有行
FROM cookie order by cookieid, createtime;
結果：
cookie1 2015-04-10 1 1 1 26 1 6 26
cookie1 2015-04-11 5 6 6 26 6 13 25
cookie1 2015-04-12 7 13 13 26 13 16 20
cookie1 2015-04-13 3 16 16 26 16 18 13
cookie1 2015-04-14 2 18 18 26 17 21 10
cookie1 2015-04-15 4 22 22 26 16 20 8
cookie1 2015-04-16 4 26 26 26 13 13 4
解釋：
pv1: 分組內從起點到當前行的pv累積，如，11號的pv1=10號的pv+11號的pv, 12號=10號+11號+12號
pv2: 同pv1
pv3: 分組內(cookie1)所有的pv累加
pv4: 分組內當前行+往前3行，11號=10號+11號， 12號=10號+11號+12號， 13號=10號+11號+12號+13號， 14號=11號+12號+13號+14號
pv5: 分組內當前行+往前3行+往后1行，如，14號=11號+12號+13號+14號+15號=5+7+3+2+4=21
pv6: 分組內當前行+往后所有行，如，13號=13號+14號+15號+16號=3+2+4+4=13，14號=14號+15號+16號=2+4+4=10
擴展：
如果不指定ROWS BETWEEN,默認為從起點到當前行;
如果不指定ORDER BY，則將分組內所有值累加;
關鍵是理解ROWS BETWEEN含義,也叫做WINDOW子句：
PRECEDING：往前
FOLLOWING：往后
CURRENT ROW：當前行
UNBOUNDED：起點，UNBOUNDED PRECEDING 表示從前面的起點， UNBOUNDED FOLLOWING：表示到后面的終點
avg，min，max，和 sum
其他AVG，MIN，MAX，和SUM用法一樣。只需要把sum函數，改成avg，min，max，sum等就可以。
SELECT cookieid, createtime, pv,
round(AVG(pv) OVER(PARTITION BY cookieid ORDER BY createtime), 2) AS pv1, --默
認為從起點到當前行
round(AVG(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN
UNBOUNDED PRECEDING AND CURRENT ROW), 2) AS pv2, --從起點到當前行，結果同pv1
round(AVG(pv) OVER(PARTITION BY cookieid), 2) AS pv3, --分組內所有行
round(AVG(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3
PRECEDING AND CURRENT ROW), 2) AS pv4, --當前行+往前3行
round(AVG(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3
PRECEDING AND 1 FOLLOWING), 2) AS pv5, --當前行+往前3行+往后1行
round(AVG(pv) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN
CURRENT ROW AND UNBOUNDED FOLLOWING), 2) AS pv6 --當前行+往后所有行
FROM cookie;
執行結果：
cookie1 2015-04-16 4 3.71 3.71 3.71 3.25 3.25 4.0
cookie1 2015-04-15 4 3.67 3.67 3.71 4.0 4.0 4.0
cookie1 2015-04-14 2 3.6 3.6 3.71 4.25 4.2 3.33
cookie1 2015-04-13 3 4.0 4.0 3.71 4.0 3.6 3.25
cookie1 2015-04-12 7 4.33 4.33 3.71 4.33 4.0 4.0
cookie1 2015-04-11 5 3.0 3.0 3.71 3.0 4.33 4.17
cookie1 2015-04-10 1 1.0 1.0 3.71 1.0 3.0 3.71
3、ntile, row_number, rank, dense_rank
準備數據：cookie2.txt
cookie1,2015-04-10,1
cookie1,2015-04-11,5
cookie1,2015-04-12,7
cookie1,2015-04-13,3
cookie1,2015-04-14,2
cookie1,2015-04-15,4
cookie1,2015-04-16,4
cookie2,2015-04-10,2
cookie2,2015-04-11,3
cookie2,2015-04-12,5
cookie2,2015-04-13,6
cookie2,2015-04-14,3
cookie2,2015-04-15,9
cookie2,2015-04-16,7
建表導入數據相關操作：
create database if not exists myhive;
use myhive;
drop table if exists cookie2;
create table cookie2(cookieid string, createtime string, pv int) row format
delimited fields terminated by ',';
load data local inpath "/home/bigdata/cookie2.txt" into table cookie2;
select * from cookie2;
ntile
NTILE(n)，用于將分組數據按照順序切分成n片，返回當前切片值
NTILE不支持ROWS BETWEEN，比如 NTILE(2) OVER(PARTITION BY cookieid ORDER BY
createtime ROWS BETWEEN 3 PRECEDING AND CURRENT ROW)
如果切片不均勻，默認增加第一個切片的分布
SQL語句實例：
SELECT cookieid,createtime,pv,
NTILE(2) OVER(PARTITION BY cookieid ORDER BY createtime) AS rn1, --分組內將數據
分成2片
NTILE(3) OVER(PARTITION BY cookieid ORDER BY createtime) AS rn2, --分組內將數據
分成3片
NTILE(4) OVER(ORDER BY createtime) AS rn3 --將所有數據分成4片
FROM cookie2
ORDER BY cookieid,createtime;
結果：
cookie1 2015-04-10 1 1 1 1
cookie1 2015-04-11 5 1 1 1
cookie1 2015-04-12 7 1 1 2
cookie1 2015-04-13 3 1 2 2
cookie1 2015-04-14 2 2 2 3
cookie1 2015-04-15 4 2 3 4
cookie1 2015-04-16 4 2 3 4
cookie2 2015-04-10 2 1 1 1
cookie2 2015-04-11 3 1 1 1
cookie2 2015-04-12 5 1 1 2
cookie2 2015-04-13 6 1 2 2
cookie2 2015-04-14 3 2 2 3
cookie2 2015-04-15 9 2 3 3
cookie2 2015-04-16 7 2 3 4
比如，統計一個cookie，pv數最多的前1/3的天
SELECT cookieid, createtime, pv,
NTILE(3) OVER(PARTITION BY cookieid ORDER BY pv DESC) AS rn
FROM cookie2;
結果：
cookie1 2015-04-12 7 1
cookie1 2015-04-11 5 1
cookie1 2015-04-16 4 1
cookie1 2015-04-15 4 2
cookie1 2015-04-13 3 2
cookie1 2015-04-14 2 3
cookie1 2015-04-10 1 3
cookie2 2015-04-15 9 1
cookie2 2015-04-16 7 1
cookie2 2015-04-13 6 1
cookie2 2015-04-12 5 2
cookie2 2015-04-11 3 2
cookie2 2015-04-14 3 3
cookie2 2015-04-10 2 3
其中：rn = 1 的記錄，就是我們想要的結果
row_number
ROW_NUMBER() – 從1開始，按照順序，生成分組內記錄的序列 –比如，按照pv降序排列，生成分組內每天的pv名次，ROW_NUMBER() 的應用場景非常多，再比如，獲取分組內排序第一的記錄;獲取一個session中的第一條refer等。
// 分組排序
SELECT cookieid, createtime, pv,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn
FROM cookie2;
結果：
cookie1 2015-04-12 7 1
cookie1 2015-04-11 5 2
cookie1 2015-04-16 4 3
cookie1 2015-04-15 4 4
cookie1 2015-04-13 3 5
cookie1 2015-04-14 2 6
cookie1 2015-04-10 1 7
cookie2 2015-04-15 9 1
cookie2 2015-04-16 7 2
cookie2 2015-04-13 6 3
cookie2 2015-04-12 5 4
cookie2 2015-04-11 3 5
cookie2 2015-04-14 3 6
cookie2 2015-04-10 2 7
所以如果需要取每一組的前3名，只需要rn<=3即可
rank 和 dense_rank
RANK() 生成數據項在分組中的排名，排名相等會在名次中留下空位
DENSE_RANK() 生成數據項在分組中的排名，排名相等會在名次中不會留下空位
SQL語句實例：
SELECT cookieid, createtime, pv,
RANK() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn1,
DENSE_RANK() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn2,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY pv DESC) AS rn3
FROM cookie2
WHERE cookieid = 'cookie1';
結果：
cookie1 2015-04-12 7 1 1 1
cookie1 2015-04-11 5 2 2 2
cookie1 2015-04-16 4 3 3 3
cookie1 2015-04-15 4 3 3 4
cookie1 2015-04-13 3 5 4 5
cookie1 2015-04-14 2 6 5 6
cookie1 2015-04-10 1 7 6 7
三者對比總結：
row_number 按順序編號，不留空位
rank 按順序編號，相同的值編相同號，留空位
dense_rank 按順序編號，相同的值編相同的號，不留空位
4、cume_dist, percent_rank
數據準備：cookie3.txt
d1,user1,1000
d1,user2,2000
d1,user3,3000
d2,user4,4000
d2,user5,5000
建表導入數據相關操作：
create database if not exists myhive;
use myhive;
drop table if exists cookie3;
create table cookie3(dept string, userid string, sal int) row format delimited
fields terminated by ',';
load data local inpath "/home/bigdata/cookie3.txt" into table cookie3;
select * from cookie3;
cume_dist
–CUME_DIST 小于等于當前值的行數/分組內總行數
比如，統計小于等于當前薪水的人數，所占總人數的比例
SELECT dept, userid, sal,
round(CUME_DIST() OVER(ORDER BY sal), 2) AS rn1,
round(CUME_DIST() OVER(PARTITION BY dept ORDER BY sal), 2) AS rn2
FROM cookie3;
結果：
d1 user1 1000 0.2 0.33
d1 user2 2000 0.4 0.67
d1 user3 3000 0.6 1.0
d2 user4 4000 0.8 0.5
d2 user5 5000 1.0 1.0
SQL語句實例：
SELECT dept, userid, sal,
round(CUME_DIST() OVER(ORDER BY sal), 2) AS rn1,
round(CUME_DIST() OVER(PARTITION BY dept ORDER BY sal desc), 2) AS rn2
FROM cookie3;
結果：
d1 user3 3000 0.6 0.33
d1 user2 2000 0.4 0.67
d1 user1 1000 0.2 1.0
d2 user5 5000 1.0 0.5
d2 user4 4000 0.8 1.0
percent_rank
PERCENT_RANK 分組內當前行的RANK值-1/分組內總行數-1
SQL語句實例：
SELECT dept, userid, sal,
PERCENT_RANK() OVER(ORDER BY sal) AS rn1, --分組內
RANK() OVER(ORDER BY sal) AS rn11, --分組內RANK值
SUM(1) OVER(PARTITION BY NULL) AS rn12, --分組內總行數
PERCENT_RANK() OVER(PARTITION BY dept ORDER BY sal) AS rn2
FROM cookie3;
結果：
d1 user1 1000 0.0 1 5 0.0
d1 user2 2000 0.25 2 5 0.5
d1 user3 3000 0.5 3 5 1.0
d2 user4 4000 0.75 4 5 0.0
d2 user5 5000 1.0 5 5 1.0
SQL語句實例：
SELECT dept, userid, sal,
PERCENT_RANK() OVER(PARTITION BY dept ORDER BY sal) AS rn1, --分組內
RANK() OVER(ORDER BY sal) AS rn11, --分組內RANK值
SUM(1) OVER(PARTITION BY NULL) AS rn12, --分組內總行數
PERCENT_RANK() OVER(PARTITION BY dept ORDER BY sal) AS rn2
FROM cookie3;
執行結果：
d2 user5 5000 1.0 5 5 1.0
d2 user4 4000 0.0 4 5 0.0
d1 user3 3000 1.0 3 5 1.0
d1 user2 2000 0.5 2 5 0.5
d1 user1 1000 0.0 1 5 0.0
5、lag, lead, frist_value, last_value
數據準備：cookie4.txt
cookie1,2015-04-10 10:00:02,url2
cookie1,2015-04-10 10:00:00,url1
cookie1,2015-04-10 10:03:04,1url3
cookie1,2015-04-10 10:50:05,url6
cookie1,2015-04-10 11:00:00,url7
cookie1,2015-04-10 10:10:00,url4
cookie1,2015-04-10 10:50:01,url5
cookie2,2015-04-10 10:00:02,url22
cookie2,2015-04-10 10:00:00,url11
cookie2,2015-04-10 10:03:04,1url33
cookie2,2015-04-10 10:50:05,url66
cookie2,2015-04-10 11:00:00,url77
cookie2,2015-04-10 10:10:00,url44
cookie2,2015-04-10 10:50:01,url55
建表導入數據相關操作：
create database if not exists myhive;
use myhive;
drop table if exists cookie4;
create table cookie4(cookieid string, createtime string, url string) row format
delimited fields terminated by ',';
load data local inpath "/home/bigdata/cookie4.txt" into table cookie4;
select * from cookie4;
lag
LAG(col,n,DEFAULT) 用于統計窗口內往上第n行值
第一個參數為列名，
第二個參數為往上第n行（可選，默認為1），
第三個參數為默認值（當往上第n行為NULL時候，取默認值，如不指定，則為NULL）
SQL語句實例：
SELECT cookieid, createtime, url,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY createtime) AS rn,
LAG(createtime,1,'1970-01-01 00:00:00') OVER(PARTITION BY cookieid ORDER BY
createtime) AS last_1_time,
LAG(createtime,2) OVER(PARTITION BY cookieid ORDER BY createtime) AS last_2_time
FROM cookie4;
結果數據：
cookieid createtime url rn last_1_time
last_2_time
cookie1 2015-04-10 10:00:00 url1 1 1970-01-01 00:00:00 NULL
cookie1 2015-04-10 10:00:02 url2 2 2015-04-10 10:00:00 NULL
cookie1 2015-04-10 10:03:04 1url3 3 2015-04-10 10:00:02 2015-04-10 10:00:00
cookie1 2015-04-10 10:10:00 url4 4 2015-04-10 10:03:04 2015-04-10 10:00:02
cookie1 2015-04-10 10:50:01 url5 5 2015-04-10 10:10:00 2015-04-10 10:03:04
cookie1 2015-04-10 10:50:05 url6 6 2015-04-10 10:50:01 2015-04-10 10:10:00
cookie1 2015-04-10 11:00:00 url7 7 2015-04-10 10:50:05 2015-04-10 10:50:01
cookie2 2015-04-10 10:00:00 url11 1 1970-01-01 00:00:00 NULL
cookie2 2015-04-10 10:00:02 url22 2 2015-04-10 10:00:00 NULL
cookie2 2015-04-10 10:03:04 1url33 3 2015-04-10 10:00:02 2015-04-10 10:00:00
cookie2 2015-04-10 10:10:00 url44 4 2015-04-10 10:03:04 2015-04-10 10:00:02
cookie2 2015-04-10 10:50:01 url55 5 2015-04-10 10:10:00 2015-04-10 10:03:04
cookie2 2015-04-10 10:50:05 url66 6 2015-04-10 10:50:01 2015-04-10 10:10:00
cookie2 2015-04-10 11:00:00 url77 7 2015-04-10 10:50:05 2015-04-10 10:50:01
解釋：
last_1_time: 指定了往上第1行的值，default為'1970-01-01 00:00:00'
cookie1第一行，往上1行為NULL,因此取默認值 1970-01-01 00:00:00
cookie1第三行，往上1行值為第二行值，2015-04-10 10:00:02
cookie1第六行，往上1行值為第五行值，2015-04-10 10:50:01
last_2_time: 指定了往上第2行的值，為指定默認值
cookie1第一行，往上2行為NULL
cookie1第二行，往上2行為NULL
cookie1第四行，往上2行為第二行值，2015-04-10 10:00:02
cookie1第七行，往上2行為第五行值，2015-04-10 10:50:01
lead
與LAG相反
LEAD(col,n,DEFAULT) 用于統計窗口內往下第n行值
第一個參數為列名，
第二個參數為往下第n行（可選，默認為1），
第三個參數為默認值（當往下第n行為NULL時候，取默認值，如不指定，則為NULL）
SQL語句實例：
SELECT cookieid, createtime, url,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY createtime) AS rn,
LEAD(createtime,1,'1970-01-01 00:00:00') OVER(PARTITION BY cookieid ORDER BY
createtime) AS next_1_time,
LEAD(createtime,2) OVER(PARTITION BY cookieid ORDER BY createtime) AS
next_2_time
FROM cookie4;
結果：
cookieid createtime url rn next_1_time
next_2_time
cookie1 2015-04-10 10:00:00 url1 1 2015-04-10 10:00:02 2015-04-
10 10:03:04
cookie1 2015-04-10 10:00:02 url2 2 2015-04-10 10:03:04 2015-04-
10 10:10:00
cookie1 2015-04-10 10:03:04 1url3 3 2015-04-10 10:10:00 2015-04-
10 10:50:01
cookie1 2015-04-10 10:10:00 url4 4 2015-04-10 10:50:01 2015-04-
10 10:50:05
cookie1 2015-04-10 10:50:01 url5 5 2015-04-10 10:50:05 2015-04-
10 11:00:00
cookie1 2015-04-10 10:50:05 url6 6 2015-04-10 11:00:00 NULL
cookie1 2015-04-10 11:00:00 url7 7 1970-01-01 00:00:00 NULL
cookie2 2015-04-10 10:00:00 url11 1 2015-04-10 10:00:02 2015-04-
10 10:03:04
cookie2 2015-04-10 10:00:02 url22 2 2015-04-10 10:03:04 2015-04-
10 10:10:00
cookie2 2015-04-10 10:03:04 1url33 3 2015-04-10 10:10:00 2015-04-
10 10:50:01
cookie2 2015-04-10 10:10:00 url44 4 2015-04-10 10:50:01 2015-04-
10 10:50:05
cookie2 2015-04-10 10:50:01 url55 5 2015-04-10 10:50:05 2015-04-
10 11:00:00
cookie2 2015-04-10 10:50:05 url66 6 2015-04-10 11:00:00 NULL
cookie2 2015-04-10 11:00:00 url77 7 1970-01-01 00:00:00 NULL
first_value
取分組內排序后，截止到當前行，第一個值
SQL語句實例：
SELECT cookieid, createtime, url,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY createtime) AS rn,
FIRST_VALUE(url) OVER(PARTITION BY cookieid ORDER BY createtime) AS first1
FROM cookie4;
結果：
cookie1 2015-04-10 10:00:00 url1 1 url1
cookie1 2015-04-10 10:00:02 url2 2 url1
cookie1 2015-04-10 10:03:04 1url3 3 url1
cookie1 2015-04-10 10:10:00 url4 4 url1
cookie1 2015-04-10 10:50:01 url5 5 url1
cookie1 2015-04-10 10:50:05 url6 6 url1
cookie1 2015-04-10 11:00:00 url7 7 url1
cookie2 2015-04-10 10:00:00 url11 1 url11
cookie2 2015-04-10 10:00:02 url22 2 url11
cookie2 2015-04-10 10:03:04 1url33 3 url11
cookie2 2015-04-10 10:10:00 url44 4 url11
cookie2 2015-04-10 10:50:01 url55 5 url11
cookie2 2015-04-10 10:50:05 url66 6 url11
cookie2 2015-04-10 11:00:00 url77 7 url11
last_value
取分組內排序后，截止到當前行，最后一個值
SQL語句實例:
SELECT cookieid, createtime, url,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY createtime) AS rn,
LAST_VALUE(url) OVER(PARTITION BY cookieid ORDER BY createtime) AS last1
FROM cookie4;
結果數據：
cookie1 2015-04-10 10:00:00 url1 1 url1
cookie1 2015-04-10 10:00:02 url2 2 url2
cookie1 2015-04-10 10:03:04 1url3 3 1url3
cookie1 2015-04-10 10:10:00 url4 4 url4
cookie1 2015-04-10 10:50:01 url5 5 url5
cookie1 2015-04-10 10:50:05 url6 6 url6
cookie1 2015-04-10 11:00:00 url7 7 url7
cookie2 2015-04-10 10:00:00 url11 1 url11
cookie2 2015-04-10 10:00:02 url22 2 url22
cookie2 2015-04-10 10:03:04 1url33 3 1url33
cookie2 2015-04-10 10:10:00 url44 4 url44
cookie2 2015-04-10 10:50:01 url55 5 url55
cookie2 2015-04-10 10:50:05 url66 6 url66
cookie2 2015-04-10 11:00:00 url77 7 url77
問題：如果不指定ORDER BY，則默認按照記錄在文件中的偏移量進行排序，會出現錯誤的結果
// 求得每組的最后一個值： 排倒序，然后取FIRST_VALUE
SQL語句：
SELECT cookieid, createtime, url,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY createtime) AS rn,
LAST_VALUE(url) OVER(PARTITION BY cookieid ORDER BY createtime) AS last1,
FIRST_VALUE(url) OVER(PARTITION BY cookieid ORDER BY createtime DESC) AS
last2
FROM cookie4
ORDER BY cookieid,createtime;
結果：
cookie1 2015-04-10 10:00:00 url1 1 url1 url7
cookie1 2015-04-10 10:00:02 url2 2 url2 url7
cookie1 2015-04-10 10:03:04 1url3 3 1url3 url7
cookie1 2015-04-10 10:10:00 url4 4 url4 url7
cookie1 2015-04-10 10:50:01 url5 5 url5 url7
cookie1 2015-04-10 10:50:05 url6 6 url6 url7
cookie1 2015-04-10 11:00:00 url7 7 url7 url7
cookie2 2015-04-10 10:00:00 url11 1 url11 url77
cookie2 2015-04-10 10:00:02 url22 2 url22 url77
cookie2 2015-04-10 10:03:04 1url33 3 1url33 url77
cookie2 2015-04-10 10:10:00 url44 4 url44 url77
cookie2 2015-04-10 10:50:01 url55 5 url55 url77
cookie2 2015-04-10 10:50:05 url66 6 url66 url77
cookie2 2015-04-10 11:00:00 url77 7 url77 url77
6、 grouping sets, grouping__id, cube,rollup
這幾個分析函數通常用于OLAP中，不能累加，而且需要根據不同維度上鉆和下鉆的指標統計，比如，分小時、天、月的UV數
官網介紹：https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation%2C+Cube%2
C+Grouping+and+Rollup
數據準備：cookie5.txt
2015-03,2015-03-10,cookie1
2015-03,2015-03-10,cookie5
2015-03,2015-03-12,cookie7
2015-04,2015-04-12,cookie3
2015-04,2015-04-13,cookie2
2015-04,2015-04-13,cookie4
2015-04,2015-04-16,cookie4
2015-03,2015-03-10,cookie2
2015-03,2015-03-10,cookie3
2015-04,2015-04-12,cookie5
2015-04,2015-04-13,cookie6
2015-04,2015-04-15,cookie3
2015-04,2015-04-15,cookie2
2015-04,2015-04-16,cookie1
建表導入數據相關：
create database if not exists myhive;
use myhive;
drop table if exists cookie5;
create table cookie5(month string, day string, cookieid string) row format
delimited fields terminated by ',';
load data local inpath "/home/bigdata/cookie5.txt" into table cookie5;
select * from cookie5;
grouping sets
在一個GROUP BY查詢中，根據不同的維度組合進行聚合，等價于將不同維度的GROUP BY結果集進行UNION ALL
SQL語句實例：
SELECT month, day,
COUNT(DISTINCT cookieid) AS uv,
GROUPING__ID
FROM cookie5
GROUP BY month,day
GROUPING SETS (month,day)
ORDER BY GROUPING__ID;
其中的 GROUPING__ID，表示結果屬于哪一個分組集合。
結果：
2015-04 NULL 6 1
2015-03 NULL 5 1
NULL 2015-04-16 2 2
NULL 2015-04-15 2 2
NULL 2015-04-13 3 2
NULL 2015-04-12 2 2
NULL 2015-03-12 1 2
NULL 2015-03-10 4 2
其實這個SQL語句等價于下面這個SQL：
SELECT month,NULL,COUNT(DISTINCT cookieid) AS uv,1 AS GROUPING__ID FROM cookie5
GROUP BY month
UNION ALL
SELECT NULL,day,COUNT(DISTINCT cookieid) AS uv,2 AS GROUPING__ID FROM cookie5
GROUP BY day;
執行結果：
NULL 2015-03-10 4 2
NULL 2015-03-12 1 2
NULL 2015-04-12 2 2
NULL 2015-04-13 3 2
NULL 2015-04-15 2 2
NULL 2015-04-16 2 2
2015-03 NULL 5 1
2015-04 NULL 6 1
SQL語句：
SELECT month, day,
COUNT(DISTINCT cookieid) AS uv,
GROUPING__ID
FROM cookie5
GROUP BY month,day
GROUPING SETS (month,day,(month,day))
ORDER BY GROUPING__ID;
其中的 GROUPING__ID，表示結果屬于哪一個分組集合。
結果數據：
2015-03 2015-03-10 4 0
2015-04 2015-04-16 2 0
2015-04 2015-04-13 3 0
2015-04 2015-04-12 2 0
2015-04 2015-04-15 2 0
2015-03 2015-03-12 1 0
2015-03 NULL 5 1
2015-04 NULL 6 1
NULL 2015-04-16 2 2
NULL 2015-04-15 2 2
NULL 2015-04-13 3 2
NULL 2015-04-12 2 2
NULL 2015-03-12 1 2
NULL 2015-03-10 4 2
等價于：
SELECT month,NULL,COUNT(DISTINCT cookieid) AS uv,1 AS GROUPING__ID FROM cookie5
GROUP BY month
UNION ALL
SELECT NULL,day,COUNT(DISTINCT cookieid) AS uv,2 AS GROUPING__ID FROM cookie5
GROUP BY day
UNION ALL
SELECT month,day,COUNT(DISTINCT cookieid) AS uv,3 AS GROUPING__ID FROM cookie5
GROUP BY month,day;
cube
根據GROUP BY的維度的所有組合進行聚合
SQL語句：
SELECT month, day,
COUNT(DISTINCT cookieid) AS uv,
GROUPING__ID
FROM cookie5
GROUP BY month,day
WITH CUBE
ORDER BY GROUPING__ID;
結果：
2015-03 2015-03-10 4 0
2015-04 2015-04-16 2 0
2015-04 2015-04-13 3 0
2015-04 2015-04-12 2 0
2015-04 2015-04-15 2 0
2015-03 2015-03-12 1 0
2015-03 NULL 5 1
2015-04 NULL 6 1
NULL 2015-04-16 2 2
NULL 2015-04-15 2 2
NULL 2015-04-13 3 2
NULL 2015-04-12 2 2
NULL 2015-03-12 1 2
NULL 2015-03-10 4 2
NULL NULL 7 3
等價于：
SELECT NULL,NULL,COUNT(DISTINCT cookieid) AS uv,0 AS GROUPING__ID FROM cookie5
UNION ALL
SELECT month,NULL,COUNT(DISTINCT cookieid) AS uv,1 AS GROUPING__ID FROM cookie5
GROUP BY month
UNION ALL
SELECT NULL,day,COUNT(DISTINCT cookieid) AS uv,2 AS GROUPING__ID FROM cookie5
GROUP BY day
UNION ALL
SELECT month,day,COUNT(DISTINCT cookieid) AS uv,3 AS GROUPING__ID FROM cookie5
GROUP BY month,day;
rollup
是CUBE的子集，以最左側的維度為主，從該維度進行層級聚合
比如，以month維度進行層級聚合，SQL語句：
SELECT month, day, COUNT(DISTINCT cookieid) AS uv, GROUPING__ID
FROM cookie5
GROUP BY month,day WITH ROLLUP ORDER BY GROUPING__ID;
可以實現這樣的上鉆過程：月天的UV->月的UV->總UV
結果：
2015-04 2015-04-16 2 0
2015-04 2015-04-15 2 0
2015-04 2015-04-13 3 0
2015-04 2015-04-12 2 0
2015-03 2015-03-12 1 0
2015-03 2015-03-10 4 0
2015-04 NULL 6 1
2015-03 NULL 5 1
NULL NULL 7 3
把month和day調換順序，則以day維度進行層級聚合：SQL語句：
SELECT day, month, COUNT(DISTINCT cookieid) AS uv, GROUPING__ID
FROM cookie5
GROUP BY day,month WITH ROLLUP ORDER BY GROUPING__ID;
可以實現這樣的上鉆過程：天月的UV->天的UV->總UV
這里，根據天和月進行聚合，和根據天聚合結果一樣，因為有父子關系，如果是其他維度組合的話，就會不一樣
