1 Hive Metastore
1.1相關概念
Hive Metastore有三種配置方式，分別是：
Embedded Metastore Database (Derby)內嵌模式
Local Metastore Server本地元存儲
Remote Metastore Server遠程元存儲
1.1 Metadata、Metastore作用
metadata即元數據。元數據包含用Hive創建的database、tabel等的元信息。
元數據存儲在關系型數據庫中。如Derby、MySQL等。
Metastore的作用是：客戶端連接metastore服務，metastore再去連接MySQL數據庫來存取元數據。有了metastore服務，就可以有多個客戶端同時連接，而且這些客戶端不需要知道MySQL數據庫的用戶名和密碼，只需要連接metastore 服務即可。
1.2三種配置方式區別
內嵌模式使用的是內嵌的Derby數據庫來存儲元數據，也不需要額外起Metastore服務。這個是默認的，配置簡單，但是一次只能一個客戶端連接，適用于用來實驗，不適用于生產環境。
本地元存儲和遠程元存儲都采用外部數據庫來存儲元數據，目前支持的數據庫有：MySQL、Postgres、Oracle、MS SQL Server.在這里我們使用MySQL。
本地元存儲和遠程元存儲的區別是：本地元存儲不需要單獨起metastore服務，用的是跟hive在同一個進程里的metastore服務。遠程元存儲需要單獨起metastore服務，然后每個客戶端都在配置文件里配置連接到該metastore服務。遠程元存儲的metastore服務和hive運行在不同的進程里。
在生產環境中，建議用遠程元存儲來配置Hive Metastore。
1.3配置文件相關：
配置文件為hivemetastore-site.xml?或在hive-site.xml
hive.metastore.urisHive connects to one of these URIs to make metadata requests to a remote Metastore (comma separated list of URIs)
javax.jdo.option.ConnectionURLJDBC connection string for the data store which contains metadata
javax.jdo.option.ConnectionDriverNameJDBC Driver class name for the data store which contains metadata
hive.metastore.locallocal or remote metastore (removed as of Hive 0.10: If hive.metastore.uris is empty local mode is assumed, remote otherwise)
hive.metastore.warehouse.dirURI of the default location for native tables
javax.jdo.option.ConnectionUserName<user name>
javax.jdo.option.ConnectionPassword<password>
Data Nucleus Auto Start
Configuring datanucleus.autoStartMechanism is highly recommended
Configuring auto start for data nucleus is highly recommended. See HIVE-4762 for more details.
<property>
???<name>datanucleus.autoStartMechanism</name>
???<value>SchemaTable</value>
?</property>
2 Hive 數據存儲
Hive結構和常規的數據庫差別不大，也是下面多個數據庫，每個數據庫下面多個表格
數據存儲位置配置在
?<property>
????<name>hive.metastore.warehouse.dir</name>
????<value>/user/hive/warehouse</value>
????<description>location of default database for the warehouse</description>
??</property>
Hive數據存儲的路徑為/user/hive/warehouse，該路徑為hdfs的路徑，其具體路徑配置在
$HADOOP_HOME/etc/hadoop中的配置文件core-site.xml和hdfs-site.xml
core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/home/leesf/program/hadoop/tmp</value>
<description>Abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
hdfs-site.xml配置文件內容
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/leesf/program/hadoop/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/leesf/program/hadoop/tmp/dfs/data</value>
</property>
<property>
??<name>dfs.http.address</name>
??<value>192.168.65.128:50070</value>
</property>
</configuration>
3 Hive 操作
1.1:查看所有的數據庫: hive>show databases;
1.2:使用數據庫default; hive>use default;
1.3:查看數據庫信息: hive>describe database default;
1.4:顯示的展示當前使用的數據庫:hive>set hive.cli.print.current.db=true;
1.5:Hive顯示表中標題行: hive>set hive.cli.print.header=true;
1.6:創建數據庫命令: hive>create database test;
1.7:切換當前的數據庫: hive>use test;
4 hiveserver2和beeline
連接hive有兩種方式
Hive cli和hiveserver2 beeline
$ $HIVE_HOME/bin/hive
或者
?$ $HIVE_HOME/bin/hiveserver2
?$ $HIVE_HOME/bin/beeline -u jdbc:hive2://$HS2_HOST:$HS2_PORT
?HiveCLI is now deprecated in favor of Beeline, as it lacks the multi-user, security, and other capabilities of HiveServer2.
Hivecli已經不推薦使用了，因為不支持多用戶，安全性，以及一些其他的屬性。
使用beeline連接的時候，如果出現錯誤：
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous (state=08S01,code=0)
則需要修改hadoop的core-site.xml添加如下內容，并重啟服務器。
<property>
???<name>hadoop.proxyuser.root.hosts</name>
???<value>*</value>
</property>
<property>
???<name>hadoop.proxyuser.root.groups</name>
???<value>*</value>
</property>
其中紅色標注的部分，是你使用beeline連接的用戶名
Connection URL for Remote or Embedded Mode
The JDBC connection URL format has the prefix jdbc:hive2:// and the Driver class is org.apache.hive.jdbc.HiveDriver. Note that this is different from the old HiveServer.
For a remote server, the URL format is jdbc:hive2://<host>:<port>/<db>;initFile=<file> (default port for HiveServer2 is 10000).
For an embedded server, the URL format is jdbc:hive2:///;initFile=<file> (no host or port).
4.2使用hiveserver2 配置用戶名和密碼
Hive-site.xml中配置
?<property>
????<name>hive.server2.authentication</name>
????<value>NONE</value>
????<description>
??????Expects one of [nosasl, none, ldap, kerberos, pam, custom].
??????Client authentication types.
????????NONE: no authentication check
????????LDAP: LDAP/AD based authentication
????????KERBEROS: Kerberos/GSSAPI authentication
????????CUSTOM: Custom authentication provider
????????????????(Use with property hive.server2.custom.authentication.class)
????????PAM: Pluggable authentication module
????????NOSASL: ?Raw transport
????</description>
??</property>
配置為CUSTOM，之后自己編寫java驗證類，打包成jar文件，放置在hive的lib目錄下，并配置
<property>
????<name>hive.server2.custom.authentication.class</name>
????<value>test.SampleAuthenticator </value>
????<description>
??????Custom authentication class. Used when property
??????'hive.server2.authentication' is set to 'CUSTOM'. Provided class
??????must be a proper implementation of the interface
??????org.apache.hive.service.auth.PasswdAuthenticationProvider. HiveServer2
??????will call its Authenticate(user, passed) method to authenticate requests.
??????The implementation may optionally implement Hadoop's
??????org.apache.hadoop.conf.Configurable class to grab Hive's Configuration object.
????</description>
??</property>
以下代碼為java驗證類
package test;
import java.util.Hashtable;
import javax.security.sasl.AuthenticationException;
import org.apache.hive.service.auth.PasswdAuthenticationProvider;
/*
?javac -cp $HIVE_HOME/lib/hive-service-0.12.0-cdh5.0.0-beta-2.jar SampleAuthenticator.java -d .
?jar cf sampleauth.jar hive
?cp sampleauth.jar $HIVE_HOME/lib/.
*/
public class SampleAuthenticator implements PasswdAuthenticationProvider {
??Hashtable<String, String> store = null;
??public SampleAuthenticator () {
????store = new Hashtable<String, String>();
????store.put("user1", "passwd1");
????store.put("user2", "passwd2");
??}
??@Override
??public void Authenticate(String user, String ?password)
??????throws AuthenticationException {
????String storedPasswd = store.get(user);
????if (storedPasswd != null && storedPasswd.equals(password))
??????return;
????throw new AuthenticationException("SampleAuthenticator: Error validating user");
??}
}
