Hive+mysql安装文档格式.docx
- 文档编号:22830361
- 上传时间:2023-02-05
- 格式:DOCX
- 页数:10
- 大小:20.18KB
Hive+mysql安装文档格式.docx
《Hive+mysql安装文档格式.docx》由会员分享,可在线阅读,更多相关《Hive+mysql安装文档格式.docx(10页珍藏版)》请在冰豆网上搜索。
[root@hadoop-masterhadoop]#source/etc/profile
Metastore
metastore是Hive元数据集中存放地。
它包括两部分:
服务和后台数据存储。
有三种方式配置metastore:
内嵌metastore、本地metastore以及远程metastore。
本次搭建中采用MySQL作为远程仓库,部署在hadoop-master节点上,hive服务端也安装在hive-master上,hive客户端即hadoop-slave访问hive服务器。
MySQL安装
安装依赖包
#yuminstallgccgcc-c++ncurses-devel-y
安装cmake
#wgethttp:
//www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz
#tarzxvfcmake-2.8.12.tar.gz
#cdcmake-2.8.12
#./bootstrap
#make&
&
makeinstall
创建用户的相应目录
#groupaddmysql
#useradd-gmysqlmysql
#mkdir-p/data/mysql/
#mkdir-p/data/mysql/data/
#mkdir-p/data/mysql/log/
获取MySQL安装包并安装
6
7
8
9
10
11
12
13
14
15
16
17
#wget
#tarzxvfmysql-5.6.25.tar.gz
#cdmysql-5.6.25
#cmake\
-DCMAKE_INSTALL_PREFIX=/data/mysql\
-DMYSQL_UNIX_ADDR=/data/mysql/mysql.sock\
-DDEFAULT_CHARSET=utf8\
-DDEFAULT_COLLATION=utf8_general_ci\
-DWITH_INNOBASE_STORAGE_ENGINE=1\
-DWITH_ARCHIVE_STORAGE_ENGINE=1\
-DWITH_BLACKHOLE_STORAGE_ENGINE=1\
-DMYSQL_DATADIR=/data/mysql/data\
-DMYSQL_TCP_PORT=3306\
-DENABLE_DOWNLOADS=1
如果报错找不到CMakeCache.txt则说明没安装ncurses-devel
修改目录权限
#chmod+w/data/mysql/
#chown-Rmysql:
mysql/data/mysql/
#ln-s/data/mysql/lib/libmysqlclient.so.18/usr/lib/libmysqlclient.so.18
#ln-s/data/mysql/mysql.sock/tmp/mysql.sock
初始化数据库
#cp/data/mysql/support-files/my-f/etc/f
#cp/data/mysql/support-files/mysql.server/etc/init.d/mysqld
#/data/mysql/scripts/mysql_install_db--user=mysql--defaults-file=/etc/f--basedir=/data/mysql--datadir=/data/mysql/data
启动MySQL服务
#chmod+x/etc/init.d/mysqld
#servicemysqldstart
#ln–s/data/mysql/bin/mysql/usr/bin/
初始化密码
#mysql-uroot-h127.0.0.1-p
mysql>
SETPASSWORD=PASSWORD('
123456'
);
创建Hive用户
CREATEUSER'
hive'
IDENTIFIEDBY'
;
GRANTALLPRIVILEGESON*.*TO'
@'
hadoop-master'
WITHGRANTOPTION;
flushprivileges;
Hive用户登录
[hadoop@hadoop-master~]mysql-hhadoop-master-uhive
setpassword=password('
创建Hive数据库
createdatabasehive;
配置Hive
修改配置文件
进入到hive的配置文件目录下,找到hive-default.xml.template,cp份为hive-default.xml
另创建hive-site.xml并添加参数
18
19
20
21
22
23
24
25
26
[hadoop@hadoop-masterconf]$pwd
/home/hadoop/apache-hive-1.2.1-bin/conf
[hadoop@hadoop-masterconf]$vihive-site.xml
<
configuration>
<
property>
name>
javax.jdo.option.ConnectionURL<
/name>
value>
jdbc:
mysql:
//hadoop-master:
3306/hive?
createDatabaseIfNotExist=true<
/value>
description>
JDBCconnectstringforaJDBCmetastore<
/description>
/property>
javax.jdo.option.ConnectionDriverName<
com.mysql.jdbc.Driver<
DriverclassnameforaJDBCmetastore<
javax.jdo.option.ConnectionUserName<
hive<
usernametouseagainstmetastoredatabase<
javax.jdo.option.ConnectionPassword<
passwordtouseagainstmetastoredatabase<
/configuration>
JDBC下载
上面网址失效,使用下面的
apache-hive-1.2.1-bindfshadoop-2.7.1Hsourcetmp
[hadoop@hadoop-master~]$cpmysql-connector-java-5.1.33-bin.jarapache-hive-1.2.1-bin/lib/
Hive客户端配置
[hadoop@hadoop-master~]$scp-rapache-hive-1.2.1-bin/hadoop@hadoop-slave:
/home/hadoop
[hadoop@hadoop-slaveconf]$vihive-site.xml
<
hive.metastore.uris<
thrift:
9083<
Hive启动
要启动metastore服务
[hadoop@hadoop-master~]$hive--servicemetastore&
[hadoop@hadoop-master~]$jps
10288RunJar#多了一个进程
9365NameNode
9670SecondaryNameNode
11096Jps
9944NodeManager
9838ResourceManager
9471DataNode
Hive服务器端访问
[hadoop@hadoop-master~]$hive
Logginginitializedusingconfigurationinjar:
file:
/home/hadoop/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!
/hive-log4j.properties
hive>
showdatabases;
OK
default
src
Timetaken:
1.332seconds,Fetched:
2row(s)
usesrc;
0.037seconds
createtabletest1(idint);
0.572seconds
showtables;
abc
test
test1
0.057seconds,Fetched:
3row(s)
Hive客户端访问
27
[hadoop@hadoop-slaveconf]$hive
Logginginitializedusingconfigurationinjar:
1.022seconds,Fetched:
2row(s)
0.057seconds
0.218seconds,Fetched:
3row(s)
createtabletest2(idint,namestring);
5.518seconds
test2
0.102seconds,Fetched:
4row(s)
好了,测试完毕,已经安装成功了。
安装问题纠错
Hive数据库编码问题
错误描述:
hive进入后可以创建数据库,但是无法创建表
createtabletable_test(idstring,namestring);
FAILED:
ExecutionError,returncode1fromorg.apache.hadoop.hive.ql.exec.DDLTask.MetaException(message:
javax.jdo.JDODataStoreException:
Anexceptionwasthrownwhileadding/validatingclass(es):
Specifiedkeywastoolong;
maxkeylengthis767bytes
解决办法:
登录mysql修改下hive数据库的编码方式
alterdatabasehivecharactersetlatin1;
防火墙问题
Hive服务器开启了iptables服务,hive本机可以访问hive服务,hive的客户端hadoop-slave访问报错
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
Exceptioninthread"
main"
java.lang.RuntimeException:
Unabletoinstantiateorg.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:
522)
atorg.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:
677)
atorg.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:
621)
atsun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)
atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
62)
atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43)
atjava.lang.reflect.Method.invoke(Method.java:
483)
atorg.apache.hadoop.util.RunJar.run(RunJar.java:
221)
atorg.apache.hadoop.util.RunJar.main(RunJar.java:
136)
Causedby:
atorg.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:
1523)
atorg.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<
init>
(RetryingMetaStoreClient.java:
86)
atorg.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:
132)
104)
atorg.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:
3005)
atorg.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:
3024)
503)
...8more
java.lang.reflect.InvocationTargetException
atsun.reflect.NativeConstructorAccessorImpl.newInstance0(NativeMethod)
atsun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:
atsun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:
45)
atjava.lang.reflect.Constructor.newInstance(Constructor.java:
408)
atorg.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:
1521)
...14more
MetaException(message:
CouldnotconnecttometastoreusinganyoftheURIsprovided.Mostrecentfailure:
org.apache.thrift.transport.TTransportException:
.NoRouteToHostException:
Noroutetohost
atorg.apache.thrift.transport.TSocket.open(TSocket.java:
187)
atorg.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:
420)
atorg.apache.hadoop.hive.metastore.HiveMetaStore
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- Hive mysql 安装
![提示](https://static.bdocx.com/images/bang_tan.gif)