「图文」apache-hive-1.2.1-bin 安装

内容导航：

「图文」apache-hive-1.2.1-bin 安装

sqoop导入到hive权限问题，有谁能解决

ubuntu 下载hive时显示error is not recoverable:exiting now?

hadoop伪分布式安装部署HIVE怎么做

一、「图文」apache-hive-1.2.1-bin 安装

apache-hive-1.2.1-bin 安装

更多资源:https://github.com/opensourceteams

技能标签

下载apache hive 安装包进行apache-hive-1.2.1-bin.tar.gz安装配置mysql存储元数据配置HDFS存储 /user/hive/warehouse该Hive版本1.2.1默认集成在(Spark1.6 到 Spark2.4)

下载

下载安装包apache-hive-1.2.1-bin.tar.gz: https://archive.apache.org/dist/hive/hive-1.2.1由于spark-2.4.0-bin-hadoop2.7 预编译好的捆绑hive版本为hive-1.2.1,为了方便所以选用版本apache-hive-1.2.1-bin如果spark支持其它版本的hive,需要重新手动编译spark源码也可以实现

安装

解压安装包

tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /opt/module/bigdata/

配置环境变量

export HIVE_HOME=/opt/module/bigdata/apache-hive-1.2.1-binexport PATH=$HIVE_HOME/lib:#PATH

HDFS新建目录

bin/hadoop fs -mkdir -p /user/hive/warehouse #创建目录bin/hadoop fs -chmod -R 777 /user/hive/warehouse #新建的目录赋予读写权限bin/hadoop fs -mkdir -p /tmp/hive/#新建/tmp/hive/目录bin/hadoop fs -chmod -R 777 /tmp/hive #目录赋予读写权限#用以下命令检查目录是否创建成功bin/hadoop fs -ls /user/hivebin/hadoop fs -ls /tmp/hive

连接mysql数据库

复制jdbc连接驱动

cp mysql-connector-java-8.0.13.jar $HIVE_HOME/lib/

新建hive-site.xml数据库相关的配置

touch hive-site.xml文件

<?xml version="1.0" encoding="UTF-8" standalone="no"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value>  </property>  <property> <name>hive.exec.scratchdir</name> <value>/tmp/hive</value>  </property>  <property> <name>javax.jdo.option.ConnectionURL</name>  <value>jdbc:mysql://macbookmysql.com:3306/hive?createDatabaseIfNotExist=true</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.cj.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>admin</value> <description>username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>000000</value> <description>password to use against metastore database</description> </property></configuration>

新建hive-env.sh

cp hive-env.sh.template

# Licensed to the Apache Software Foundation (ASF) under one# or more contributor license agreements. See the NOTICE file# distributed with this work for additional information# regarding copyright ownership. The ASF licenses this file# to you under the Apache License, Version 2.0 (the# "License"); you may not use this file except in compliance# with the License. You may obtain a copy of the License at## http:///pgc-image/ba9b260ee9644310974b808b4e17db6f~noop.image?_iz=58558&from=article.pc_detail&x-expires=1667472590&x-signature=TOcdbG21tHj%2BSARnoPxgV4vF4F8%3D" img_width="1966" img_height="652" image_type="1" mime_type="image/png" web_uri="pgc-image/ba9b260ee9644310974b808b4e17db6f">
启动hive metastore服务
启动该服务会开一个 9083端口netstat -ntlup 可以查看服务端占用的端口
hive --service metastore & 
停止hive metastore服务
ps -ef|grep hivekill -9 pid
启动hiveserver服务 (该服务可以不启动)
hive --service hiveserver2 & 
操作
启动 hive
hive
退出 hive命令
exit;
创建database
create database;
显示所有的database
show databases;
显示所有的表
show tables;
创建表
CREATE TABLE IF NOT EXISTS employee ( name String, salary String) COMMENT 'Employee details' ROW FORMAT DELIMITED FIELDS TERMINATED BY '' LINES TERMINATED BY '' STORED AS TEXTFILE;
插入数据
数据文件 employee.txt
小明 100000小王 50000小李 60000
插入数据文件到hive上
LOAD DATA LOCAL INPATH '/home/liuwen/temp/employee.txt' OVERWRITE INTO TABLE employee;
查看表数据
 select * from employee;
问题处理
无法访问spark-assembly-*.jar
[liuwen@standalone lib]$ schematool -initSchema -dbType mysqlls: 无法访问/opt/module/bigdata/spark-2.4.0-bin-without-hadoop/lib/spark-assembly-*.jar: 没有那个文件或目录Metastore connection URL: jdbc:mysql://macbookmysql.com:3306/hive?createDatabaseIfNotExist=trueMetastore Connection Driver : com.mysql.cj.jdbc.DriverMetastore connection User: adminStarting metastore schema initialization to 1.2.0Initialization script hive-schema-1.2.0.mysql.sqlError: Duplicate key name 'PCS_STATS_IDX' (state=42000,code=1061)org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!*** schemaTool failed ***
处理方法处理$HIVE_HOME/bin/hive
 #sparkAssemblyPath=`ls ${SPARK_HOME}/lib/spark-assembly-*.jar` sparkAssemblyPath=`ls ${SPARK_HOME}/jars/*.jar`
 
一、sqoop导入到hive权限问题，有谁能解决
 不是权限问题。需要为sqoop配置你使用的hive。
1、找到/sqoop-1.4.4/conf下的sqoop-env-template.sh 文件，将这个文件重命名为sqoop-env.sh ；
2、编辑sqoop-env.sh 文件，将你的hive的安装目录配上就OK。
 export HIVE_HOME=/cloud/apache-hive-1.2.1-bin 
二、ubuntu 下载hive时显示error is not recoverable:exiting now?
 很明显，你的./下并没有这个压缩包，你确定是下载到家目录下面？找找压缩包的具体目录，在那个目录下打开终端执行这个命令，或者用绝对路径。 
三、hadoop伪分布式安装部署HIVE怎么做
 添加下面三行：
export JAVA_HOME=/home/software/jdk1.8.0_151
export HADOOP_HOME=/home/software/hadoop-2.6.5export HIVE_HOME=/home/software/apache-hive-1.2.2-bin