主页 / ubuntu / 问题 / 1073035
Accepted
SriniShine
Asked: 2018-09-08 02:04:44 +0800 CST

失败：HiveException java.lang.RuntimeException：无法实例化 org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

  • 772

我正在尝试使用 Hadoop 3.0 运行 Hive 3.1。以下是我的系统配置：

Ubuntu 18.04.1 LTS
Hadoop  version 3.0.3
Hive 3.1.0
Derby 10.14.2

当我执行显示表时；查询我收到以下错误。

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

以下是配置单元日志文件中的详细错误。

2018-09-05T11:38:25,952  INFO [main] conf.HiveConf: Found configuration file file:/usr/local/apache-hive-3.1.0-bin/conf/hive-site.xml
2018-09-05T11:38:30,549  INFO [main] SessionState: Hive Session ID = 826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:35,948  INFO [main] SessionState:
Logging initialized using configuration in jar:file:/usr/local/apache-hive-3.1.0-bin/lib/hive-common-3.1.0.jar!/hive-log4j2.properties Asy$
2018-09-05T11:38:47,015  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop
2018-09-05T11:38:47,069  INFO [main] session.SessionState: Created local directory: /tmp/mydir
2018-09-05T11:38:47,096  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:47,104  INFO [main] session.SessionState: Created local directory: /tmp/mydir/826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:47,122  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/826ec55c-7fca-4fff-baa5-b5a010e5af89/_$
2018-09-05T11:38:47,125  INFO [main] conf.HiveConf: Using the default value passed in for log id: 826ec55c-7fca-4fff-baa5-b5a010e5af89
2018-09-05T11:38:47,126  INFO [main] session.SessionState: Updating thread name to 826ec55c-7fca-4fff-baa5-b5a010e5af89 main
2018-09-05T11:38:50,476  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.HiveMetaStore: 0: Opening raw store with implementatio$
2018-09-05T11:38:50,695  WARN [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set$
2018-09-05T11:38:50,714  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.ObjectStore: ObjectStore, initialize called
2018-09-05T11:38:50,717  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file file:/usr/local/apa$
2018-09-05T11:38:50,719  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Unable to find config file hivemetastore-sit$
2018-09-05T11:38:50,720  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file null
2018-09-05T11:38:50,722  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Unable to find config file metastore-site.xml
2018-09-05T11:38:50,722  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file null

蜂巢站点.xml

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:;databaseName=metastore_db;create=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>

.profile 中的环境变量（我正在尝试配置由其他人完成的安装，因此即使手动运行 Hadoop，环境变量也设置在 .profile 而不是 .bashrc 中）

#HIVE
export HIVE_HOME=/usr/local/apache-hive-3.1.0-bin
export HIVE_CONF_DIR=/usr/local/apache-hive-3.1.0-bin/conf
export PATH=$HIVE_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:/usr/local/hadoop/lib/*:.
export CLASSPATH=$CLASSPATH:/usr/local/apache-hive-3.1.0-bin/lib/*:.

#DERBY
DERBY_HOME=/usr/local/db-derby-10.14.2.0-bin
export PATH=$PATH:$DERBY_HOME/bin
export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytool$

错误消息表明我没有遇到配置单元配置的 metastore-site.xml 文件。

18.04 hadoop hive

1 个回答

  1. Best Answer

    我认为 Hive 期望在当前工作目录中找到 Derby 数据库。Hive 将创建数据库，因为create=true在javax.jdo.option.ConnectionURL property，但不会初始化数据库中的模式。

    我的配置：

    • Ubuntu 18.04.1 LTS
    • Hadoop 3.1.1
    • 蜂巢 3.1.0

    • 来自 Ubuntu 存储库的 Apache Derby（当前版本：10.14.1.0-1ubuntu1）

      sudo apt install derby-tools libderby-java libderbyclient-java

    根据Hive 文档，我们需要明确运行 ShemaTool。

    我决定将我的 Derby 数据库保存在 HIVE_HOME（在我的例子中是 /opt/hive-3.1.0-bin）目录中。您可以在hive-site.xml中设置 Derby 数据库位置：

    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:/opt/hive-3.1.0-bin/metastore_db;databaseName=metastore_db;create=true</value>
</property>

    确保还没有 Derby 数据库：

    $ ls $HIVE_HOME/meta*
ls: cannot access '/opt/hive-3.1.0-bin/meta*': No such file or directory

    启动蜂巢：

    $ pwd
/home/hadoop
$ hive
hive> show tables;
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
hive> exit;

    现在我们得到了 Derby 数据库，但它不起作用：

    $ ls -d $HIVE_HOME/meta*
/opt/hive-3.1.0-bin/metastore_db

    所以，我们需要创建元数据模式：

    $ rm -rf $HIVE_HOME/metastore_db
$ cd $HIVE_HOME
$ schematool -initSchema -dbType derby
...
Initialization script completed
schemaTool completed

    现在 Hive 正在按预期工作：

    $ cd
$ hive
hive> show tables;
OK
Time taken: 0.803 seconds
    • 1

相关问题