主页 / computer / 问题

问题[hadoop](computer)

Martin Hope
Smith Cruise
Asked: 2019-10-16 00:12:13 +0800 CST
  • 9
  • 蜂巢 3.1.2
  • Hadoop 3.2.1

当我在命令行中运行 hive 时，它​​会告诉我以下错误消息：

which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
        at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)
        at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554)
        at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448)
        at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5141)
        at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5099)
        at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:97)
        at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:81)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:699)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

我应该怎么办？

hadoop
Martin Hope
Geerten Verweij
Asked: 2019-04-24 05:35:39 +0800 CST
  • 5

我正在尝试在 Azure HDinsight 集群上运行 Halyard。

升降索：https ://merck.github.io/Halyard/

Azure 版本的 HBase：https ://docs.microsoft.com/en-us/azure/hdinsight/hbase/apache-hbase-overview

以下步骤成功：

  1. SSH 到主 HBase 节点：-ssh.azurehdinsight.net，端口为 22
  2. 使用 list commond 运行 HBase shell。
  3. 下载 Halyard sdk
  4. 运行 ./console 并创建测试存储库

我在下一步尝试打开该存储库以进行一些测试查询时失败了。出现以下错误：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/client/TableDescriptorBuilder
        at com.msd.gin.halyard.common.HalyardTableUtils.getTable(HalyardTableUtils.java:162)
        at com.msd.gin.halyard.common.HalyardTableUtils.getTable(HalyardTableUtils.java:146)
        at com.msd.gin.halyard.sail.HBaseSail.initialize(HBaseSail.java:201)
        at com.msd.gin.halyard.sail.HBaseSail.initialize(HBaseSail.java:194)
        at org.eclipse.rdf4j.repository.sail.SailRepository.initializeInternal(SailRepository.java:151)
        at org.eclipse.rdf4j.repository.base.AbstractRepository.initialize(AbstractRepository.java:34)
        at org.eclipse.rdf4j.repository.manager.LocalRepositoryManager.createRepository(LocalRepositoryManager.java:273)
        at org.eclipse.rdf4j.repository.manager.RepositoryManager.getRepository(RepositoryManager.java:424)
        at org.eclipse.rdf4j.console.command.Open.openRepository(Open.java:80)
        at org.eclipse.rdf4j.console.command.Open.execute(Open.java:65)
        at org.eclipse.rdf4j.console.Console.executeCommand(Console.java:376)
        at org.eclipse.rdf4j.console.Console.start(Console.java:337)
        at org.eclipse.rdf4j.console.Console.main(Console.java:239)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.client.TableDescriptorBuilder
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 13 more

我不知道如何解决这个问题。kinit 步骤可能是必不可少的吗？这个 Azure-ready-made-made-HBase 不提供对 HBase 和 Hadoop java 类的访问吗？

欢迎任何帮助，包括参考从哪里开始了解我的预期堆栈中的元素，因为我没有经验。

azure hadoop