EMR-flinksql运行失败问题

芒果1年前技术文章969

运行flinksql
sql-client.sh
报错：
[root@emr1 bin]# ./sql-client.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/dtstack/Flink/flink_pkg/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/dtstack/Tez/tez_pkg/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
10:55:25.408 [main] ERROR org.apache.flink.table.factories.FactoryUtil - Could not load service provider for factories.
java.util.ServiceConfigurationError: org.apache.flink.table.factories.Factory: Provider org.apache.flink.table.formats.raw.RawFormatFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[?:1.8.0_144]
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) [flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) [flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.createTableEnvironment(ExecutionContext.java:114) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.<init>(ExecutionContext.java:66) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:246) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Sets
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    ... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Sets
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_144]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_144]
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    ... 14 more
10:55:25.420 [main] ERROR org.apache.flink.table.client.SqlClient - SQL Client must stop. Unexpected exception. This is a bug. Please consider filing an issue.
org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:165) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.createTableEnvironment(ExecutionContext.java:114) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.<init>(ExecutionContext.java:66) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:246) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
Caused by: org.apache.flink.table.api.TableException: Could not load service provider for factories.
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:627) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Caused by: java.util.ServiceConfigurationError: org.apache.flink.table.factories.Factory: Provider org.apache.flink.table.formats.raw.RawFormatFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[?:1.8.0_144]
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Sets
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Sets
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_144]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_144]
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Exception in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected exception. This is a bug. Please consider filing an issue.
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:201)
    at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
Caused by: org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:165)
    at org.apache.flink.table.client.gateway.context.ExecutionContext.createTableEnvironment(ExecutionContext.java:114)
    at org.apache.flink.table.client.gateway.context.ExecutionContext.<init>(ExecutionContext.java:66)
    at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:246)
    at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87)
    at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87)
    at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88)
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187)
    ... 1 more
Caused by: org.apache.flink.table.api.TableException: Could not load service provider for factories.
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:627)
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378)
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156)
    ... 8 more
Caused by: java.util.ServiceConfigurationError: org.apache.flink.table.factories.Factory: Provider org.apache.flink.table.formats.raw.RawFormatFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623)
    ... 10 more
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Sets
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
    ... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Sets
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 21 more
检查发现有个lib包的版本后缀存在问题

F382BD31-0482-46D8-9ED0-25D7BD554D39.png
删除后重新运行解决
6EDA7728-7489-442D-8149-A1692489AD10.png
放入kafka的时候需要驱动 不然会报错
094EFAC2-A0D7-44C8-8FC0-55CBF68D4C45.png
maven地址
https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-kafka_2.12
当前环境的flink是1.14.5的所以我们选择
https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-kafka_2.12/1.14.5
放到flink/lib下
2D0D1835-5F71-4BC0-92C7-0C34CDE82874.png


标签: 大数据运维运维
返回列表

上一篇：单节点Kafka部署并开启Kerberos配置

下一篇：ORC、Parquet 等列式存储的优点

相关文章

HDP部署Tez UI

HDP部署Tez UI

首先部署tomcat在官网下载apache-tomcat-9.0.22.tar.gz[root@hdp02 opt]# wget https://archive.apache.org/dist/tom...

Hive删除外部表

Hive删除外部表

删除外部表操作例：1.首先我们创建一个外部表create EXTERNAL table if not exists mgtest(id int, name string)row format deli...

Hive架构图及Hive SQL的执行流程

Hive架构图及Hive SQL的执行流程

1、Hive产生背景MapReduce编程的不便性HDFS上的文件缺少Schema(表名，名称，ID等，为数据库对象的集合)2、Hive是什么Hive的使用场景是什么？基于Hadoop做一些数据清洗啊...

HDP-Yarn开启CPU调度和隔离

HDP-Yarn开启CPU调度和隔离

进入到ambari主界面 点击yarn 点击config CPU Scheduling and Isolation 设置为enable修改高级配置点击ADVANCED搜索需要修改的配yarn.node...

单节点Kafka部署并开启Kerberos配置

安装Kerberosserver节点安装kerberos相关软件yum install -y krb5-server krb5-workstation krb5-libs修改配置文件krb5.conf...

大数据高可用系列--kudu高可用应急方案

大数据高可用系列--kudu高可用应急方案

1 设置机架感知1.1 前置说明    1.9版本后的kudu已经支持机架感知（cdh6之后的版本中的kudu已支持），由于kudu的每个Tablet一般是三副...

发表评论    

◎欢迎参与讨论，请在这里发表您的看法、交流您的观点。

©Copyrights 2016-2022 YUNCHE 浙ICP备2021017017号