EMR-flinksql运行失败问题

芒果7个月前技术文章366

运行flinksql
sql-client.sh
报错:
[root@emr1 bin]# ./sql-client.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/dtstack/Flink/flink_pkg/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/dtstack/Tez/tez_pkg/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
10:55:25.408 [main] ERROR org.apache.flink.table.factories.FactoryUtil - Could not load service provider for factories.
java.util.ServiceConfigurationError: org.apache.flink.table.factories.Factory: Provider org.apache.flink.table.formats.raw.RawFormatFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[?:1.8.0_144]
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) [flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) [flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.createTableEnvironment(ExecutionContext.java:114) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.<init>(ExecutionContext.java:66) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:246) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Sets
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    ... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Sets
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_144]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_144]
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    ... 14 more
10:55:25.420 [main] ERROR org.apache.flink.table.client.SqlClient - SQL Client must stop. Unexpected exception. This is a bug. Please consider filing an issue.
org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:165) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.createTableEnvironment(ExecutionContext.java:114) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.<init>(ExecutionContext.java:66) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:246) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161) [flink-sql-client_2.11-1.14.5.jar:1.14.5]
Caused by: org.apache.flink.table.api.TableException: Could not load service provider for factories.
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:627) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Caused by: java.util.ServiceConfigurationError: org.apache.flink.table.factories.Factory: Provider org.apache.flink.table.formats.raw.RawFormatFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[?:1.8.0_144]
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Sets
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Sets
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_144]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_144]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_144]
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144) ~[flink-table-blink_2.11-1.12.5.jar:1.12.5]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_144]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_144]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_144]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_144]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_144]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_144]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_144]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_144]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378) ~[flink-table_2.11-1.14.5.jar:1.14.5]
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156) ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
    ... 8 more
Exception in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected exception. This is a bug. Please consider filing an issue.
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:201)
    at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
Caused by: org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:165)
    at org.apache.flink.table.client.gateway.context.ExecutionContext.createTableEnvironment(ExecutionContext.java:114)
    at org.apache.flink.table.client.gateway.context.ExecutionContext.<init>(ExecutionContext.java:66)
    at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:246)
    at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:87)
    at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:87)
    at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88)
    at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187)
    ... 1 more
Caused by: org.apache.flink.table.api.TableException: Could not load service provider for factories.
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:627)
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:378)
    at org.apache.flink.table.client.gateway.context.ExecutionContext.lookupExecutor(ExecutionContext.java:156)
    ... 8 more
Caused by: java.util.ServiceConfigurationError: org.apache.flink.table.factories.Factory: Provider org.apache.flink.table.formats.raw.RawFormatFactory could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:623)
    ... 10 more
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/collect/Sets
    at org.apache.flink.table.formats.raw.RawFormatFactory.<clinit>(RawFormatFactory.java:144)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
    ... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.collect.Sets
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 21 more
检查发现有个lib包的版本后缀存在问题

F382BD31-0482-46D8-9ED0-25D7BD554D39.png
删除后重新运行解决
6EDA7728-7489-442D-8149-A1692489AD10.png
放入kafka的时候需要驱动 不然会报错
094EFAC2-A0D7-44C8-8FC0-55CBF68D4C45.png
maven地址
https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-kafka_2.12
当前环境的flink是1.14.5的所以我们选择
https://mvnrepository.com/artifact/org.apache.flink/flink-sql-connector-kafka_2.12/1.14.5
放到flink/lib下
2D0D1835-5F71-4BC0-92C7-0C34CDE82874.png


相关文章

磁盘分盘脚本分享

磁盘分区脚本名称:mg_fdisk.sh#!/bin/bashif [ "$#" -ne 1 ]; then  echo "请传入磁盘参数"  exit 1fidisk=$1# 检查磁盘是否存在if...

hive创建hbase映射表

hive创建hbase映射表

hbase创建表,导入数据/opt/app/hbase-2.1.0/bin/hbase shell查看已有表,创建新表,查看表结构listcreate 'student', 'info', 'scor...

chengying-6.0登入接口逆向

chengying-6.0登入接口逆向

版本更新首先是登入的加密url:http://172.16.121.70/login参数1. username:admin@dtstack.com2. password:614bb9438210c69...

寻找CPU使用率高进程方法

寻找CPU使用率高进程方法

背景节点报CPU使用率高,需要定位是什么进程占用CPU使用率高。CPU使用率持续较高在对应节点使用 “top”命令,然后键盘输入“P”,即按照CPU使用率排序进程。执行ps -ef | grep &l...

EMR-java配置国密SM4加密

EMR-java配置国密SM4加密

首先找到bcprov-jdk15on-1.56.jar这个包<dependency>    <groupId>org.bouncycastle</groupId> ...

IDC:疫情下,第三方云管理服务市场逆势增长!

IDC:疫情下,第三方云管理服务市场逆势增长!

IDC预测,中国第三方云管理服务在2019年到2023年间将保持54.7%的增长率,2023年市场规模预计达到32.1亿美元。2019年第三方云管理服务市场呈现出如下特点:云管理服务成为众多服务商的战...

发表评论    

◎欢迎参与讨论,请在这里发表您的看法、交流您的观点。