trino容器对接alluxio（四）
前提：本文是基于已经部署了trino容器和 集群中部署了alluxio组件的基础上，进行的trino与alluxio的组件对接，alluxio已经开启了高可用模式。本文的冒烟测试是在trino对接ldap后并且ranger已经对接了metastore权限后进行的。配置增加是在基于trino容器对接hive后，创建的hive_conf的configmap的基础上增加的。
前提文章见：helm安装部署trino对接hive（一）、trino容器对接ldap（二）、ranger对接metastore（三）
安装部署
1、增加alluxio配置
在容器hive_conf本地路径下，修改core-site.xml和hdfs-site.xml配置文件，在core-site.xml和hdfs-site.xml中增加alluxio相关配置信息。
core-site.xml:
<property> <name>alluxio.master.rpc.addresses</name> <value>hadoop001:19998,hadoop002:19998,hadoop003:19998</value> </property>
hdfs-site.xml:
<property> <name>alluxio.master.nameservices</name> <value>my-alluxio-cluster</value> </property> <property> <!-- 配置alluxio 的名称，多个用逗号分割 --> <name>alluxio.master.nameservices.my-alluxio-cluster</name> <value>master1,master2,master3</value> </property> <property> <name>alluxio.master.rpc.address.my-alluxio-cluster.master1</name> <value>hadoop001:19998</value> </property> <property> <name>alluxio.master.rpc.address.my-alluxio-cluster.master2</name> <value>hadoop002:19998</value> </property> <property> <name>alluxio.master.rpc.address.my-alluxio-cluster.master3</name> <value>hadoop003:19998</value> </property>
2、重新创建hive_conf 的configmap
kubectl -n trino-test delete cm hive-conf kubectl -n trino-test create cm hive-conf --from-file=hive_conf/
3、重启trino容器
helm -n trino-test uninstall trino helm install trino /root/jiuyue/helm/trino/ -n trino-test
冒烟测试
/root/jiuyue/image/trino/trino-cli-416-executable.jar --server https://172.16.121.114:31443 --keystore-path /root/jiuyue/helm/conf/trino/ldap_conf/trino.jks --keystore-password admin@123 --catalog hive --user test --password --debug #输入test用户密码 Password: trino> show schemas; Schema -------------------- default hudi_hive information_schema spark_hudi spark_k8s_hudi test test2 test3 (8 rows) trino> trino> use test; USE trino:test> show tables; Table ---------------------- alluxio_user2 alluxio_user3 alluxio_user4 alluxio_user5 alluxio_user6 alluxio_user7 alluxiotest fare_adjustment hive_student hive_student1 hudi_table merge_source student test test_flink_alluxio test_flink_hdfs test_flink_hudi test_hive_alluxio test_hive_hdfs test_spark_alluxio test_spark_alluxio33 test_spark_hdfs test_spark_hudi testcc u_user u_user2 trino:test> select * from alluxio_user5 limit 10; userid | age | gender | occupation | zipcode --------+------+--------+------------+--------- 109 | 8888 | F | zhangsan | 49087 9527 | 200 | F | ikun | 95033 110 | 8888 | F | 李四 | 49087 9528 | 200 | M | ikun | 95033 1 | 24 | M | technician | 85711 2 | 53 | F | other | 94043 3 | 23 | M | writer | 32067 4 | 24 | M | technician | 43537 5 | 33 | F | other | 15213 6 | 42 | M | executive | 98101 (10 rows) trino:test> select count(*) from alluxio_user5; _col0 ------- 948 trino:test> INSERT INTO alluxio_user5 values (111, 8888, 'F','李四','490879'); INSERT: 1 row Query 20231124_051849_00026_yvgj6, FINISHED, 3 nodes https://172.16.121.114:31443/ui/query.html?20231124_051849_00026_yvgj6 Splits: 50 total, 50 done (100.00%) CPU Time: 0.1s total, 0 rows/s, 0B/s, 19% active Per Node: 0.0 parallelism, 0 rows/s, 0B/s Parallelism: 0.1 Peak Memory: 3.7KB 1.85 [0 rows, 0B] [0 rows/s, 0B/s] trino:test> select * from alluxio_user5 where userid = 111; userid | age | gender | occupation | zipcode --------+------+--------+------------+--------- 111 | 8888 | F | 李四 | 490879 111 | 57 | M | engineer | 90630 (2 rows) trino:test> select count(*) from alluxio_user5; _col0 ------- 949
在测试trino创建alluxio表的时候，可以创建外部表，但是由于trino本身不支持插入外部表，所以无法向alluxio表中插入数据。并且建表的时候支持external_location关键字，不支持location。
