ranger对接metastore
前提:本文前提是基于集群中已经安装部署了ranger组件、hive组件的情况下,增加ranger metastore插件的对接。
安装部署
1、ranger metastore插件编译
插件下载
https://git.lrting.top/xiaozhch5/ranger-metastore-plugin.git
下载后进行安装部署
tar -zxvf ranger-metastore-plugin-master.tar.gz
修改pom文件为指定版本,本文部署的版本是2.4
agents-audit/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0-SNAPSHOT</version> <version>2.4.0</version> <relativePath>..</relativePath> </parent> <dependencies> agents-common/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0-SNAPSHOT</version> <version>2.4.0</version> <relativePath>..</relativePath> </parent> <dependencies> agents-cred/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0</version> <relativePath>..</relativePath> </parent> agents-installer/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0-SNAPSHOT</version> <version>2.4.0</version> <relativePath>..</relativePath> </parent> <dependencies> credentialbuilder/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0</version> <relativePath>..</relativePath> </parent> metastore-agent/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0</version> <relativePath>..</relativePath> </parent> ranger-plugin-classloader/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0</version> </parent> ranger-util/pom.xml <parent> <groupId>org.apache.ranger</groupId> <artifactId>ranger-metastore-plugin</artifactId> <version>2.4.0</version> </parent> pom.xml <modelVersion>4.0.0</modelVersion> <artifactId>ranger-metastore-plugin</artifactId> <groupId>org.apache.ranger</groupId> <version>2.4.0</version> <name>Hive Metastore Security Plugin</name> <description>Hive Metastore Security Plugins</description>
进行编译
mvn clean compile package install assembly:assembly -DskipTests
2、插件包解压安装
#进行解压 tar -zxvf ranger-metastore-plugin-2.4.0-metastore-plugin.tar.gz -C ranger-metastore-plugin
3、修改配置进行部署
vim install.properties #进行配置 POLICY_MGR_URL=http://172.16.121.0:6080 REPOSITORY_NAME=metastore COMPONENT_INSTALL_DIR_NAME=/opt/hive #audit相关配置 XAAUDIT.SOLR.ENABLE=true XAAUDIT.SOLR.URL=http://172.16.121.0:8983/solr/ranger_audits XAAUDIT.SOLR.USER=NONE XAAUDIT.SOLR.PASSWORD=NONE XAAUDIT.SOLR.ZOOKEEPER=hadoop001:2181,hadoop002:2181,hadoop003:2181/ranger_audits XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hive/audit/solr/spool #Solr Audit Provder XAAUDIT.SOLR.IS_ENABLED=true XAAUDIT.SOLR.MAX_QUEUE_SIZE=1 XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000 XAAUDIT.SOLR.SOLR_URL=http://172.16.121.0:8983/solr/ranger_audits
./enable-metastore-plugin.sh
4、ranger页面设置metastore权限
测试链接连通性
冒烟测试
以test用户为例,目前test用户对hive的test库、hudi_hive库中的表有所有权限,以test测试hudi_hive库。
/root/jiuyue/image/trino/trino-cli-416-executable.jar --server https://172.16.121.114:31443 --keystore-path /root/jiuyue/helm/conf/trino/ldap_conf/trino.jks --keystore-password admin@123 --catalog hudi --user test --password --debug trino> use hudi_hive; USE trino:hudi_hive> show tables; Table ------------------ fare_adjustment host_hudi hudi_table hudi_table2 hudi_table_keyed student tbl_customer (7 rows) Query 20231124_072453_00016_mizze, FINISHED, 3 nodes https://172.16.121.114:31443/ui/query.html?20231124_072453_00016_mizze Splits: 36 total, 36 done (100.00%) CPU Time: 0.0s total, 304 rows/s, 9.04KB/s, 35% active Per Node: 0.0 parallelism, 9 rows/s, 276B/s Parallelism: 0.1 Peak Memory: 2.35KB 0.26 [7 rows, 213B] [27 rows/s, 829B/s] trino:hudi_hive> select * from hudi_table2 limit 1; _hoodie_commit_time | _hoodie_commit_seqno | _hoodie_record_key | _hoodie_partition_path | _hoodie_file_> ---------------------+-----------------------+--------------------------------------+------------------------+-----------------------------------------> 20231123161346626 | 20231123161346626_2_0 | 3eeb61f7-c2b0-4636-99bd-5d7a5a1d2c04 | city=chennai | b0b19f4e-7530-4df6-ab29-638afcea17ef-0_2> (1 row)
能正常查询 。
将test用户权限进行调整。
将库的权限由hudi_hive库改为test库
看权限已经更新,重新登录查询,无法进行查看,没有权限
/root/jiuyue/image/trino/trino-cli-416-executable.jar --server https://172.16.121.114:31443 --keystore-path /root/jiuyue/helm/conf/trino/ldap_conf/trino.jks --keystore-password admin@123 --catalog hudi --user test --password --debug