ranger对接metastore

九月11个月前技术文章575

前提:本文前提是基于集群中已经安装部署了ranger组件、hive组件的情况下,增加ranger metastore插件的对接。

安装部署

1、ranger metastore插件编译

插件下载

https://git.lrting.top/xiaozhch5/ranger-metastore-plugin.git

下载后进行安装部署

tar -zxvf ranger-metastore-plugin-master.tar.gz

修改pom文件为指定版本,本文部署的版本是2.4

agents-audit/pom.xml
<parent>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-metastore-plugin</artifactId>
<version>2.4.0-SNAPSHOT</version>
<version>2.4.0</version>
<relativePath>..</relativePath>
</parent>
<dependencies>
 
agents-common/pom.xml
<parent>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-metastore-plugin</artifactId>
<version>2.4.0-SNAPSHOT</version>
<version>2.4.0</version>
<relativePath>..</relativePath>
</parent>
<dependencies>

agents-cred/pom.xml
    <parent>
        <groupId>org.apache.ranger</groupId>
        <artifactId>ranger-metastore-plugin</artifactId>
        <version>2.4.0</version>
        <relativePath>..</relativePath>
    </parent>

agents-installer/pom.xml
<parent>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-metastore-plugin</artifactId>
<version>2.4.0-SNAPSHOT</version>
<version>2.4.0</version>
<relativePath>..</relativePath>
</parent>
<dependencies>

credentialbuilder/pom.xml
    <parent>
        <groupId>org.apache.ranger</groupId>
        <artifactId>ranger-metastore-plugin</artifactId>
        <version>2.4.0</version>
        <relativePath>..</relativePath>
    </parent>

metastore-agent/pom.xml
   <parent>
        <groupId>org.apache.ranger</groupId>
        <artifactId>ranger-metastore-plugin</artifactId>
        <version>2.4.0</version>
        <relativePath>..</relativePath>
    </parent>

ranger-plugin-classloader/pom.xml
  <parent>
        <groupId>org.apache.ranger</groupId>
        <artifactId>ranger-metastore-plugin</artifactId>
        <version>2.4.0</version>
    </parent>

ranger-util/pom.xml
    <parent>
        <groupId>org.apache.ranger</groupId>
        <artifactId>ranger-metastore-plugin</artifactId>
        <version>2.4.0</version>
    </parent>

pom.xml
    <modelVersion>4.0.0</modelVersion>
    <artifactId>ranger-metastore-plugin</artifactId>
    <groupId>org.apache.ranger</groupId>
    <version>2.4.0</version>
    <name>Hive Metastore Security Plugin</name>
    <description>Hive Metastore Security Plugins</description>

进行编译

mvn clean compile package install assembly:assembly -DskipTests

2、插件包解压安装

#进行解压
tar -zxvf ranger-metastore-plugin-2.4.0-metastore-plugin.tar.gz -C ranger-metastore-plugin

1.png

3、修改配置进行部署

vim install.properties

#进行配置
POLICY_MGR_URL=http://172.16.121.0:6080
REPOSITORY_NAME=metastore
COMPONENT_INSTALL_DIR_NAME=/opt/hive

#audit相关配置
XAAUDIT.SOLR.ENABLE=true             
XAAUDIT.SOLR.URL=http://172.16.121.0:8983/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=hadoop001:2181,hadoop002:2181,hadoop003:2181/ranger_audits
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hive/audit/solr/spool

#Solr Audit Provder
XAAUDIT.SOLR.IS_ENABLED=true
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://172.16.121.0:8983/solr/ranger_audits
./enable-metastore-plugin.sh

4、ranger页面设置metastore权限




测试链接连通性

冒烟测试

以test用户为例,目前test用户对hive的test库、hudi_hive库中的表有所有权限,以test测试hudi_hive库。

 /root/jiuyue/image/trino/trino-cli-416-executable.jar --server https://172.16.121.114:31443 --keystore-path /root/jiuyue/helm/conf/trino/ldap_conf/trino.jks --keystore-password admin@123  --catalog hudi --user test --password --debug
trino> use hudi_hive;
USE
trino:hudi_hive> show tables;
      Table       
------------------
 fare_adjustment  
 host_hudi        
 hudi_table       
 hudi_table2      
 hudi_table_keyed 
 student          
 tbl_customer     
(7 rows)

Query 20231124_072453_00016_mizze, FINISHED, 3 nodes
https://172.16.121.114:31443/ui/query.html?20231124_072453_00016_mizze
Splits: 36 total, 36 done (100.00%)
CPU Time: 0.0s total,   304 rows/s, 9.04KB/s, 35% active
Per Node: 0.0 parallelism,     9 rows/s,   276B/s
Parallelism: 0.1
Peak Memory: 2.35KB
0.26 [7 rows, 213B] [27 rows/s, 829B/s]

trino:hudi_hive> select * from hudi_table2 limit 1;
 _hoodie_commit_time | _hoodie_commit_seqno  |          _hoodie_record_key          | _hoodie_partition_path |                            _hoodie_file_>
---------------------+-----------------------+--------------------------------------+------------------------+----------------------------------------->
 20231123161346626   | 20231123161346626_2_0 | 3eeb61f7-c2b0-4636-99bd-5d7a5a1d2c04 | city=chennai           | b0b19f4e-7530-4df6-ab29-638afcea17ef-0_2>
(1 row)

6.png

能正常查询 。

将test用户权限进行调整。

将库的权限由hudi_hive库改为test库

看权限已经更新,重新登录查询,无法进行查看,没有权限

/root/jiuyue/image/trino/trino-cli-416-executable.jar --server https://172.16.121.114:31443 --keystore-path /root/jiuyue/helm/conf/trino/ldap_conf/trino.jks --keystore-password admin@123  --catalog hudi --user test --password --debug

9.png



相关文章

helm安装部署trino对接hive(一)

helm安装部署trino对接hive(一)

前提:本文前提是基于hive组件已经提前安装的情况下,安装部署好trino容器之后进行对hive组件的对接。helm trino地址:https://artifacthub.io/packages/h...

trino容器对接ldap(二)

trino容器对接ldap(二)

前提:本文前提是在trino容器已经对接上hive组件,并且ldap已经部署完成的基础上进行的对接。前提文章见:helm安装部署trino对接hive(一)安装部署1、设置证书因为不确定是在哪台机器中...

chengying-4.0登入接口逆向

chengying-4.0登入接口逆向

首先是登入的加密url:http://xxxxx/api/v2/user/login参数1. username:admin@dtstack.com2. password:614bb9438210c69...

Keepalived安装部署

Keepalived安装部署

安装keepalivedyum 安装keepalivedyum install keepalivedkeepalived.conf配置文件进行调整,其中state设置主备状态,interface设置对...

Mac安装Hadoop文档-保姆级操作(二)

Mac安装Hadoop文档-保姆级操作(二)

配置hadoop进入hadoop的目录:cd /opt/homebrew/Cellar/hadoop/3.3.6/libexec/etc/hadoop修改core-site.xml<config...

dolphinscheduler单机部署

dolphinscheduler单机部署

官网链接:https://dolphinscheduler.apache.org本次测试版本为:https://dolphinscheduler.apache.org/zh-cn/download/3...

发表评论    

◎欢迎参与讨论,请在这里发表您的看法、交流您的观点。