Details
Description
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
` partition ` VARCHAR (20)
)
PARTITIONED BY (`partition`)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
Attachments
Issue Links
Activity
hudi-bot commented on pull request #3611:
URL: https://github.com/apache/hudi/pull/3611#issuecomment-914085520
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- 178767948e906f673d6d4a357c65c11bc574f619 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3611:
URL: https://github.com/apache/hudi/pull/3611#issuecomment-914085520
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- 178767948e906f673d6d4a357c65c11bc574f619 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2063)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3613:
URL: https://github.com/apache/hudi/pull/3613
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
partition VARCHAR (20)
)
PARTITIONED BY (partition)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3611:
URL: https://github.com/apache/hudi/pull/3611
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3613:
URL: https://github.com/apache/hudi/pull/3613
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu commented on pull request #3613:
URL: https://github.com/apache/hudi/pull/3613#issuecomment-914111512
111
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot commented on pull request #3613:
URL: https://github.com/apache/hudi/pull/3613#issuecomment-914114267
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- b7972ad260742b097ffaf740e8bd258bb49d4b23 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3613:
URL: https://github.com/apache/hudi/pull/3613#issuecomment-914114267
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- b7972ad260742b097ffaf740e8bd258bb49d4b23 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2064)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu removed a comment on pull request #3613:
URL: https://github.com/apache/hudi/pull/3613#issuecomment-914111512
111
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3613:
URL: https://github.com/apache/hudi/pull/3613
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu commented on pull request #3613:
URL: https://github.com/apache/hudi/pull/3613#issuecomment-914128031
1111
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu removed a comment on pull request #3613:
URL: https://github.com/apache/hudi/pull/3613#issuecomment-914128031
1111
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3613:
URL: https://github.com/apache/hudi/pull/3613
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
partition VARCHAR (20)
)
PARTITIONED BY (partition)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3613:
URL: https://github.com/apache/hudi/pull/3613
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3615:
URL: https://github.com/apache/hudi/pull/3615
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
partition VARCHAR (20)
)
PARTITIONED BY (partition)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot commented on pull request #3615:
URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- d8ccec907403ee123b5a66a43ea99677f46be5a4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3615:
URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- d8ccec907403ee123b5a66a43ea99677f46be5a4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3615:
URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- d8ccec907403ee123b5a66a43ea99677f46be5a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3615:
URL: https://github.com/apache/hudi/pull/3615
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3619:
URL: https://github.com/apache/hudi/pull/3619
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
partition VARCHAR (20)
)
PARTITIONED BY (partition)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot commented on pull request #3619:
URL: https://github.com/apache/hudi/pull/3619#issuecomment-914439925
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- 8a640c1fd1342fa75a8089efa8ba549e1b94f5ba UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3619:
URL: https://github.com/apache/hudi/pull/3619#issuecomment-914439925
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- 8a640c1fd1342fa75a8089efa8ba549e1b94f5ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2076)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3619:
URL: https://github.com/apache/hudi/pull/3619
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3621:
URL: https://github.com/apache/hudi/pull/3621
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
partition VARCHAR (20)
)
PARTITIONED BY (partition)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot commented on pull request #3621:
URL: https://github.com/apache/hudi/pull/3621#issuecomment-914737057
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- 824df17f32aaff74bfb8d5d4a1001b7b320c4915 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3621:
URL: https://github.com/apache/hudi/pull/3621#issuecomment-914737057
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- 824df17f32aaff74bfb8d5d4a1001b7b320c4915 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2090)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu closed pull request #3621:
URL: https://github.com/apache/hudi/pull/3621
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3622:
URL: https://github.com/apache/hudi/pull/3622
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
partition VARCHAR (20)
)
PARTITIONED BY (partition)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot commented on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- f2eaffe6a6bdd3270319ceac5541efda4e42b302 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
]
}-->
-
- CI report:
- f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
danny0405 commented on a change in pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#discussion_r703976033
##########
File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java
##########
@@ -369,6 +374,8 @@
conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions);
conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix);
conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp);
+ conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos);
+ conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal);
return conf;
Review comment:
Should decide if `config.kerberosPrincipal` is null first.
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -191,6 +191,16 @@ object DataSourceWriteOptions {
.withAlternatives("hoodie.datasource.write.storage.type")
.withDocumentation("The table type for the underlying data, for this write. This can’t change between writes.")
+ val HIVE_SYNC_USE_KERBEROS: ConfigProperty[Boolean] = ConfigProperty
+ .key("hoodie.datasource.hive_sync.use_kerberos")
+ .defaultValue(false)
+ .withDocumentation("Whether to use Kerberos authentication.")
+
+ val HIVE_SYNC_KERBEROS_PRINCIPAL: ConfigProperty[String] = ConfigProperty
+ .key("hive_sync.kerberos_principal")
+ .noDefaultValue()
Review comment:
hoodie.datasource.hive_sync.kerberos_principal ? And move this option together with the other hive sync options.
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
,
{ "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" } ]
}-->
-
- CI report:
- f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091)
- aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu commented on a change in pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#discussion_r704001915
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -191,6 +191,16 @@ object DataSourceWriteOptions {
.withAlternatives("hoodie.datasource.write.storage.type")
.withDocumentation("The table type for the underlying data, for this write. This can’t change between writes.")
+ val HIVE_SYNC_USE_KERBEROS: ConfigProperty[Boolean] = ConfigProperty
+ .key("hoodie.datasource.hive_sync.use_kerberos")
+ .defaultValue(false)
+ .withDocumentation("Whether to use Kerberos authentication.")
+
+ val HIVE_SYNC_KERBEROS_PRINCIPAL: ConfigProperty[String] = ConfigProperty
+ .key("hive_sync.kerberos_principal")
+ .noDefaultValue()
Review comment:
Has been adjusted
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
,
{ "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" },
{ "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" } ]
}-->
-
- CI report:
- f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091)
- aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN
- ec23d38f4f7c400361059023eddeac128e5e32e5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu commented on a change in pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#discussion_r704002183
##########
File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java
##########
@@ -369,6 +374,8 @@
conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions);
conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix);
conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp);
+ conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos);
+ conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal);
return conf;
Review comment:
I added judgment to config.kerberosprincipal , setting the value if Kerberos is enabled and null otherwise
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
,
{ "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" },
{ "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" } ]
}-->
-
- CI report:
- f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091)
- aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN
- ec23d38f4f7c400361059023eddeac128e5e32e5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
danny0405 commented on a change in pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#discussion_r704011037
##########
File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java
##########
@@ -369,6 +374,8 @@
conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions);
conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix);
conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp);
+ conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos);
+ conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal);
return conf;
Review comment:
I mean Flink Configuration does not support set up `null` as value now.
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
hudi-bot edited a comment on pull request #3622:
URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [
,
{ "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" },
{ "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" } ]
}-->
-
- CI report:
- aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN
- ec23d38f4f7c400361059023eddeac128e5e32e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
test-wangxiaoyu opened a new pull request #3611:
URL: https://github.com/apache/hudi/pull/3611
For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
Hive.get(configuration).getMSC()
Methods before
Perform Kerberos authentication on the Flink client
Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
I added two parameters
1.hive_sync.use_kerberos
2.hive_sync.kerberos_principal
Use the following method to enable Hive Kerberos access control
CREATE TABLE t2(
Uuid VARCHAR (20),
The name VARCHAR (10),
The age INT,
Ts TIMESTAMP (3),
` partition ` VARCHAR (20)
)
PARTITIONED BY (`partition`)
with(
'connector' = 'hudi',
'hive_sync.enable'='true',
'hive_sync.db'='test',
'hive_sync.table'='t2',
'hive_sync.mode'='hms',
'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
'hive_sync.metastore.uris'='thrift://ip:9083',
'hive_sync.use_kerberos' = 'true',
'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
)
–
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org