Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-2402

Hive Sync supports Kerberos authentication

Details

    Description

      For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
      Hive.get(configuration).getMSC()
      Methods before
      Perform Kerberos authentication on the Flink client
      Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
      I added two parameters
      1.hive_sync.use_kerberos
      2.hive_sync.kerberos_principal
      Use the following method to enable Hive Kerberos access control
      CREATE TABLE t2(
      Uuid VARCHAR (20),
      The name VARCHAR (10),
      The age INT,
      Ts TIMESTAMP (3),
      ` partition ` VARCHAR (20)
      )
      PARTITIONED BY (`partition`)
      with(
      'connector' = 'hudi',
      'hive_sync.enable'='true',
      'hive_sync.db'='test',
      'hive_sync.table'='t2',
      'hive_sync.mode'='hms',
      'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
      'hive_sync.metastore.uris'='thrift://ip:9083',
      'hive_sync.use_kerberos' = 'true',
      'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
      )

      Attachments

        Activity

          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3611:
          URL: https://github.com/apache/hudi/pull/3611

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          ` partition ` VARCHAR (20)
          )
          PARTITIONED BY (`partition`)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3611: URL: https://github.com/apache/hudi/pull/3611 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), ` partition ` VARCHAR (20) ) PARTITIONED BY (`partition`) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot commented on pull request #3611:
          URL: https://github.com/apache/hudi/pull/3611#issuecomment-914085520

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "178767948e906f673d6d4a357c65c11bc574f619", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "178767948e906f673d6d4a357c65c11bc574f619", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:
          • 178767948e906f673d6d4a357c65c11bc574f619 UNKNOWN

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot commented on pull request #3611: URL: https://github.com/apache/hudi/pull/3611#issuecomment-914085520 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "178767948e906f673d6d4a357c65c11bc574f619", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "178767948e906f673d6d4a357c65c11bc574f619", "triggerType" : "PUSH" } ] }--> CI report: 178767948e906f673d6d4a357c65c11bc574f619 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3611:
          URL: https://github.com/apache/hudi/pull/3611#issuecomment-914085520

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "178767948e906f673d6d4a357c65c11bc574f619", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2063", "triggerID" : "178767948e906f673d6d4a357c65c11bc574f619", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3611: URL: https://github.com/apache/hudi/pull/3611#issuecomment-914085520 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "178767948e906f673d6d4a357c65c11bc574f619", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2063", "triggerID" : "178767948e906f673d6d4a357c65c11bc574f619", "triggerType" : "PUSH" } ] }--> CI report: 178767948e906f673d6d4a357c65c11bc574f619 Azure: [PENDING] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2063 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          partition VARCHAR (20)
          )
          PARTITIONED BY (partition)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3613: URL: https://github.com/apache/hudi/pull/3613 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), partition VARCHAR (20) ) PARTITIONED BY (partition) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3611:
          URL: https://github.com/apache/hudi/pull/3611

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3611: URL: https://github.com/apache/hudi/pull/3611 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3613: URL: https://github.com/apache/hudi/pull/3613 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu commented on pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613#issuecomment-914111512

          111

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu commented on pull request #3613: URL: https://github.com/apache/hudi/pull/3613#issuecomment-914111512 111 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot commented on pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613#issuecomment-914114267

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:
          • b7972ad260742b097ffaf740e8bd258bb49d4b23 UNKNOWN

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot commented on pull request #3613: URL: https://github.com/apache/hudi/pull/3613#issuecomment-914114267 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "triggerType" : "PUSH" } ] }--> CI report: b7972ad260742b097ffaf740e8bd258bb49d4b23 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613#issuecomment-914114267

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2064", "triggerID" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3613: URL: https://github.com/apache/hudi/pull/3613#issuecomment-914114267 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2064", "triggerID" : "b7972ad260742b097ffaf740e8bd258bb49d4b23", "triggerType" : "PUSH" } ] }--> CI report: b7972ad260742b097ffaf740e8bd258bb49d4b23 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2064 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu removed a comment on pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613#issuecomment-914111512

          111

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu removed a comment on pull request #3613: URL: https://github.com/apache/hudi/pull/3613#issuecomment-914111512 111 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3613: URL: https://github.com/apache/hudi/pull/3613 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu commented on pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613#issuecomment-914128031

          1111

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu commented on pull request #3613: URL: https://github.com/apache/hudi/pull/3613#issuecomment-914128031 1111 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu removed a comment on pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613#issuecomment-914128031

          1111

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu removed a comment on pull request #3613: URL: https://github.com/apache/hudi/pull/3613#issuecomment-914128031 1111 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          partition VARCHAR (20)
          )
          PARTITIONED BY (partition)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3613: URL: https://github.com/apache/hudi/pull/3613 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), partition VARCHAR (20) ) PARTITIONED BY (partition) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3613:
          URL: https://github.com/apache/hudi/pull/3613

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3613: URL: https://github.com/apache/hudi/pull/3613 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3615:
          URL: https://github.com/apache/hudi/pull/3615

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          partition VARCHAR (20)
          )
          PARTITIONED BY (partition)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3615: URL: https://github.com/apache/hudi/pull/3615 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), partition VARCHAR (20) ) PARTITIONED BY (partition) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot commented on pull request #3615:
          URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:
          • d8ccec907403ee123b5a66a43ea99677f46be5a4 UNKNOWN

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot commented on pull request #3615: URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "triggerType" : "PUSH" } ] }--> CI report: d8ccec907403ee123b5a66a43ea99677f46be5a4 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3615:
          URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068", "triggerID" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3615: URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068", "triggerID" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "triggerType" : "PUSH" } ] }--> CI report: d8ccec907403ee123b5a66a43ea99677f46be5a4 Azure: [PENDING] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3615:
          URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068", "triggerID" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3615: URL: https://github.com/apache/hudi/pull/3615#issuecomment-914151953 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068", "triggerID" : "d8ccec907403ee123b5a66a43ea99677f46be5a4", "triggerType" : "PUSH" } ] }--> CI report: d8ccec907403ee123b5a66a43ea99677f46be5a4 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2068 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3615:
          URL: https://github.com/apache/hudi/pull/3615

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3615: URL: https://github.com/apache/hudi/pull/3615 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3619:
          URL: https://github.com/apache/hudi/pull/3619

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          partition VARCHAR (20)
          )
          PARTITIONED BY (partition)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3619: URL: https://github.com/apache/hudi/pull/3619 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), partition VARCHAR (20) ) PARTITIONED BY (partition) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot commented on pull request #3619:
          URL: https://github.com/apache/hudi/pull/3619#issuecomment-914439925

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:
          • 8a640c1fd1342fa75a8089efa8ba549e1b94f5ba UNKNOWN

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot commented on pull request #3619: URL: https://github.com/apache/hudi/pull/3619#issuecomment-914439925 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "triggerType" : "PUSH" } ] }--> CI report: 8a640c1fd1342fa75a8089efa8ba549e1b94f5ba UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3619:
          URL: https://github.com/apache/hudi/pull/3619#issuecomment-914439925

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "status" : "CANCELED", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2076", "triggerID" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3619: URL: https://github.com/apache/hudi/pull/3619#issuecomment-914439925 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "status" : "CANCELED", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2076", "triggerID" : "8a640c1fd1342fa75a8089efa8ba549e1b94f5ba", "triggerType" : "PUSH" } ] }--> CI report: 8a640c1fd1342fa75a8089efa8ba549e1b94f5ba Azure: [CANCELED] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2076 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3619:
          URL: https://github.com/apache/hudi/pull/3619

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3619: URL: https://github.com/apache/hudi/pull/3619 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3621:
          URL: https://github.com/apache/hudi/pull/3621

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          partition VARCHAR (20)
          )
          PARTITIONED BY (partition)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3621: URL: https://github.com/apache/hudi/pull/3621 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), partition VARCHAR (20) ) PARTITIONED BY (partition) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot commented on pull request #3621:
          URL: https://github.com/apache/hudi/pull/3621#issuecomment-914737057

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:
          • 824df17f32aaff74bfb8d5d4a1001b7b320c4915 UNKNOWN

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot commented on pull request #3621: URL: https://github.com/apache/hudi/pull/3621#issuecomment-914737057 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "triggerType" : "PUSH" } ] }--> CI report: 824df17f32aaff74bfb8d5d4a1001b7b320c4915 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3621:
          URL: https://github.com/apache/hudi/pull/3621#issuecomment-914737057

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2090", "triggerID" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3621: URL: https://github.com/apache/hudi/pull/3621#issuecomment-914737057 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2090", "triggerID" : "824df17f32aaff74bfb8d5d4a1001b7b320c4915", "triggerType" : "PUSH" } ] }--> CI report: 824df17f32aaff74bfb8d5d4a1001b7b320c4915 Azure: [PENDING] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2090 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu closed pull request #3621:
          URL: https://github.com/apache/hudi/pull/3621

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu closed pull request #3621: URL: https://github.com/apache/hudi/pull/3621 – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu opened a new pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622

          For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata
          Hive.get(configuration).getMSC()
          Methods before
          Perform Kerberos authentication on the Flink client
          Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive
          I added two parameters
          1.hive_sync.use_kerberos
          2.hive_sync.kerberos_principal
          Use the following method to enable Hive Kerberos access control
          CREATE TABLE t2(
          Uuid VARCHAR (20),
          The name VARCHAR (10),
          The age INT,
          Ts TIMESTAMP (3),
          partition VARCHAR (20)
          )
          PARTITIONED BY (partition)
          with(
          'connector' = 'hudi',
          'hive_sync.enable'='true',
          'hive_sync.db'='test',
          'hive_sync.table'='t2',
          'hive_sync.mode'='hms',
          'path' = 'hdfs://ip:8020/warehouse/hudi/t2',
          'hive_sync.metastore.uris'='thrift://ip:9083',
          'hive_sync.use_kerberos' = 'true',
          'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA'
          )

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu opened a new pull request #3622: URL: https://github.com/apache/hudi/pull/3622 For Hive with Kerberos enabled, THE HMS of HUDI cannot access the Hive metadata Hive.get(configuration).getMSC() Methods before Perform Kerberos authentication on the Flink client Set the parameters for enabling Kerberos authentication and the configuration parameters for transmitting hive Principal to Hive I added two parameters 1.hive_sync.use_kerberos 2.hive_sync.kerberos_principal Use the following method to enable Hive Kerberos access control CREATE TABLE t2( Uuid VARCHAR (20), The name VARCHAR (10), The age INT, Ts TIMESTAMP (3), partition VARCHAR (20) ) PARTITIONED BY (partition) with( 'connector' = 'hudi', 'hive_sync.enable'='true', 'hive_sync.db'='test', 'hive_sync.table'='t2', 'hive_sync.mode'='hms', 'path' = 'hdfs://ip:8020/warehouse/hudi/t2', 'hive_sync.metastore.uris'='thrift://ip:9083', 'hive_sync.use_kerberos' = 'true', 'hive_sync.kerberos_principal' = 'hive/_HOST@BIGDATA' ) – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot commented on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:
          • f2eaffe6a6bdd3270319ceac5541efda4e42b302 UNKNOWN

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot commented on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } ] }--> CI report: f2eaffe6a6bdd3270319ceac5541efda4e42b302 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } ] }--> CI report: f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [PENDING] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } ] }--> CI report: f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          danny0405 commented on a change in pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#discussion_r703976033

          ##########
          File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java
          ##########
          @@ -369,6 +374,8 @@
          conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions);
          conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix);
          conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp);
          + conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos);
          + conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal);
          return conf;

          Review comment:
          Should decide if `config.kerberosPrincipal` is null first.

          ##########
          File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
          ##########
          @@ -191,6 +191,16 @@ object DataSourceWriteOptions {
          .withAlternatives("hoodie.datasource.write.storage.type")
          .withDocumentation("The table type for the underlying data, for this write. This can’t change between writes.")

          + val HIVE_SYNC_USE_KERBEROS: ConfigProperty[Boolean] = ConfigProperty
          + .key("hoodie.datasource.hive_sync.use_kerberos")
          + .defaultValue(false)
          + .withDocumentation("Whether to use Kerberos authentication.")
          +
          + val HIVE_SYNC_KERBEROS_PRINCIPAL: ConfigProperty[String] = ConfigProperty
          + .key("hive_sync.kerberos_principal")
          + .noDefaultValue()

          Review comment:
          hoodie.datasource.hive_sync.kerberos_principal ? And move this option together with the other hive sync options.

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - danny0405 commented on a change in pull request #3622: URL: https://github.com/apache/hudi/pull/3622#discussion_r703976033 ########## File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java ########## @@ -369,6 +374,8 @@ conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions); conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix); conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp); + conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos); + conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal); return conf; Review comment: Should decide if `config.kerberosPrincipal` is null first. ########## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ########## @@ -191,6 +191,16 @@ object DataSourceWriteOptions { .withAlternatives("hoodie.datasource.write.storage.type") .withDocumentation("The table type for the underlying data, for this write. This can’t change between writes.") + val HIVE_SYNC_USE_KERBEROS: ConfigProperty [Boolean] = ConfigProperty + .key("hoodie.datasource.hive_sync.use_kerberos") + .defaultValue(false) + .withDocumentation("Whether to use Kerberos authentication.") + + val HIVE_SYNC_KERBEROS_PRINCIPAL: ConfigProperty [String] = ConfigProperty + .key("hive_sync.kerberos_principal") + .noDefaultValue() Review comment: hoodie.datasource.hive_sync.kerberos_principal ? And move this option together with the other hive sync options. – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ,

          { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } , { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" } ] }--> CI report: f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091 ) aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu commented on a change in pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#discussion_r704001915

          ##########
          File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
          ##########
          @@ -191,6 +191,16 @@ object DataSourceWriteOptions {
          .withAlternatives("hoodie.datasource.write.storage.type")
          .withDocumentation("The table type for the underlying data, for this write. This can’t change between writes.")

          + val HIVE_SYNC_USE_KERBEROS: ConfigProperty[Boolean] = ConfigProperty
          + .key("hoodie.datasource.hive_sync.use_kerberos")
          + .defaultValue(false)
          + .withDocumentation("Whether to use Kerberos authentication.")
          +
          + val HIVE_SYNC_KERBEROS_PRINCIPAL: ConfigProperty[String] = ConfigProperty
          + .key("hive_sync.kerberos_principal")
          + .noDefaultValue()

          Review comment:
          Has been adjusted

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu commented on a change in pull request #3622: URL: https://github.com/apache/hudi/pull/3622#discussion_r704001915 ########## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ########## @@ -191,6 +191,16 @@ object DataSourceWriteOptions { .withAlternatives("hoodie.datasource.write.storage.type") .withDocumentation("The table type for the underlying data, for this write. This can’t change between writes.") + val HIVE_SYNC_USE_KERBEROS: ConfigProperty [Boolean] = ConfigProperty + .key("hoodie.datasource.hive_sync.use_kerberos") + .defaultValue(false) + .withDocumentation("Whether to use Kerberos authentication.") + + val HIVE_SYNC_KERBEROS_PRINCIPAL: ConfigProperty [String] = ConfigProperty + .key("hive_sync.kerberos_principal") + .noDefaultValue() Review comment: Has been adjusted – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ,

          { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" }

          ,

          { "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } , { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" } , { "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" } ] }--> CI report: f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091 ) aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN ec23d38f4f7c400361059023eddeac128e5e32e5 UNKNOWN <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          test-wangxiaoyu commented on a change in pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#discussion_r704002183

          ##########
          File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java
          ##########
          @@ -369,6 +374,8 @@
          conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions);
          conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix);
          conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp);
          + conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos);
          + conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal);
          return conf;

          Review comment:
          I added judgment to config.kerberosprincipal , setting the value if Kerberos is enabled and null otherwise

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - test-wangxiaoyu commented on a change in pull request #3622: URL: https://github.com/apache/hudi/pull/3622#discussion_r704002183 ########## File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java ########## @@ -369,6 +374,8 @@ conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions); conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix); conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp); + conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos); + conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal); return conf; Review comment: I added judgment to config.kerberosprincipal , setting the value if Kerberos is enabled and null otherwise – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ,

          { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" }

          ,

          { "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } , { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" } , { "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "PENDING", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" } ] }--> CI report: f2eaffe6a6bdd3270319ceac5541efda4e42b302 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091 ) aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN ec23d38f4f7c400361059023eddeac128e5e32e5 Azure: [PENDING] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          danny0405 commented on a change in pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#discussion_r704011037

          ##########
          File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java
          ##########
          @@ -369,6 +374,8 @@
          conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions);
          conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix);
          conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp);
          + conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos);
          + conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal);
          return conf;

          Review comment:
          I mean Flink Configuration does not support set up `null` as value now.

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - danny0405 commented on a change in pull request #3622: URL: https://github.com/apache/hudi/pull/3622#discussion_r704011037 ########## File path: hudi-flink/src/main/java/org/apache/hudi/streamer/FlinkStreamerConfig.java ########## @@ -369,6 +374,8 @@ conf.setBoolean(FlinkOptions.HIVE_SYNC_IGNORE_EXCEPTIONS, config.hiveSyncIgnoreExceptions); conf.setBoolean(FlinkOptions.HIVE_SYNC_SKIP_RO_SUFFIX, config.hiveSyncSkipRoSuffix); conf.setBoolean(FlinkOptions.HIVE_SYNC_SUPPORT_TIMESTAMP, config.hiveSyncSupportTimestamp); + conf.setBoolean(FlinkOptions.HIVE_SYNC_USE_KERBEROS, config.useKerberos); + conf.setString(FlinkOptions.HIVE_SYNC_KERBEROS_PRINCIPAL, config.kerberosPrincipal); return conf; Review comment: I mean Flink Configuration does not support set up `null` as value now. – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
          githubbot ASF GitHub Bot added a comment -

          hudi-bot edited a comment on pull request #3622:
          URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618

          <!--
          Meta data
          {
          "version" : 1,
          "metaDataEntries" : [

          { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "DELETED", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" }

          ,

          { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" }

          ,

          { "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" }

          ]
          }-->

            1. CI report:

          <details>
          <summary>Bot commands</summary>
          @hudi-bot supports the following commands:

          • `@hudi-bot run travis` re-run the last Travis build
          • `@hudi-bot run azure` re-run the last Azure build
            </details>

          –
          This is an automated message from the Apache Git Service.
          To respond to the message, please log on to GitHub and use the
          URL above to go to the specific comment.

          To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

          For queries about this service, please contact Infrastructure at:
          users@infra.apache.org

          githubbot ASF GitHub Bot added a comment - hudi-bot edited a comment on pull request #3622: URL: https://github.com/apache/hudi/pull/3622#issuecomment-914748618 <!-- Meta data { "version" : 1, "metaDataEntries" : [ { "hash" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "status" : "DELETED", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2091", "triggerID" : "f2eaffe6a6bdd3270319ceac5541efda4e42b302", "triggerType" : "PUSH" } , { "hash" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "status" : "UNKNOWN", "url" : "TBD", "triggerID" : "aae3c22b5ab176daf990c1fa03384702baa87a25", "triggerType" : "PUSH" } , { "hash" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "status" : "FAILURE", "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094", "triggerID" : "ec23d38f4f7c400361059023eddeac128e5e32e5", "triggerType" : "PUSH" } ] }--> CI report: aae3c22b5ab176daf990c1fa03384702baa87a25 UNKNOWN ec23d38f4f7c400361059023eddeac128e5e32e5 Azure: [FAILURE] ( https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2094 ) <details> <summary>Bot commands</summary> @hudi-bot supports the following commands: `@hudi-bot run travis` re-run the last Travis build `@hudi-bot run azure` re-run the last Azure build </details> – This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org

          People

            Unassigned Unassigned
            王晓宇 王晓宇
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 3h
                3h
                Remaining:
                Remaining Estimate - 3h
                3h
                Logged:
                Time Spent - Not Specified
                Not Specified