Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8341

Transaction information in config file can grow excessively large

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.14.0
    • Fix Version/s: 0.14.0
    • Component/s: Transactions
    • Labels:
      None
    • Release Note:
      Hide
      A new environment variable hive.script.operator.env.blacklist was added in 0.14. Its default value is "hive.txn.valid.txns,hive.script.operator.env.blacklist"

      By default all values in the HiveConf object are converted to environment variables of the same name as the key (with '.' (dot) converted to '_' (underscore)) and set as part of the script operator's environment. However, some values can grow large or are not amenable to translation to environment variables. This value gives a comma separated list of configuration values that will not be set in the environment when calling a script operator. By default the valid transaction list is excluded, as it can grow large and is sometimes compressed, which does not translate well to an environment variable.
      Show
      A new environment variable hive.script.operator.env.blacklist was added in 0.14. Its default value is "hive.txn.valid.txns,hive.script.operator.env.blacklist" By default all values in the HiveConf object are converted to environment variables of the same name as the key (with '.' (dot) converted to '_' (underscore)) and set as part of the script operator's environment. However, some values can grow large or are not amenable to translation to environment variables. This value gives a comma separated list of configuration values that will not be set in the environment when calling a script operator. By default the valid transaction list is excluded, as it can grow large and is sometimes compressed, which does not translate well to an environment variable.

      Description

      In our testing we have seen cases where the transaction list grows very large. We need a more efficient way of communicating the list.

        Attachments

        1. HIVE-8341.2.patch
          16 kB
          Alan Gates
        2. HIVE-8341.3.patch
          16 kB
          Alan Gates
        3. HIVE-8341.patch
          6 kB
          Alan Gates

          Activity

            People

            • Assignee:
              alangates Alan Gates
              Reporter:
              alangates Alan Gates
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: