Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8341

Transaction information in config file can grow excessively large

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.14.0
    • 0.14.0
    • Transactions
    • None
    • Hide
      A new environment variable hive.script.operator.env.blacklist was added in 0.14. Its default value is "hive.txn.valid.txns,hive.script.operator.env.blacklist"

      By default all values in the HiveConf object are converted to environment variables of the same name as the key (with '.' (dot) converted to '_' (underscore)) and set as part of the script operator's environment. However, some values can grow large or are not amenable to translation to environment variables. This value gives a comma separated list of configuration values that will not be set in the environment when calling a script operator. By default the valid transaction list is excluded, as it can grow large and is sometimes compressed, which does not translate well to an environment variable.
      Show
      A new environment variable hive.script.operator.env.blacklist was added in 0.14. Its default value is "hive.txn.valid.txns,hive.script.operator.env.blacklist" By default all values in the HiveConf object are converted to environment variables of the same name as the key (with '.' (dot) converted to '_' (underscore)) and set as part of the script operator's environment. However, some values can grow large or are not amenable to translation to environment variables. This value gives a comma separated list of configuration values that will not be set in the environment when calling a script operator. By default the valid transaction list is excluded, as it can grow large and is sometimes compressed, which does not translate well to an environment variable.

    Description

      In our testing we have seen cases where the transaction list grows very large. We need a more efficient way of communicating the list.

      Attachments

        1. HIVE-8341.patch
          6 kB
          Alan Gates
        2. HIVE-8341.2.patch
          16 kB
          Alan Gates
        3. HIVE-8341.3.patch
          16 kB
          Alan Gates

        Activity

          People

            gates Alan Gates
            gates Alan Gates
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: