Hive
  1. Hive
  2. HIVE-1889

add an option (hive.index.compact.file.ignore.hdfs) to ignore HDFS location stored in index files.

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: Indexing
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In the current code, we use the whole hdfs path in index files to choose splits to run mapreduce job.

      If the data got moved or the name of the cluster got changed, the index data should still be usable.

      1. HIVE-1889.1.patch
        4 kB
        He Yongqiang
      2. HIVE-1889.2.patch
        5 kB
        He Yongqiang

        Issue Links

          Activity

          Hide
          Namit Jain added a comment -

          Committed. Thanks Yongqiang

          Show
          Namit Jain added a comment - Committed. Thanks Yongqiang
          Hide
          Namit Jain added a comment -

          +1

          Show
          Namit Jain added a comment - +1
          Hide
          He Yongqiang added a comment -

          @Carl,
          added the option name to the title of this jira.

          Show
          He Yongqiang added a comment - @Carl, added the option name to the title of this jira.
          Hide
          Carl Steinbach added a comment -

          @Yongqiang: can you please add the name of the new property to this ticket so that people can find it via search?

          Also, do we lose anything in terms of functionality or performance by ignoring the HDFS path information?
          If the answer is no then we should probably just ignore this information by default instead of adding a new
          configuration property.

          Show
          Carl Steinbach added a comment - @Yongqiang: can you please add the name of the new property to this ticket so that people can find it via search? Also, do we lose anything in terms of functionality or performance by ignoring the HDFS path information? If the answer is no then we should probably just ignore this information by default instead of adding a new configuration property.
          Hide
          Namit Jain added a comment -

          Can you add the new parameter in HiveConf.java and hive-default.xml ?

          Show
          Namit Jain added a comment - Can you add the new parameter in HiveConf.java and hive-default.xml ?

            People

            • Assignee:
              He Yongqiang
              Reporter:
              He Yongqiang
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development