Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2051

getInputSummary() to call FileSystem.getContentSummary() in parallel

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      getInputSummary() now call FileSystem.getContentSummary() one by one, which can be extremely slow when the number of input paths are huge. By calling those functions in parallel, we can cut latency in most cases.

        Attachments

        1. HIVE-2051.5.patch
          12 kB
          Siying Dong
        2. HIVE-2051.4.patch
          7 kB
          Siying Dong
        3. HIVE-2051.3.patch
          7 kB
          Siying Dong
        4. HIVE-2051.2.patch
          7 kB
          Siying Dong
        5. HIVE-2051.1.patch
          6 kB
          Siying Dong

          Activity

            People

            • Assignee:
              sdong Siying Dong
              Reporter:
              sdong Siying Dong
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: