Hive
  1. Hive
  2. HIVE-2051

getInputSummary() to call FileSystem.getContentSummary() in parallel

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      getInputSummary() now call FileSystem.getContentSummary() one by one, which can be extremely slow when the number of input paths are huge. By calling those functions in parallel, we can cut latency in most cases.

      1. HIVE-2051.5.patch
        12 kB
        Siying Dong
      2. HIVE-2051.4.patch
        7 kB
        Siying Dong
      3. HIVE-2051.3.patch
        7 kB
        Siying Dong
      4. HIVE-2051.2.patch
        7 kB
        Siying Dong
      5. HIVE-2051.1.patch
        6 kB
        Siying Dong

        Activity

        Siying Dong created issue -
        Siying Dong made changes -
        Field Original Value New Value
        Assignee Siying Dong [ sdong ]
        Siying Dong made changes -
        Attachment HIVE-2051.1.patch [ 12473439 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Carl Steinbach made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2051.2.patch [ 12473627 ]
        Siying Dong made changes -
        Attachment HIVE-2051.3.patch [ 12473628 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Siying Dong made changes -
        Attachment HIVE-2051.4.patch [ 12473964 ]
        Siying Dong made changes -
        Attachment HIVE-2051.5.patch [ 12474151 ]
        Siying Dong made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Fix Version/s 0.8.0 [ 12316178 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Siying Dong
            Reporter:
            Siying Dong
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development