Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5193

Columnar Pushdown for RC/ORC File not happening in HCatLoader

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0, 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: HCatalog
    • Labels:
    • Release Note:
      (Columnar pushdown will now work with HCatLoader - basically, what this means is that when using RC/ORC tables, not all rows are necessarily read and processed if pig would discard (by filtering) it anyway. This allows reads to be a bit faster.)
    • Tags:
      hcatalog, columnar pushdown

      Description

      Currently the HCatLoader is not taking advantage of the ColumnProjectionUtils. where it could skip columns during read. The information is available in Pig it just needs to get to the Readers.
      Viraj

        Attachments

        1. HIVE-5193.2.patch
          5 kB
          Viraj Bhat
        2. HIVE-5193.patch
          5 kB
          Viraj Bhat

          Issue Links

            Activity

              People

              • Assignee:
                viraj Viraj Bhat
                Reporter:
                viraj Viraj Bhat
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: