Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10114

Split strategies for ORC

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 1.2.0
    • None
    • None

    Description

      ORC split generation does not have clearly defined strategies for different scenarios (many small orc files, few small orc files, many large files etc.). Few strategies like storing the file footer in orc split, making entire file as a orc split already exists. This JIRA to make the split generation simpler, support different strategies for various use cases (BI, ETL, ACID etc.) and to lay the foundation for HIVE-7428.

      Attachments

        1. HIVE-10114.5.patch
          50 kB
          Prasanth Jayachandran
        2. HIVE-10114.4.patch
          50 kB
          Prasanth Jayachandran
        3. HIVE-10114.3.patch
          50 kB
          Prasanth Jayachandran
        4. HIVE-10114.2.patch
          51 kB
          Prasanth Jayachandran
        5. HIVE-10114.1.patch
          50 kB
          Prasanth Jayachandran

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-11043 ORC split strategies should adapt based on number of files

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-11546 Projected columns read size should be scaled to split size for ORC Splits

          • Major - Major loss of function.
          • Closed
          links to

          Web Link Review Board

          Activity

            People

              prasanth_j Prasanth Jayachandran
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: