Hive
  1. Hive
  2. HIVE-2068

Speed up query "select xx,xx from xxx LIMIT xxx" if no filtering or aggregation

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, "select xx,xx from xxx where ...(only partition conditions) LIMIT xxx" will start a MapReduce job with input to be the whole table or partition. The latency can be huge if the table or partition is big. We could reduce number of input files to speed up the queries.

      1. HIVE-2068.6.patch
        79 kB
        Siying Dong
      2. HIVE-2068.5.patch
        79 kB
        Siying Dong
      3. HIVE-2068.4.patch
        80 kB
        Siying Dong
      4. HIVE-2068.3.patch
        78 kB
        Siying Dong
      5. HIVE-2068.2.patch
        77 kB
        Siying Dong
      6. HIVE-2068.1.patch
        72 kB
        Siying Dong

        Activity

        Siying Dong created issue -
        Siying Dong made changes -
        Field Original Value New Value
        Attachment HIVE-2068.1.patch [ 12474856 ]
        Siying Dong made changes -
        Attachment HIVE-2068.2.patch [ 12474952 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2068.3.patch [ 12475074 ]
        Namit Jain made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2068.4.patch [ 12475549 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Namit Jain made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2068.5.patch [ 12476070 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2068.6.patch [ 12476472 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Siying Dong made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2068.6.patch [ 12476472 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Siying Dong made changes -
        Attachment HIVE-2068.6.patch [ 12476653 ]
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Fix Version/s 0.8.0 [ 12316178 ]
        Component/s Query Processor [ 12312586 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Siying Dong
            Reporter:
            Siying Dong
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development