Hive
  1. Hive
  2. HIVE-4732

Reduce or eliminate the expensive Schema equals() check for AvroSerde

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Labels:
      None

      Description

      The AvroSerde spends a significant amount of time checking schema equality. Changing to compare hashcodes (which can be computed once then reused) will improve performance.

      1. HIVE-4732.1.patch
        1 kB
        Mark Wagner
      2. HIVE-4732.4.patch
        17 kB
        Mohammad Kamrul Islam
      3. HIVE-4732.5.patch
        18 kB
        Mohammad Kamrul Islam
      4. HIVE-4732.6.patch
        18 kB
        Mohammad Kamrul Islam
      5. HIVE-4732.7.patch
        18 kB
        Mohammad Kamrul Islam
      6. HIVE-4732.v1.patch
        16 kB
        Mohammad Kamrul Islam
      7. HIVE-4732.v4.patch
        17 kB
        Mohammad Kamrul Islam

        Activity

        Mark Wagner created issue -
        Mark Wagner made changes -
        Field Original Value New Value
        Attachment HIVE-4732.1.patch [ 12587728 ]
        Mark Wagner made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.11.1 [ 12324446 ]
        Fix Version/s 0.12.0 [ 12324312 ]
        Ashutosh Chauhan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Affects Version/s 0.11.0 [ 12323587 ]
        Affects Version/s 0.12.0 [ 12324312 ]
        Fix Version/s 0.12.0 [ 12324312 ]
        Fix Version/s 0.11.1 [ 12324446 ]
        Mark Wagner made changes -
        Assignee Mark Wagner [ mwagner ] Mohammad Kamrul Islam [ kamrul ]
        Mohammad Kamrul Islam made changes -
        Summary Speed up AvroSerde by checking hashcodes instead of equality Reduce or eliminate the expensive Schema equals() check for AvroSerde
        Mohammad Kamrul Islam made changes -
        Attachment HIVE-4732.v1.patch [ 12591925 ]
        Mohammad Kamrul Islam made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Edward Capriolo made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Mohammad Kamrul Islam made changes -
        Attachment HIVE-4732.v4.patch [ 12603116 ]
        Mohammad Kamrul Islam made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Mohammad Kamrul Islam made changes -
        Attachment HIVE-4732.4.patch [ 12603120 ]
        Mohammad Kamrul Islam made changes -
        Attachment HIVE-4732.5.patch [ 12603500 ]
        Mohammad Kamrul Islam made changes -
        Attachment HIVE-4732.6.patch [ 12604084 ]
        Mohammad Kamrul Islam made changes -
        Attachment HIVE-4732.7.patch [ 12604184 ]
        Ashutosh Chauhan made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.13.0 [ 12324986 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Mohammad Kamrul Islam
            Reporter:
            Mark Wagner
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development