Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17994

Vectorization: Serialization bottlenecked on irrelevant hashmap lookup

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 3.0.0
    • None
    • None

    Description

      On machines with slower NUMA, the hashmap lookup for TypeInfo::getPrimitiveCategory is the slowest part of the vectorized serialization loops. The static object references run hot with the NUMA access speeds penalizing half the threads.

      This lookup is done for every column, for every row - though vectorization enforces that this type cannot change at all.

      Attachments

        1. vec-serialize-hashmap.png
          153 kB
          Gopal Vijayaraghavan
        2. HIVE-17994.01.patch
          5 kB
          Matt McCline
        3. HIVE-17994.02.patch
          5 kB
          Matt McCline
        4. HIVE-17994.03.patch
          5 kB
          Matt McCline
        5. HIVE-17994.04.patch
          2 kB
          Matt McCline
        6. HIVE-17994.05.patch
          2 kB
          Matt McCline
        7. HIVE-17994.06.patch
          22 kB
          Matt McCline

        Activity

          People

            mmccline Matt McCline
            gopalv Gopal Vijayaraghavan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: