Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4113

TezUtils::createByteStringFromConf should use snappy instead of DeflaterOutputStream

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • None
    • 0.10.0
    • None
    • None

    Description

      Under concurrent workload, where lots of short running DAGs were submitted in Hive, HS2 spikes up heavily on CPU due to TezUtils::createByteStringFromConf

      Attachments

        1. Screenshot 2020-01-10 at 6.32.31 AM.png
          971 kB
          Rajesh Balamohan
        2. TEZ-4113.1.patch
          2 kB
          Rajesh Balamohan

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. TEZ-4319 Add explicit dependency on snappy-java to tez-api

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              rajesh.balamohan Rajesh Balamohan
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: