Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      Compression runtime is increased by 13% as well as throughput is decreased by 24% in 0.23 when compared to 0.20.204 on
      a 350 node size cluster.

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          What's the test/benchmark used for this result?

          Show
          Todd Lipcon added a comment - What's the test/benchmark used for this result?
          Hide
          Vinay Kumar Thota added a comment -

          Ran compression benchmark with following environments.

          Codec type : BLOCK
          Codec : com.hadoop.compression.lzo.LzoCodec

          Show
          Vinay Kumar Thota added a comment - Ran compression benchmark with following environments. Codec type : BLOCK Codec : com.hadoop.compression.lzo.LzoCodec
          Hide
          Scott Carey added a comment -

          Is this also a sub-task of MAPREDUCE-3561?

          Show
          Scott Carey added a comment - Is this also a sub-task of MAPREDUCE-3561 ?
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Is this also a sub-task of MAPREDUCE-3561?

          Yeah, I couldn't find this ticket before.

          But my guess is most of those sub-tasks are related and most likely will be fixed in one shot. Let's see.

          Show
          Vinod Kumar Vavilapalli added a comment - Is this also a sub-task of MAPREDUCE-3561 ? Yeah, I couldn't find this ticket before. But my guess is most of those sub-tasks are related and most likely will be fixed in one shot. Let's see.
          Hide
          Amol Kekre added a comment -

          We would post new compression numbers post MAPREDUCE-3641

          Show
          Amol Kekre added a comment - We would post new compression numbers post MAPREDUCE-3641
          Hide
          Arun C Murthy added a comment -

          This was mostly due to misconfiguration, but MAPREDUCE-3641 does help a lot.

          Show
          Arun C Murthy added a comment - This was mostly due to misconfiguration, but MAPREDUCE-3641 does help a lot.
          Hide
          Amol Kekre added a comment -

          Compression is now 10.4% faster in .23 (compared to .205/1.0). Please close this jira

          Show
          Amol Kekre added a comment - Compression is now 10.4% faster in .23 (compared to .205/1.0). Please close this jira
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Thanks for the update Amol. Thanks to Sid and Vinay who've been running this and verifying it.

          This is fixed now after the recent commits tracked at MAPREDUCE-3561.

          Show
          Vinod Kumar Vavilapalli added a comment - Thanks for the update Amol. Thanks to Sid and Vinay who've been running this and verifying it. This is fixed now after the recent commits tracked at MAPREDUCE-3561 .

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinay Kumar Thota
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development