Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      Compression runtime is increased by 13% as well as throughput is decreased by 24% in 0.23 when compared to 0.20.204 on
      a 350 node size cluster.

        Issue Links

          Activity

          Vinay Kumar Thota created issue -
          Mahadev konar made changes -
          Field Original Value New Value
          Priority Major [ 3 ] Critical [ 2 ]
          Hide
          Todd Lipcon added a comment -

          What's the test/benchmark used for this result?

          Show
          Todd Lipcon added a comment - What's the test/benchmark used for this result?
          Hide
          Vinay Kumar Thota added a comment -

          Ran compression benchmark with following environments.

          Codec type : BLOCK
          Codec : com.hadoop.compression.lzo.LzoCodec

          Show
          Vinay Kumar Thota added a comment - Ran compression benchmark with following environments. Codec type : BLOCK Codec : com.hadoop.compression.lzo.LzoCodec
          Vinay Kumar Thota made changes -
          Summary Compression runtime is increased by 13% as well as throughput is decreased by 24% in 0.23 Compression benchmark run-time increased by 13% in 0.23
          Amol Kekre made changes -
          Priority Critical [ 2 ] Blocker [ 1 ]
          Arun C Murthy made changes -
          Assignee Vinod Kumar Vavilapalli [ vinodkv ]
          Hide
          Scott Carey added a comment -

          Is this also a sub-task of MAPREDUCE-3561?

          Show
          Scott Carey added a comment - Is this also a sub-task of MAPREDUCE-3561 ?
          Vinod Kumar Vavilapalli made changes -
          Parent MAPREDUCE-3561 [ 12535237 ]
          Issue Type Bug [ 1 ] Sub-task [ 7 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Is this also a sub-task of MAPREDUCE-3561?

          Yeah, I couldn't find this ticket before.

          But my guess is most of those sub-tasks are related and most likely will be fixed in one shot. Let's see.

          Show
          Vinod Kumar Vavilapalli added a comment - Is this also a sub-task of MAPREDUCE-3561 ? Yeah, I couldn't find this ticket before. But my guess is most of those sub-tasks are related and most likely will be fixed in one shot. Let's see.
          Hide
          Amol Kekre added a comment -

          We would post new compression numbers post MAPREDUCE-3641

          Show
          Amol Kekre added a comment - We would post new compression numbers post MAPREDUCE-3641
          Hide
          Arun C Murthy added a comment -

          This was mostly due to misconfiguration, but MAPREDUCE-3641 does help a lot.

          Show
          Arun C Murthy added a comment - This was mostly due to misconfiguration, but MAPREDUCE-3641 does help a lot.
          Arun C Murthy made changes -
          Link This issue is blocked by MAPREDUCE-3641 [ MAPREDUCE-3641 ]
          Hide
          Amol Kekre added a comment -

          Compression is now 10.4% faster in .23 (compared to .205/1.0). Please close this jira

          Show
          Amol Kekre added a comment - Compression is now 10.4% faster in .23 (compared to .205/1.0). Please close this jira
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Thanks for the update Amol. Thanks to Sid and Vinay who've been running this and verifying it.

          This is fixed now after the recent commits tracked at MAPREDUCE-3561.

          Show
          Vinod Kumar Vavilapalli added a comment - Thanks for the update Amol. Thanks to Sid and Vinay who've been running this and verifying it. This is fixed now after the recent commits tracked at MAPREDUCE-3561 .
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 0.23.1 [ 12318883 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          42d 7h 54m 1 Vinod Kumar Vavilapalli 23/Jan/12 21:46
          Resolved Resolved Closed Closed
          41d 5h 2m 1 Arun C Murthy 05/Mar/12 02:49

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinay Kumar Thota
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development