Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7255

Performance regression bug caused by locking code

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.20.203.0
    • None
    • None
    • None

    Description

      While testing the 0.20.203 branch we have discovered quite a severe performance regression caused by the following checkin:

      http://svn.apache.org/viewvc?view=revision&revision=1079260

      This locking seems to cause a pause of a few seconds between tasks, example with DFSIOTest:

      CURRENT CODE

      11/05/02 21:16:28 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
      11/05/02 21:16:28 INFO fs.TestDFSIO:            Date & time: Mon May 02 21:16:28 EDT 2011
      11/05/02 21:16:28 INFO fs.TestDFSIO:        Number of files: 5
      11/05/02 21:16:28 INFO fs.TestDFSIO: Total MBytes processed: 500
      11/05/02 21:16:28 INFO fs.TestDFSIO:      Throughput mb/sec: 303.3980582524272
      11/05/02 21:16:28 INFO fs.TestDFSIO: Average IO rate mb/sec: 327.69122314453125
      11/05/02 21:16:28 INFO fs.TestDFSIO:  IO rate std deviation: 78.95767116969083
      11/05/02 21:16:28 INFO fs.TestDFSIO:     Test exec time sec: 19.476
      11/05/02 21:16:28 INFO fs.TestDFSIO: 
      

      WITH PATCH REMOVED

      11/05/02 21:19:03 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
      11/05/02 21:19:03 INFO fs.TestDFSIO:            Date & time: Mon May 02 21:19:03 EDT 2011
      11/05/02 21:19:03 INFO fs.TestDFSIO:        Number of files: 5
      11/05/02 21:19:03 INFO fs.TestDFSIO: Total MBytes processed: 500
      11/05/02 21:19:03 INFO fs.TestDFSIO:      Throughput mb/sec: 366.03221083455344
      11/05/02 21:19:03 INFO fs.TestDFSIO: Average IO rate mb/sec: 366.35528564453125
      11/05/02 21:19:03 INFO fs.TestDFSIO:  IO rate std deviation: 11.020931463080379
      11/05/02 21:19:03 INFO fs.TestDFSIO:     Test exec time sec: 3.543
      11/05/02 21:19:03 INFO fs.TestDFSIO: 
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            tjake T Jake Luciani

            Dates

              Created:
              Updated:

              Issue deployment