Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4798

TestJobHistoryServer fails some times with 'java.lang.AssertionError: Address already in use'

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.3
    • Fix Version/s: 1.1.2
    • Component/s: jobhistoryserver, test
    • Labels:
    • Environment:

      Red Hat Ent Server 6.2

      Description

      UT Failure in IHC 1.0.3: org.apache.hadoop.mapred.TestJobHistoryServer. This UT fails sometimes.

      The error message is:
      'Testcase: testHistoryServerStandalone took 5.376 sec
      Caused an ERROR
      Address already in use
      java.lang.AssertionError: Address already in use
      at org.apache.hadoop.mapred.TestJobHistoryServer.testHistoryServerStandalone(TestJobHistoryServer.java:113)'

        Issue Links

          Activity

          Hide
          Matt Foley added a comment -

          Closed upon successful release of 1.1.2.

          Show
          Matt Foley added a comment - Closed upon successful release of 1.1.2.
          Hide
          sam liu added a comment -

          Thanks Eric! Pls let me know if there is any further action

          Show
          sam liu added a comment - Thanks Eric! Pls let me know if there is any further action
          Hide
          Eric Yang added a comment -

          Thanks Sam.

          Show
          Eric Yang added a comment - Thanks Sam.
          Hide
          Eric Yang added a comment -

          I just committed this to 1.x branches. JobHistoryServer doesn't exist in trunk, hence no action there.

          Show
          Eric Yang added a comment - I just committed this to 1.x branches. JobHistoryServer doesn't exist in trunk, hence no action there.
          Hide
          Eric Yang added a comment -

          +1 looks good.

          Show
          Eric Yang added a comment - +1 looks good.
          Hide
          sam liu added a comment -

          Eric,
          I updated the patch for 1.0.3 and uploaded the patch for branch-1.
          For trunk, there is no TestJobHistoryServer.java, so I did not generate another patch for trunk.
          Thanks!

          Show
          sam liu added a comment - Eric, I updated the patch for 1.0.3 and uploaded the patch for branch-1. For trunk, there is no TestJobHistoryServer.java, so I did not generate another patch for trunk. Thanks!
          Hide
          sam liu added a comment -

          Eric, I updated the patch. Thanks!

          Show
          sam liu added a comment - Eric, I updated the patch. Thanks!
          Hide
          Eric Yang added a comment -

          The indentation still has problem. In addition, could you generate a patch for branch-1, and trunk? Thanks

          Show
          Eric Yang added a comment - The indentation still has problem. In addition, could you generate a patch for branch-1, and trunk? Thanks
          Hide
          sam liu added a comment -

          Hi Eric, according to your suggestions, I updated the patch:

          • Indent the code properly
          • Set 4 retries of checking the URL redirect method, and add synthetic logic to pause 1, 2, 3, and 4 seconds upon each retry
          Show
          sam liu added a comment - Hi Eric, according to your suggestions, I updated the patch: Indent the code properly Set 4 retries of checking the URL redirect method, and add synthetic logic to pause 1, 2, 3, and 4 seconds upon each retry
          Hide
          Eric Yang added a comment -

          Could we indent the if block properly? In addition, could we do more to check the URL redirect method in a while loop with a maximum of 4 retries? We can add synthetic logic to pause 1, 2, 3, and 4 seconds upon each retry. Hence, the worst case would be delayed by 10 seconds, but it can recover earlier than 10 seconds in most cases. This could reduce the amount of time required to execute this test case.

          Show
          Eric Yang added a comment - Could we indent the if block properly? In addition, could we do more to check the URL redirect method in a while loop with a maximum of 4 retries? We can add synthetic logic to pause 1, 2, 3, and 4 seconds upon each retry. Hence, the worst case would be delayed by 10 seconds, but it can recover earlier than 10 seconds in most cases. This could reduce the amount of time required to execute this test case.
          Hide
          sam liu added a comment -

          Just attached the patch now, but need the review of committer

          Show
          sam liu added a comment - Just attached the patch now, but need the review of committer
          Hide
          sam liu added a comment -

          The failures always happen in the step 'String redirectUrl = getRedirectUrl(job.getTrackingURL())' of testHistoryServerStandalone(). In the failure case, when executing into the method getRedirectUrl(job.getTrackingURL()), it always fails to execute 'Assert.assertEquals(status, HttpURLConnection.HTTP_MOVED_TEMP)' because the status value now is 200, but the HttpURLConnection.HTTP_MOVED_TEMP value is 302. In the successful cases., the status value will be 302.
          If add a break(sleep 10 sec) between 'int status = client.executeMethod(method)' and 'method.setFollowRedirects(false)', this case will be stable. I tried: 10 executions with 0 failure.

          Show
          sam liu added a comment - The failures always happen in the step 'String redirectUrl = getRedirectUrl(job.getTrackingURL())' of testHistoryServerStandalone(). In the failure case, when executing into the method getRedirectUrl(job.getTrackingURL()), it always fails to execute 'Assert.assertEquals(status, HttpURLConnection.HTTP_MOVED_TEMP)' because the status value now is 200, but the HttpURLConnection.HTTP_MOVED_TEMP value is 302. In the successful cases., the status value will be 302. If add a break(sleep 10 sec) between 'int status = client.executeMethod(method)' and 'method.setFollowRedirects(false)', this case will be stable. I tried: 10 executions with 0 failure.
          Hide
          sam liu added a comment -

          Current issue is different from the existing TestJobHistoryServer related JIRAs, because it has a special failure reason: 'java.lang.AssertionError: Address already in use'. BTW, in my testing, this unit test has 1 failure in 6 executions.

          Show
          sam liu added a comment - Current issue is different from the existing TestJobHistoryServer related JIRAs, because it has a special failure reason: 'java.lang.AssertionError: Address already in use'. BTW, in my testing, this unit test has 1 failure in 6 executions.

            People

            • Assignee:
              sam liu
              Reporter:
              sam liu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 3h
                3h
                Remaining:
                Remaining Estimate - 3h
                3h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development