Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3427

streaming tests fail with MR2

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.1, 2.0.0-alpha
    • 0.23.1
    • contrib/streaming, mrv2
    • None

    Description

      After Mavenizing streaming and getting its testcases to use the MiniMRCluster wrapper (MAPREDUCE-3169), 4 testcases fail to pass.

      Following is an assessment of those failures. Note that the testcases have been tweaked only to set the streaming JAR and yarn as the framework.

      (If these issues are unrelated we should create sub-tasks for each one of them).

      TestStreamingCombiner, fails because returned counters don't match assertion. However, counters printed in the test output indicate values that would satisfy the assertion. As Tom has indicated it seems MR/YARN are not passing back counter information to the client API.

      TestStreamingBadRecords, the job is failing with the following exception

      Application application_1321575850006_0001 failed 1 times due to AM Container for 
      appattempt_1321575850006_0001_000001 exited with  exitCode: 127 due to: 
      .Failing this attempt.. Failing the application.
      

      Difficult to troubleshoot because there are not task logs from Mini MR/YARN run.

      TestStreamingStatus fails in validateTaskStatus() in the following assertion

      expected:<[before consuming input > sort]> but was:<[SUCCEEDED]>
      

      TestUlimit fails with

      org.junit.ComparisonFailure: output is wrong expected:<[786432]> but was:<[unlimited]>
      

      Attachments

        1. MR-3427.1.patch
          4 kB
          Hitesh Shah

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hitesh Hitesh Shah
            tucu00 Alejandro Abdelnur
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment