Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Not A Problem
    • Affects Version/s: 0.22.0
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None
    • Environment:

      Physical cluster based on 0.22-SNAPSHOT

      Description

      An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via -jt cmd. arg. then everything is working as expected.

        Activity

        Hide
        Konstantin Boudnik added a comment -

        Steps to reproduce:

        • setup cluster from these bits
        • apply simple configs
        • start cluster
        • run TestDFSIO:
          $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-mapred-test-0.22.0-SNAPSHOT.jar TestDFSIO -write -nrFiles 100 -fileSize 10MB
          
        Show
        Konstantin Boudnik added a comment - Steps to reproduce: setup cluster from these bits apply simple configs start cluster run TestDFSIO: $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-mapred-test-0.22.0-SNAPSHOT.jar TestDFSIO -write -nrFiles 100 -fileSize 10MB
        Hide
        Konstantin Boudnik added a comment -

        You'll see the following error message

        11/05/31 01:13:57 INFO util.NativeCodeLoader: Loaded the native-hadoop library
        11/05/31 01:13:57 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/cos1445850318/.staging/job_local_0001
        ENOENT: No such file or directory
        

        If the same test is executed with -jt jobtracker:port command line argument everything works fine.

        It would seem like a configuration problem unless this is the only test/example with doesn't work properly. Sleep jobs, Pi, randomwriter jobs are working just fine.

        Show
        Konstantin Boudnik added a comment - You'll see the following error message 11/05/31 01:13:57 INFO util.NativeCodeLoader: Loaded the native-hadoop library 11/05/31 01:13:57 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/cos1445850318/.staging/job_local_0001 ENOENT: No such file or directory If the same test is executed with -jt jobtracker:port command line argument everything works fine. It would seem like a configuration problem unless this is the only test/example with doesn't work properly. Sleep jobs, Pi, randomwriter jobs are working just fine.
        Hide
        Konstantin Boudnik added a comment -

        Turns out not a problem of the test. Rather the configuration was using an old way of configuring jobtracker address (mapred.job.tracker). Is this an intended behavior to simply ignore the old configuration parameter rather than issue a deprecation warning?

        Show
        Konstantin Boudnik added a comment - Turns out not a problem of the test. Rather the configuration was using an old way of configuring jobtracker address (mapred.job.tracker). Is this an intended behavior to simply ignore the old configuration parameter rather than issue a deprecation warning?
        Hide
        Todd Lipcon added a comment -

        The deprecated config ought to work. When was this particular build from? Did it include the rather-recent fix for HADOOP-7287 (committed May 23)? If so and this still didn't work, it seems like a bug to me.

        Show
        Todd Lipcon added a comment - The deprecated config ought to work. When was this particular build from? Did it include the rather-recent fix for HADOOP-7287 (committed May 23)? If so and this still didn't work, it seems like a bug to me.
        Hide
        Konstantin Boudnik added a comment -

        I don't remember exact build numbers but they all were happening after 5/23 for sure.

        Show
        Konstantin Boudnik added a comment - I don't remember exact build numbers but they all were happening after 5/23 for sure.

          People

          • Assignee:
            Konstantin Boudnik
            Reporter:
            Konstantin Boudnik
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development