Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      JobTracker-side of MAPREDUCE-6794.
      There are many unused properties in JTConfig.java. Let's remove them.

      1. mapreduce6796.001.patch
        9 kB
        Haibo Chen
      2. mapreduce6796.002.patch
        47 kB
        Haibo Chen
      3. mapreduce6796.002.patch
        47 kB
        Haibo Chen

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 46s trunk passed
          +1 compile 6m 55s trunk passed
          +1 checkstyle 1m 33s trunk passed
          +1 mvnsite 2m 9s trunk passed
          +1 mvneclipse 0m 57s trunk passed
          +1 findbugs 3m 1s trunk passed
          +1 javadoc 1m 29s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 31s the patch passed
          +1 compile 6m 52s the patch passed
          +1 javac 6m 52s the patch passed
          -1 checkstyle 1m 33s root: The patch generated 6 new + 496 unchanged - 27 fixed = 502 total (was 523)
          +1 mvnsite 2m 8s the patch passed
          +1 mvneclipse 0m 56s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 31s the patch passed
          +1 javadoc 1m 29s the patch passed
          +1 unit 7m 12s hadoop-common in the patch passed.
          +1 unit 2m 43s hadoop-mapreduce-client-core in the patch passed.
          +1 unit 3m 7s hadoop-mapreduce-client-hs in the patch passed.
          +1 unit 0m 54s hadoop-sls in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          56m 56s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836185/mapreduce6796.001.patch
          JIRA Issue MAPREDUCE-6796
          Optional Tests asflicense mvnsite compile javac javadoc mvninstall unit findbugs checkstyle xml
          uname Linux 184d47136398 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f646fe3
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6791/artifact/patchprocess/diff-checkstyle-root.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6791/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6791/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 6m 55s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 2m 9s trunk passed +1 mvneclipse 0m 57s trunk passed +1 findbugs 3m 1s trunk passed +1 javadoc 1m 29s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 31s the patch passed +1 compile 6m 52s the patch passed +1 javac 6m 52s the patch passed -1 checkstyle 1m 33s root: The patch generated 6 new + 496 unchanged - 27 fixed = 502 total (was 523) +1 mvnsite 2m 8s the patch passed +1 mvneclipse 0m 56s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 31s the patch passed +1 javadoc 1m 29s the patch passed +1 unit 7m 12s hadoop-common in the patch passed. +1 unit 2m 43s hadoop-mapreduce-client-core in the patch passed. +1 unit 3m 7s hadoop-mapreduce-client-hs in the patch passed. +1 unit 0m 54s hadoop-sls in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 56m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836185/mapreduce6796.001.patch JIRA Issue MAPREDUCE-6796 Optional Tests asflicense mvnsite compile javac javadoc mvninstall unit findbugs checkstyle xml uname Linux 184d47136398 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f646fe3 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6791/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6791/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6791/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          haibochen Haibo Chen added a comment -

          Checkstyle issues are unrelated.

          Show
          haibochen Haibo Chen added a comment - Checkstyle issues are unrelated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Looks good to me. I'm thinking we can remove the following properties

          • mapreduce.jobtracker.http.address
          • mapreduce.jobtracker.handler.count
          • mapreduce.jobtracker.restart.recover
          • mapreduce.jobtracker.taskscheduler
          • mapreduce.jobtracker.instrumentation
          • mapreduce.jobtracker.maxtasks.perjob
          • mapreduce.jobtracker.heartbeats.in.second
          • mapreduce.jobtracker.persist.jobstatus.hours
          • mapreduce.jobtracker.persist.jobstatus.dir
          • mapreduce.jobtracker.retirejobs
          • mapreduce.jobtracker.retiredjobs.cache.size
          • mapreduce.jobtracker.taskscheduler.taskalloc.capacitypad
          • mapreduce.jobtracker.jobinit.threads
          • mapreduce.jobtracker.expire.trackers.interval
          • mapreduce.jobtracker.taskscheduler.maxrunningtasks.perjob
          • mapreduce.jobtracker.hosts.filename
          • mapreduce.jobtracker.hosts.exclude.filename
          • mapreduce.jobtracker.jobhistory.lru.cache.size
          • mapreduce.jobtracker.jobhistory.block.size
          • mapreduce.jobtracker.jobhistory.completed.location
          • mapreduce.jobtracker.jobhistory.location
          • mapreduce.jobtracker.blacklist.average.threshold
          • mapreduce.jobtracker.tasktracker.maxblacklists
          • mapreduce.jobtracker.split.metainfo.maxsize
          • mapreduce.jobtracker.webinterface.trusted

          because these parameters are used only in ConfigUtil.java. What do you think?

          Show
          ajisakaa Akira Ajisaka added a comment - Looks good to me. I'm thinking we can remove the following properties mapreduce.jobtracker.http.address mapreduce.jobtracker.handler.count mapreduce.jobtracker.restart.recover mapreduce.jobtracker.taskscheduler mapreduce.jobtracker.instrumentation mapreduce.jobtracker.maxtasks.perjob mapreduce.jobtracker.heartbeats.in.second mapreduce.jobtracker.persist.jobstatus.hours mapreduce.jobtracker.persist.jobstatus.dir mapreduce.jobtracker.retirejobs mapreduce.jobtracker.retiredjobs.cache.size mapreduce.jobtracker.taskscheduler.taskalloc.capacitypad mapreduce.jobtracker.jobinit.threads mapreduce.jobtracker.expire.trackers.interval mapreduce.jobtracker.taskscheduler.maxrunningtasks.perjob mapreduce.jobtracker.hosts.filename mapreduce.jobtracker.hosts.exclude.filename mapreduce.jobtracker.jobhistory.lru.cache.size mapreduce.jobtracker.jobhistory.block.size mapreduce.jobtracker.jobhistory.completed.location mapreduce.jobtracker.jobhistory.location mapreduce.jobtracker.blacklist.average.threshold mapreduce.jobtracker.tasktracker.maxblacklists mapreduce.jobtracker.split.metainfo.maxsize mapreduce.jobtracker.webinterface.trusted because these parameters are used only in ConfigUtil.java. What do you think?
          Hide
          haibochen Haibo Chen added a comment -

          Agreed. I'll update the patch to accommodate the change.

          Show
          haibochen Haibo Chen added a comment - Agreed. I'll update the patch to accommodate the change.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 17s Maven dependency ordering for branch
          +1 mvninstall 8m 15s trunk passed
          +1 compile 7m 6s trunk passed
          +1 checkstyle 1m 33s trunk passed
          +1 mvnsite 2m 10s trunk passed
          +1 mvneclipse 0m 56s trunk passed
          +1 findbugs 3m 2s trunk passed
          +1 javadoc 1m 29s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 34s the patch passed
          +1 compile 6m 54s the patch passed
          +1 javac 6m 54s the patch passed
          -1 checkstyle 1m 32s root: The patch generated 5 new + 401 unchanged - 122 fixed = 406 total (was 523)
          +1 mvnsite 2m 8s the patch passed
          +1 mvneclipse 0m 56s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 34s the patch passed
          +1 javadoc 1m 29s the patch passed
          -1 unit 7m 19s hadoop-common in the patch failed.
          +1 unit 2m 45s hadoop-mapreduce-client-core in the patch passed.
          +1 unit 3m 4s hadoop-mapreduce-client-hs in the patch passed.
          +1 unit 0m 54s hadoop-sls in the patch passed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          58m 55s



          Reason Tests
          Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836652/mapreduce6796.002.patch
          JIRA Issue MAPREDUCE-6796
          Optional Tests asflicense mvnsite compile javac javadoc mvninstall unit findbugs checkstyle xml
          uname Linux e74a1e8010c6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3779190
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 8m 15s trunk passed +1 compile 7m 6s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 2m 10s trunk passed +1 mvneclipse 0m 56s trunk passed +1 findbugs 3m 2s trunk passed +1 javadoc 1m 29s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 34s the patch passed +1 compile 6m 54s the patch passed +1 javac 6m 54s the patch passed -1 checkstyle 1m 32s root: The patch generated 5 new + 401 unchanged - 122 fixed = 406 total (was 523) +1 mvnsite 2m 8s the patch passed +1 mvneclipse 0m 56s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 34s the patch passed +1 javadoc 1m 29s the patch passed -1 unit 7m 19s hadoop-common in the patch failed. +1 unit 2m 45s hadoop-mapreduce-client-core in the patch passed. +1 unit 3m 4s hadoop-mapreduce-client-hs in the patch passed. +1 unit 0m 54s hadoop-sls in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 58m 55s Reason Tests Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836652/mapreduce6796.002.patch JIRA Issue MAPREDUCE-6796 Optional Tests asflicense mvnsite compile javac javadoc mvninstall unit findbugs checkstyle xml uname Linux e74a1e8010c6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3779190 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6794/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, the test failure looks unrelated to the patch.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, the test failure looks unrelated to the patch.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk. Thanks Haibo Chen for the contribution!

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk. Thanks Haibo Chen for the contribution!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10773 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10773/)
          MAPREDUCE-6796. Remove unused properties from JTConfig.java. Contributed (aajisaka: rev de0132748da4d5cd4ef65571987b2581ff456084)

          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
          • (edit) hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/DeprecatedProperties.md
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/server/jobtracker/JTConfig.java
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10773 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10773/ ) MAPREDUCE-6796 . Remove unused properties from JTConfig.java. Contributed (aajisaka: rev de0132748da4d5cd4ef65571987b2581ff456084) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java (edit) hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json (edit) hadoop-common-project/hadoop-common/src/site/markdown/DeprecatedProperties.md (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/server/jobtracker/JTConfig.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
          Hide
          haibochen Haibo Chen added a comment -

          Thanks Akira Ajisaka for your reviews!

          Show
          haibochen Haibo Chen added a comment - Thanks Akira Ajisaka for your reviews!

            People

            • Assignee:
              haibochen Haibo Chen
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development