Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9470

eliminate duplicate FQN tests in different Hadoop modules

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.3.0, 3.0.0-alpha1
    • 2.3.0
    • test
    • None
    • Reviewed

    Description

      In different modules of Hadoop project there are tests with identical FQNs (fully qualified name).
      For example, test with FQN org.apache.hadoop.util.TestRunJar is contained in 2 modules:
      ./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestRunJar.java
      ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/util/TestRunJar.java

      Such situation causes certain problems with test result reporting and other code analysis tools (such as Clover, e.g.) because almost all the tools identify the tests by their Java FQN.

      So, I suggest to rename all such test classes to avoid duplicate FQNs in different modules. I'm attaching simple shell script that can find all such problematic test classes. Currently Hadoop trunk has 9 such test classes, they are:
      $ ~/bin/find-duplicate-fqns.sh

      1. Module [./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-classes] has 7 duplicate FQN tests:
        org.apache.hadoop.ipc.TestSocketFactory
        org.apache.hadoop.mapred.TestFileOutputCommitter
        org.apache.hadoop.mapred.TestJobClient
        org.apache.hadoop.mapred.TestJobConf
        org.apache.hadoop.mapreduce.lib.output.TestFileOutputCommitter
        org.apache.hadoop.util.TestReflectionUtils
        org.apache.hadoop.util.TestRunJar
      2. Module [./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/test-classes] has 2 duplicate FQN tests:
        org.apache.hadoop.yarn.TestRecordFactory
        org.apache.hadoop.yarn.TestRPCFactories

      Attachments

        1. HADOOP-9470-trunk--N1.patch
          153 kB
          Ivan A. Veselovsky
        2. HADOOP-9470-trunk.patch
          90 kB
          Ivan A. Veselovsky
        3. HADOOP-9470-branch-2--N1.patch
          147 kB
          Ivan A. Veselovsky
        4. HADOOP-9470-branch-0.23.patch
          89 kB
          Ivan A. Veselovsky
        5. find-duplicate-fqns.sh
          1 kB
          Ivan A. Veselovsky

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            iveselovsky Ivan A. Veselovsky
            iveselovsky Ivan A. Veselovsky
            Votes:
            0 Vote for this issue
            Watchers:
            4 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment