Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9470

eliminate duplicate FQN tests in different Hadoop modules

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0, 3.0.0-alpha1
    • Fix Version/s: 2.3.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In different modules of Hadoop project there are tests with identical FQNs (fully qualified name).
      For example, test with FQN org.apache.hadoop.util.TestRunJar is contained in 2 modules:
      ./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestRunJar.java
      ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/util/TestRunJar.java

      Such situation causes certain problems with test result reporting and other code analysis tools (such as Clover, e.g.) because almost all the tools identify the tests by their Java FQN.

      So, I suggest to rename all such test classes to avoid duplicate FQNs in different modules. I'm attaching simple shell script that can find all such problematic test classes. Currently Hadoop trunk has 9 such test classes, they are:
      $ ~/bin/find-duplicate-fqns.sh

      1. Module [./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-classes] has 7 duplicate FQN tests:
        org.apache.hadoop.ipc.TestSocketFactory
        org.apache.hadoop.mapred.TestFileOutputCommitter
        org.apache.hadoop.mapred.TestJobClient
        org.apache.hadoop.mapred.TestJobConf
        org.apache.hadoop.mapreduce.lib.output.TestFileOutputCommitter
        org.apache.hadoop.util.TestReflectionUtils
        org.apache.hadoop.util.TestRunJar
      2. Module [./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/test-classes] has 2 duplicate FQN tests:
        org.apache.hadoop.yarn.TestRecordFactory
        org.apache.hadoop.yarn.TestRPCFactories

        Attachments

        1. HADOOP-9470-trunk--N1.patch
          153 kB
          Ivan A. Veselovsky
        2. HADOOP-9470-branch-2--N1.patch
          147 kB
          Ivan A. Veselovsky
        3. HADOOP-9470-trunk.patch
          90 kB
          Ivan A. Veselovsky
        4. HADOOP-9470-branch-0.23.patch
          89 kB
          Ivan A. Veselovsky
        5. find-duplicate-fqns.sh
          1 kB
          Ivan A. Veselovsky

          Activity

            People

            • Assignee:
              iveselovsky Ivan A. Veselovsky
              Reporter:
              iveselovsky Ivan A. Veselovsky
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: