Hadoop Common
  1. Hadoop Common
  2. HADOOP-7671

Add license headers to hadoop-common/src/main/packages/templates/conf/

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      hadoop-common/src/main/packages/templates/conf/ not in the exclude list for apache-rat plugin . This causes 10 release audit warnings for missing license headers (in the properties and xml files like hdfs-site.xml)

      1. HADOOP-7671.patch
        12 kB
        Ravi Prakash
      2. HADOOP-7671.patch
        12 kB
        Ravi Prakash
      3. HADOOP-7671.patch
        0.6 kB
        Ravi Prakash

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #23 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/23/)
          HADOOP-7671 Add license headers to hadoop-common/src/main/packages/templates/conf/

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1174661
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #23 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/23/ ) HADOOP-7671 Add license headers to hadoop-common/src/main/packages/templates/conf/ stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1174661 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #18 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/18/)
          HADOOP-7671 Add license headers to hadoop-common/src/main/packages/templates/conf/

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1174661
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #18 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/18/ ) HADOOP-7671 Add license headers to hadoop-common/src/main/packages/templates/conf/ stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1174661 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg
          Hide
          Steve Loughran added a comment -

          Committed in 1174661. Thanks!

          Show
          Steve Loughran added a comment - Committed in 1174661. Thanks!
          Hide
          Steve Loughran added a comment -

          +1

          Here's the before and after output

          Before

          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 8.028s
          [INFO] Finished at: Fri Sep 23 12:06:59 BST 2011
          [INFO] Final Memory: 12M/81M
          [INFO] ------------------------------------------------------------------------
          [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check (default-cli) on project hadoop-common: Too many unapproved licenses: 10 -> [Help 1]
          [ERROR]
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR]
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
          [ERROR]
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR] mvn <goals> -rf :hadoop-common

          After

          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 7.852s
          [INFO] Finished at: Fri Sep 23 12:08:27 BST 2011
          [INFO] Final Memory: 13M/81M
          [INFO] ------------------------------------------------------------------------

          Show
          Steve Loughran added a comment - +1 Here's the before and after output Before [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 8.028s [INFO] Finished at: Fri Sep 23 12:06:59 BST 2011 [INFO] Final Memory: 12M/81M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check (default-cli) on project hadoop-common: Too many unapproved licenses: 10 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hadoop-common After [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.852s [INFO] Finished at: Fri Sep 23 12:08:27 BST 2011 [INFO] Final Memory: 13M/81M [INFO] ------------------------------------------------------------------------
          Hide
          Uma Maheswara Rao G added a comment -

          Hi Steve,

          In trunk Apache header missed only for log4j.properties.
          For Trunk fix https://issues.apache.org/jira/browse/HADOOP-7669.( Patch available)

          In 23 branch, Apache header missed for other files as well.
          Current patch in this issue should for 23 branch only.

          Thanks
          Uma

          Show
          Uma Maheswara Rao G added a comment - Hi Steve, In trunk Apache header missed only for log4j.properties. For Trunk fix https://issues.apache.org/jira/browse/HADOOP-7669.( Patch available) In 23 branch, Apache header missed for other files as well. Current patch in this issue should for 23 branch only. Thanks Uma
          Hide
          Steve Loughran added a comment -

          the patch doesn't apply to trunk. Is this a 0.23 branch only patch?

          Show
          Steve Loughran added a comment - the patch doesn't apply to trunk. Is this a 0.23 branch only patch?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496196/HADOOP-7671.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/225//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496196/HADOOP-7671.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/225//console This message is automatically generated.
          Hide
          Ravi Prakash added a comment -

          Thanks God your eyes be sharper than mine Suresh!
          I've removed those two lines.

          Show
          Ravi Prakash added a comment - Thanks God your eyes be sharper than mine Suresh! I've removed those two lines.
          Hide
          Suresh Srinivas added a comment -

          Ravi, is this necessary in taskcontroller.cfg?

          +# syntax: [prefix].[source|sink|jmx].[instance].[options]
          +# See package.html for org.apache.hadoop.metrics2 for details

          Show
          Suresh Srinivas added a comment - Ravi, is this necessary in taskcontroller.cfg? +# syntax: [prefix] . [source|sink|jmx] . [instance] . [options] +# See package.html for org.apache.hadoop.metrics2 for details
          Hide
          Ravi Prakash added a comment -

          @Tom: Ok! I've added the headers to all the files in this patch. Can you please commit it?

          Show
          Ravi Prakash added a comment - @Tom: Ok! I've added the headers to all the files in this patch. Can you please commit it?
          Hide
          Tom White added a comment -

          It would be better to add license headers to the configuration files.

          Show
          Tom White added a comment - It would be better to add license headers to the configuration files.
          Hide
          Ravi Prakash added a comment -

          The real problem was the conf files not being in exclude. Not the warnings I mentioned in my first comment.

          Show
          Ravi Prakash added a comment - The real problem was the conf files not being in exclude. Not the warnings I mentioned in my first comment.
          Hide
          Ravi Prakash added a comment -

          Looks like the conf files in src/main/packages/templates/conf/ were not being excluded from RAT. They should otherwise you get license warnings.

          Can someone please commit this?

          Show
          Ravi Prakash added a comment - Looks like the conf files in src/main/packages/templates/conf/ were not being excluded from RAT. They should otherwise you get license warnings. Can someone please commit this?
          Hide
          Ravi Prakash added a comment -

          When I run

          mvn apache-rat:check -DHadoopPatchProcess 2>&1

          I see the following error and I have no clue how to go on

          [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check (default-cli) on project hadoop-common: Too many unapproved licenses: 10 -> [Help 1]
          

          Now's as good a time as any for some help from maven/rat experts.

          Show
          Ravi Prakash added a comment - When I run mvn apache-rat:check -DHadoopPatchProcess 2>&1 I see the following error and I have no clue how to go on [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check (default-cli) on project hadoop-common: Too many unapproved licenses: 10 -> [Help 1] Now's as good a time as any for some help from maven/rat experts.
          Hide
          Ravi Prakash added a comment -

          My maven version:

          $ mvn -version
          Apache Maven 3.0.3 (r1075438; 2011-02-28 11:31:09-0600)
          Java version: 1.6.0_01, vendor: Sun Microsystems Inc.
          Java home: /usr/java/jdk1.6.0_01/jre
          Default locale: en_US, platform encoding: UTF-8
          OS name: "linux", version: "2.6.18-164.2.1.el5", arch: "i386", family: "unix"
          

          The warnings I get are:

          [WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:0.23.0-SNAPSHOT
          [WARNING] 'build.plugins.plugin.version' for org.apache.rat:apache-rat-plugin is missing. @ org.apache.hadoop:hadoop-yarn:${yarn.version}, <somepath>/hadoop-mapreduce-project/hadoop-yarn/pom.xml, line 389, column 15
          [WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:0.23.0-SNAPSHOT
          [WARNING] 'build.plugins.plugin.version' for org.apache.rat:apache-rat-plugin is missing. @ org.apache.hadoop:hadoop-mapreduce:${hadoop-mapreduce.version}, <somepath>/hadoop-mapreduce-project/pom.xml, line 353, column 16
          
          Show
          Ravi Prakash added a comment - My maven version: $ mvn -version Apache Maven 3.0.3 (r1075438; 2011-02-28 11:31:09-0600) Java version: 1.6.0_01, vendor: Sun Microsystems Inc. Java home: /usr/java/jdk1.6.0_01/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "2.6.18-164.2.1.el5", arch: "i386", family: "unix" The warnings I get are: [WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:0.23.0-SNAPSHOT [WARNING] 'build.plugins.plugin.version' for org.apache.rat:apache-rat-plugin is missing. @ org.apache.hadoop:hadoop-yarn:${yarn.version}, <somepath>/hadoop-mapreduce-project/hadoop-yarn/pom.xml, line 389, column 15 [WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:0.23.0-SNAPSHOT [WARNING] 'build.plugins.plugin.version' for org.apache.rat:apache-rat-plugin is missing. @ org.apache.hadoop:hadoop-mapreduce:${hadoop-mapreduce.version}, <somepath>/hadoop-mapreduce-project/pom.xml, line 353, column 16

            People

            • Assignee:
              Ravi Prakash
              Reporter:
              Ravi Prakash
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development