Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1511

98 Release Audit warnings on trunk and branch-0.22

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 0.23.0
    • Fix Version/s: 0.22.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There are 98 release audit warnings on trunk. See attached txt file. These must be fixed or filtered out to get back to a reasonably small number of warnings. The OK_RELEASEAUDIT_WARNINGS property in src/test/test-patch.properties should also be set appropriately in the patch that fixes this issue.

      1. releaseauditWarnings.txt
        18 kB
        Nigel Daley
      2. HDFS-1511.patch
        1 kB
        Konstantin Boudnik
      3. HDFS-1511.patch
        17 kB
        Jakob Homan
      4. HDFS-1511.patch
        17 kB
        Nigel Daley

        Activity

        Hide
        Owen O'Malley added a comment -

        Is there already a suppression list for RAT? Clearly all of the generated files should be exempted.

        Show
        Owen O'Malley added a comment - Is there already a suppression list for RAT? Clearly all of the generated files should be exempted.
        Hide
        Nigel Daley added a comment -

        Looks like the HDFS releaseaudit target doesn't filter out much:

        <fileset dir="$

        {dist.dir}

        ">
        <exclude name="CHANGES.txt"/>
        <exclude name="docs/"/>
        <exclude name="lib/jdiff/"/>
        </fileset>

        The thriftfs gen-* directories seem like obvious candidates to filter out. Any others?

        Show
        Nigel Daley added a comment - Looks like the HDFS releaseaudit target doesn't filter out much: <fileset dir="$ {dist.dir} "> <exclude name="CHANGES.txt"/> <exclude name="docs/"/> <exclude name="lib/jdiff/"/> </fileset> The thriftfs gen-* directories seem like obvious candidates to filter out. Any others?
        Hide
        Konstantin Boudnik added a comment -

        OpenOffice files (src/docs/src/documentation/resources/images/FI-framework.odg) should be going.
        cli/clitest_data
        src/test/smoke|commit|all-tests
        src/test/*xml
        webapps/**/*xml
        src/docs/releasenotes.html
        at the very least.

        Show
        Konstantin Boudnik added a comment - OpenOffice files (src/docs/src/documentation/resources/images/FI-framework.odg) should be going. cli/clitest_data src/test/smoke|commit|all-tests src/test/*xml webapps/**/*xml src/docs/releasenotes.html at the very least.
        Hide
        Konstantin Boudnik added a comment -

        First crap.

        Show
        Konstantin Boudnik added a comment - First crap.
        Hide
        Jakob Homan added a comment -

        Attached patch moves the count back to 0. Some files have licenses added, others are added to the ignore list via a liberal interpretation of http://www.apache.org/legal/src-headers.html

        Result of ant releaseaudit after patch:

        releaseaudit:
        [rat:report] 
        [rat:report] *****************************************************
        [rat:report] Summary
        [rat:report] -------
        [rat:report] Notes: 9
        [rat:report] Binaries: 22
        [rat:report] Archives: 47
        [rat:report] Standards: 610
        [rat:report] 
        [rat:report] Apache Licensed: 609
        [rat:report] Generated Documents: 1
        [rat:report] 
        [rat:report] JavaDocs are generated and so license header is optional
        [rat:report] Generated files do not required license headers
        [rat:report] 
        [rat:report] 0 Unknown Licenses
        [rat:report] 
        [rat:report] *******************************
        [rat:report] 
        [rat:report] Unapproved licenses:
        [rat:report] 
        [rat:report] 
        [rat:report] *******************************

        and

        [rat:report]  *****************************************************
        [rat:report]  Printing headers for files without AL header...
        [rat:report]  
        [rat:report]  
        
        BUILD SUCCESSFUL
        Total time: 1 minute 25 seconds

        On a side note, it's time to get rid of forrest. It was a serious pain to get up and running on my mac with OSX and JDK5 has been EOL'ed for several months, as well as the forrest project not having had a release in almost four years. I'll open a JIRA to do so, if one has not yet been.

        Show
        Jakob Homan added a comment - Attached patch moves the count back to 0. Some files have licenses added, others are added to the ignore list via a liberal interpretation of http://www.apache.org/legal/src-headers.html Result of ant releaseaudit after patch: releaseaudit: [rat:report] [rat:report] ***************************************************** [rat:report] Summary [rat:report] ------- [rat:report] Notes: 9 [rat:report] Binaries: 22 [rat:report] Archives: 47 [rat:report] Standards: 610 [rat:report] [rat:report] Apache Licensed: 609 [rat:report] Generated Documents: 1 [rat:report] [rat:report] JavaDocs are generated and so license header is optional [rat:report] Generated files do not required license headers [rat:report] [rat:report] 0 Unknown Licenses [rat:report] [rat:report] ******************************* [rat:report] [rat:report] Unapproved licenses: [rat:report] [rat:report] [rat:report] ******************************* and [rat:report] ***************************************************** [rat:report] Printing headers for files without AL header... [rat:report] [rat:report] BUILD SUCCESSFUL Total time: 1 minute 25 seconds On a side note, it's time to get rid of forrest. It was a serious pain to get up and running on my mac with OSX and JDK5 has been EOL'ed for several months, as well as the forrest project not having had a release in almost four years. I'll open a JIRA to do so, if one has not yet been.
        Hide
        Konstantin Boudnik added a comment -

        +1 patch looks good and works as expected.

        A couple of optional nits:

        • replace commit-tests, all-tests, and so on with a mask like this
          <exclude name="src/test/*-tests" />
        • replace specific resources folder location with something like
          {{<exclude name="*//resources/" /> }}
          This will allow not to update the exclude list everytime a new test list or resources folder is added into the source tree.
        • keeping the exclude list outside of the build.xml looked more appealing to me, but having it embedded into the build file is Ok too.
        Show
        Konstantin Boudnik added a comment - +1 patch looks good and works as expected. A couple of optional nits: replace commit-tests, all-tests, and so on with a mask like this <exclude name="src/test/*-tests" /> replace specific resources folder location with something like {{<exclude name="* / /resources/" /> }} This will allow not to update the exclude list everytime a new test list or resources folder is added into the source tree. keeping the exclude list outside of the build.xml looked more appealing to me, but having it embedded into the build file is Ok too.
        Hide
        Jakob Homan added a comment -

        I'm fine with those nits, if someone wants to update the patch or open a new JIRA, but I'd like to get this committed now and free Hudson. I'll commit this in the morning unless any committers have any objections.

        Show
        Jakob Homan added a comment - I'm fine with those nits, if someone wants to update the patch or open a new JIRA, but I'd like to get this committed now and free Hudson. I'll commit this in the morning unless any committers have any objections.
        Hide
        Nigel Daley added a comment -

        +1. Thanks Jakob.

        Show
        Nigel Daley added a comment - +1. Thanks Jakob.
        Hide
        Konstantin Boudnik added a comment -

        As I said, those are optional, so I don't have any issues with committing this as is.

        Show
        Konstantin Boudnik added a comment - As I said, those are optional, so I don't have any issues with committing this as is.
        Hide
        Jakob Homan added a comment -

        I've committed this to trunk. Resolving as fixed.

        Show
        Jakob Homan added a comment - I've committed this to trunk. Resolving as fixed.
        Hide
        Nigel Daley added a comment -

        Ran a test run of HDFS precommit testing. Hudson is still seeing these 4 issues after this patch:

        [rat:report] !????? trunk/build/hadoop-hdfs-1051334_HDFS-1534_PATCH-12465968/src/docs/src/documentation/resources/images/FI-framework.odg

        [rat:report] !????? trunk/build/hadoop-hdfs-1051334_HDFS-1534_PATCH-12465968/src/docs/src/documentation/resources/images/hdfsarchitecture.odg

        [rat:report] !????? trunk/build/hadoop-hdfs-1051334_HDFS-1534_PATCH-12465968/src/docs/src/documentation/resources/images/hdfsdatanodes.odg

        [rat:report] !????? trunk/build/hadoop-hdfs-1051334_HDFS-1534_PATCH-12465968/src/test/hdfs/org/apache/hadoop/hdfs/hadoop-14-dfs-dir.tgz

        Show
        Nigel Daley added a comment - Ran a test run of HDFS precommit testing. Hudson is still seeing these 4 issues after this patch: [rat:report] !????? trunk/build/hadoop-hdfs-1051334_ HDFS-1534 _PATCH-12465968/src/docs/src/documentation/resources/images/FI-framework.odg [rat:report] !????? trunk/build/hadoop-hdfs-1051334_ HDFS-1534 _PATCH-12465968/src/docs/src/documentation/resources/images/hdfsarchitecture.odg [rat:report] !????? trunk/build/hadoop-hdfs-1051334_ HDFS-1534 _PATCH-12465968/src/docs/src/documentation/resources/images/hdfsdatanodes.odg [rat:report] !????? trunk/build/hadoop-hdfs-1051334_ HDFS-1534 _PATCH-12465968/src/test/hdfs/org/apache/hadoop/hdfs/hadoop-14-dfs-dir.tgz
        Hide
        Jakob Homan added a comment -

        What is the ant target for the precommit release audit? For my patch I was running ant releaseaudit. They must be different. I didn't see any odg files.

        Show
        Jakob Homan added a comment - What is the ant target for the precommit release audit? For my patch I was running ant releaseaudit. They must be different. I didn't see any odg files.
        Hide
        Konstantin Boudnik added a comment -

        actually, you haven't include *.odg nor *.tgz into the exclude list

        Show
        Konstantin Boudnik added a comment - actually, you haven't include *.odg nor *.tgz into the exclude list
        Hide
        Jakob Homan added a comment -

        that's because those files weren't showing up in my release audit when I was running it, as above. I went through the results I was getting and fixed or bypassed each one until there were no more. I'm not sure why those files weren't being included in mine...

        Show
        Jakob Homan added a comment - that's because those files weren't showing up in my release audit when I was running it, as above. I went through the results I was getting and fixed or bypassed each one until there were no more. I'm not sure why those files weren't being included in mine...
        Hide
        Konstantin Boudnik added a comment -

        But they are in the initial text file Nigel attached. And in my initial patch as well

        Show
        Konstantin Boudnik added a comment - But they are in the initial text file Nigel attached. And in my initial patch as well
        Hide
        Nigel Daley added a comment -

        I suspect it's because releaseaudit is run in a workspace that has already had tar and doc (forrest) run in it. Like Cos said, I think we just need to add *.odg and *.tgz to the exclude list and we should be golden.

        Show
        Nigel Daley added a comment - I suspect it's because releaseaudit is run in a workspace that has already had tar and doc (forrest) run in it. Like Cos said, I think we just need to add *.odg and *.tgz to the exclude list and we should be golden.
        Hide
        Nigel Daley added a comment -

        I just committed the extra 3 lines to get Hudson patch process to work. Attaching the new complete patch.

        Show
        Nigel Daley added a comment - I just committed the extra 3 lines to get Hudson patch process to work. Attaching the new complete patch.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/ )

          People

          • Assignee:
            Jakob Homan
            Reporter:
            Nigel Daley
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development