Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11813

releasedocmaker.py should use today's date instead of unreleased

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: build
    • Labels:
    • Release Note:
      Use today instead of 'Unreleased' in releasedocmaker.py when --usetoday is given as an option.

      Description

      After discussing with a few folks, it'd be more convenient if releasedocmaker used the current date rather than unreleased when processing a version that JIRA hasn't declared released.

      1. HADOOP-11813.patch
        2 kB
        Darrell Taylor
      2. HADOOP-11813.001.patch
        2 kB
        Darrell Taylor

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2136/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • hadoop-common-project/hadoop-common/pom.xml
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • dev-support/releasedocmaker.py
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2136/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-common/CHANGES.txt dev-support/releasedocmaker.py
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/187/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • hadoop-common-project/hadoop-common/pom.xml
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • dev-support/releasedocmaker.py
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/187/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-common/CHANGES.txt dev-support/releasedocmaker.py
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/177/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • dev-support/releasedocmaker.py
        • hadoop-common-project/hadoop-common/pom.xml
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/177/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) dev-support/releasedocmaker.py hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2118 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2118/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • dev-support/releasedocmaker.py
        • hadoop-common-project/hadoop-common/pom.xml
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2118 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2118/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) dev-support/releasedocmaker.py hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #920 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/920/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • hadoop-common-project/hadoop-common/pom.xml
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • dev-support/releasedocmaker.py
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #920 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/920/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-common/CHANGES.txt dev-support/releasedocmaker.py
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/187/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • dev-support/releasedocmaker.py
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/pom.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/187/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) dev-support/releasedocmaker.py hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/pom.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7747 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7747/)
        HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743)

        • dev-support/releasedocmaker.py
        • hadoop-common-project/hadoop-common/pom.xml
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7747 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7747/ ) HADOOP-11813 . releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw) (aw: rev f325522c1423f89dced999a16d49a004b2879743) dev-support/releasedocmaker.py hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        aw Allen Wittenauer added a comment -

        +1 committing to trunk.

        Thanks!!

        Show
        aw Allen Wittenauer added a comment - +1 committing to trunk. Thanks!!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 33s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 28s There were no new javac warning messages.
        +1 javadoc 9m 37s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 common tests 22m 11s Tests passed in hadoop-common.
            56m 23s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12729231/HADOOP-11813.001.patch
        Optional Tests javadoc javac unit
        git revision trunk / f5b3847
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6226/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6226/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6226/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 28s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 common tests 22m 11s Tests passed in hadoop-common.     56m 23s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12729231/HADOOP-11813.001.patch Optional Tests javadoc javac unit git revision trunk / f5b3847 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6226/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6226/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6226/console This message was automatically generated.
        Hide
        d4rr3ll Darrell Taylor added a comment -

        New patch submitted, this simply moves the strreplace into the notableclean function. Tested with pandoc and the maven build, both produce valid html and the bullets still work in the releasenotes.

        Show
        d4rr3ll Darrell Taylor added a comment - New patch submitted, this simply moves the strreplace into the notableclean function. Tested with pandoc and the maven build, both produce valid html and the bullets still work in the releasenotes.
        Hide
        aw Allen Wittenauer added a comment -

        Yeah, the escape of the asterisk completely broke the releasenotes that use asterisks as bullet points.

        Show
        aw Allen Wittenauer added a comment - Yeah, the escape of the asterisk completely broke the releasenotes that use asterisks as bullet points.
        Hide
        aw Allen Wittenauer added a comment -

        I use:

        mvn clean site site:stage -Preleasedocs -DstagingDirectory=/tmp/hadoop-site
        

        Just so you don't have to hunt for it, output will be at: file:///tmp/hadoop-site/hadoop-project/hadoop-project-dist/hadoop-common/release/ with that command line.

        If pandoc is breaking, though, we should probably escape it. Maven's markdown to html conversion is... err, lenient, in some instances.

        Show
        aw Allen Wittenauer added a comment - I use: mvn clean site site:stage -Preleasedocs -DstagingDirectory=/tmp/hadoop-site Just so you don't have to hunt for it, output will be at: file:///tmp/hadoop-site/hadoop-project/hadoop-project-dist/hadoop-common/release/ with that command line. If pandoc is breaking, though, we should probably escape it. Maven's markdown to html conversion is... err, lenient, in some instances.
        Hide
        d4rr3ll Darrell Taylor added a comment -

        Allen - would you be able to provide me with the magical maven runes that I need to run to actually generate the HTML without having to do an entire build each time. Myself and maven have not reached an understanding yet :/

        Show
        d4rr3ll Darrell Taylor added a comment - Allen - would you be able to provide me with the magical maven runes that I need to run to actually generate the HTML without having to do an entire build each time. Myself and maven have not reached an understanding yet :/
        Hide
        d4rr3ll Darrell Taylor added a comment -

        Looking at the code now I should have probably read the comments around tableclean() and notableclean(), the are pretty clear.

        I'm going to do a bit more investigation because the asterisk that broke it for me was in the middle of a cell and not a bullet-point. Also I initially compiled the markdown using pandoc, but I'll double check if the asterisk also breaks whatever maven uses to compile the markdown.

        The jira that broke it (for pandoc) is MAPREDUCE-5785.

        | [MAPREDUCE-5785](https://issues.apache.org/jira/browse/MAPREDUCE-5785) | Derive heap size or mapreduce.*.memory.mb automatically |  Major | mr-am, task | Gera Shegalov | Gera Shegalov |
        

        I'll update with my findings later

        Show
        d4rr3ll Darrell Taylor added a comment - Looking at the code now I should have probably read the comments around tableclean() and notableclean(), the are pretty clear. I'm going to do a bit more investigation because the asterisk that broke it for me was in the middle of a cell and not a bullet-point. Also I initially compiled the markdown using pandoc, but I'll double check if the asterisk also breaks whatever maven uses to compile the markdown. The jira that broke it (for pandoc) is MAPREDUCE-5785 . | [MAPREDUCE-5785](https: //issues.apache.org/jira/browse/MAPREDUCE-5785) | Derive heap size or mapreduce.*.memory.mb automatically | Major | mr-am, task | Gera Shegalov | Gera Shegalov | I'll update with my findings later
        Hide
        aw Allen Wittenauer added a comment -

        There were no tests around this originally, do some need to be written?

        There's no tests for any of the non-Java, unfortunately.

        Small fix to escape out asterisks, these were breaking the tables

        Happen to have the JIRA that caused the issue? I suspect the asterisk escape needs to go into notableclean (==disallow table characters, because the code is building a table) rather than tableclean (==allow table characters, because user-built tables are allowed here). We definitely want asterisks in release notes so we can bullet-point things.

        I named these functions poorly. lol.

        Show
        aw Allen Wittenauer added a comment - There were no tests around this originally, do some need to be written? There's no tests for any of the non-Java, unfortunately. Small fix to escape out asterisks, these were breaking the tables Happen to have the JIRA that caused the issue? I suspect the asterisk escape needs to go into notableclean (==disallow table characters, because the code is building a table) rather than tableclean (==allow table characters, because user-built tables are allowed here). We definitely want asterisks in release notes so we can bullet-point things. I named these functions poorly. lol.
        Hide
        d4rr3ll Darrell Taylor added a comment -

        There were no tests around this originally, do some need to be written?

        Manual steps taken to test this were :

        cd hadoop-common-project/hadoop-common/src/site/markdown/release/
        python ../../../../../../dev-support/releasedocmaker.py --version 3.0.0 --index
        

        Then confirmed that files were produced in hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-SNAPSHOT/. The CHANGES.3.0.0-SNAPSHOT.md file contained the work 'Unreleased' in this case. The same command was run again with the --usetoday switch and the CHANGES.3.0.0-SNAPSHOT.md included today's date instead of 'Unreleased'.

        This was also checked through the maven build by running

        mvn pre-site -DskipTests -Preleasedocs
        

        The CHANGES.3.0.0-SNAPSHOT.md file included today's date instead of 'Unlreleased'

        Show
        d4rr3ll Darrell Taylor added a comment - There were no tests around this originally, do some need to be written? Manual steps taken to test this were : cd hadoop-common-project/hadoop-common/src/site/markdown/release/ python ../../../../../../dev-support/releasedocmaker.py --version 3.0.0 --index Then confirmed that files were produced in hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-SNAPSHOT/. The CHANGES.3.0.0-SNAPSHOT.md file contained the work 'Unreleased' in this case. The same command was run again with the --usetoday switch and the CHANGES.3.0.0-SNAPSHOT.md included today's date instead of 'Unreleased'. This was also checked through the maven build by running mvn pre-site -DskipTests -Preleasedocs The CHANGES.3.0.0-SNAPSHOT.md file included today's date instead of 'Unlreleased'
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726606/HADOOP-11813.patch
        against trunk revision c17cd4f.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6128//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6128//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726606/HADOOP-11813.patch against trunk revision c17cd4f. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6128//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6128//console This message is automatically generated.
        Hide
        d4rr3ll Darrell Taylor added a comment -
        • --usetoday option added
        • Small fix to escape out asterisks, these were breaking the tables
        • pom.xml updated to use the new argument
        Show
        d4rr3ll Darrell Taylor added a comment - --usetoday option added Small fix to escape out asterisks, these were breaking the tables pom.xml updated to use the new argument
        Hide
        aw Allen Wittenauer added a comment -

        Great! The pom.xml that calls releasedocs is in hadoop-common-project/hadoop-common/ .

        Show
        aw Allen Wittenauer added a comment - Great! The pom.xml that calls releasedocs is in hadoop-common-project/hadoop-common/ .
        Hide
        d4rr3ll Darrell Taylor added a comment -

        I'll pick this one up. One question about the pom.xml though, is that the top level one? As I can't seem to find any mention of 'releasedocs' in there.

        Show
        d4rr3ll Darrell Taylor added a comment - I'll pick this one up. One question about the pom.xml though, is that the top level one? As I can't seem to find any mention of 'releasedocs' in there.
        Hide
        aw Allen Wittenauer added a comment -

        Effectively:

        add a flag called --usetoday. If the version is unreleased and --usetoday is set, then use today's date. The pom.xml file for -Preleasedocs also needs to have this added to the argument list.

        Show
        aw Allen Wittenauer added a comment - Effectively: add a flag called --usetoday. If the version is unreleased and --usetoday is set, then use today's date. The pom.xml file for -Preleasedocs also needs to have this added to the argument list.

          People

          • Assignee:
            d4rr3ll Darrell Taylor
            Reporter:
            aw Allen Wittenauer
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development