Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.1.0
    • Component/s: build
    • Labels:
      None

      Description

      Release process shouldn't bundle non-essential bits into the release artifact.

      released source code is expected to be same with source code in branch:

      1. released source code:

      wget http://www.apache.org/dist/bigtop/bigtop-0.8.0/bigtop-0.8.0-project.tar.gz

      2. source code in branch:

      wget https://github.com/apache/bigtop/archive/release-0.8.0.zip

      3. diff code:

      [hadoop@localhost ~]$ diff -rNu bigtop-release-0.8.0 bigtop-0.8.0 | diffstat
      .idea/.name | 1
      .idea/compiler.xml | 28
      .idea/copyright/profiles_settings.xml | 5
      .idea/encodings.xml | 7
      .idea/misc.xml | 72
      .idea/modules.xml | 9
      .idea/scopes/scope_settings.xml | 5
      .idea/uiDesigner.xml | 125 +
      .idea/vcs.xml | 7
      .idea/workspace.xml | 809 ++++++++++
      bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/apache.access.2.log | 1
      bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/apache.access.log | 1
      bigtop.iml | 14
      dl/apache-tomcat-6.0.35.tar.gz |binary
      dl/hadoop-2.0.2-alpha.tar.gz |binary
      dl/spark-0.7.0.tar.gz |binary
      16 files changed, 1082 insertions, 2 deletions

      1. BIGTOP-1499.patch
        4 kB
        Konstantin Boudnik
      2. BIGTOP-1499.patch
        4 kB
        Konstantin Boudnik
      3. BIGTOP-1499.patch
        4 kB
        Konstantin Boudnik

        Activity

        Hide
        cos Konstantin Boudnik added a comment -

        I don't see dl/ in the tar.gz archive - which is expected.
        Also, running

        for i in pom.xml bigtop.mk build.gradle packages.gradle ; do echo $i; diff $i ../bigtop-0.8.0/$i; done

        I am getting no diffs at all.

        pom.xml
        bigtop.mk
        build.gradle
        packages.gradle
        

        Please explain in more details what diffs you're seeing. Otherwise, I will close it as non-repro

        Show
        cos Konstantin Boudnik added a comment - I don't see dl/ in the tar.gz archive - which is expected. Also, running for i in pom.xml bigtop.mk build.gradle packages.gradle ; do echo $i; diff $i ../bigtop-0.8.0/$i; done I am getting no diffs at all. pom.xml bigtop.mk build.gradle packages.gradle Please explain in more details what diffs you're seeing. Otherwise, I will close it as non-repro
        Hide
        rguo Guo Ruijing added a comment -

        299 wget http://www.apache.org/dist/bigtop/bigtop-0.8.0/bigtop-0.8.0-project.tar.gz
        300 tar xzvf bigtop-0.8.0-project.tar.gz
        301 cd bigtop-0.8.0
        302 ls
        303 ls dl

        [hadoop@localhost bigtop-0.8.0]$ ls dl
        apache-tomcat-6.0.35.tar.gz hadoop-2.0.2-alpha.tar.gz spark-0.7.0.tar.gz <<< not expected

        Show
        rguo Guo Ruijing added a comment - 299 wget http://www.apache.org/dist/bigtop/bigtop-0.8.0/bigtop-0.8.0-project.tar.gz 300 tar xzvf bigtop-0.8.0-project.tar.gz 301 cd bigtop-0.8.0 302 ls 303 ls dl [hadoop@localhost bigtop-0.8.0] $ ls dl apache-tomcat-6.0.35.tar.gz hadoop-2.0.2-alpha.tar.gz spark-0.7.0.tar.gz <<< not expected
        Hide
        cos Konstantin Boudnik added a comment - - edited

        Yeah, you're right - dl/ is present. Anything else besides dl/ ?
        Considering that dl/ isn't a part of the source tree I won't bother too much about it being present in the release tar-ball.

        However, release steps in the build need to be updated so next time it isn't included into the source artifact.

        Show
        cos Konstantin Boudnik added a comment - - edited Yeah, you're right - dl/ is present. Anything else besides dl/ ? Considering that dl/ isn't a part of the source tree I won't bother too much about it being present in the release tar-ball. However, release steps in the build need to be updated so next time it isn't included into the source artifact.
        Hide
        rguo Guo Ruijing added a comment - - edited

        1. update JIRA description.

        2. file diff list:
        [hadoop@localhost ~]$ diff -rNu bigtop-release-0.8.0 bigtop-0.8.0 | diffstat
        .idea/.name | 1
        .idea/compiler.xml | 28
        .idea/copyright/profiles_settings.xml | 5
        .idea/encodings.xml | 7
        .idea/misc.xml | 72
        .idea/modules.xml | 9
        .idea/scopes/scope_settings.xml | 5
        .idea/uiDesigner.xml | 125 +
        .idea/vcs.xml | 7
        .idea/workspace.xml | 809 ++++++++++
        bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/apache.access.2.log | 1
        bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/apache.access.log | 1
        bigtop.iml | 14
        dl/apache-tomcat-6.0.35.tar.gz |binary
        dl/hadoop-2.0.2-alpha.tar.gz |binary
        dl/spark-0.7.0.tar.gz |binary
        16 files changed, 1082 insertions, 2 deletions

        Show
        rguo Guo Ruijing added a comment - - edited 1. update JIRA description. 2. file diff list: [hadoop@localhost ~] $ diff -rNu bigtop-release-0.8.0 bigtop-0.8.0 | diffstat .idea/.name | 1 .idea/compiler.xml | 28 .idea/copyright/profiles_settings.xml | 5 .idea/encodings.xml | 7 .idea/misc.xml | 72 .idea/modules.xml | 9 .idea/scopes/scope_settings.xml | 5 .idea/uiDesigner.xml | 125 + .idea/vcs.xml | 7 .idea/workspace.xml | 809 ++++++++++ bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/apache.access.2.log | 1 bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/apache.access.log | 1 bigtop.iml | 14 dl/apache-tomcat-6.0.35.tar.gz |binary dl/hadoop-2.0.2-alpha.tar.gz |binary dl/spark-0.7.0.tar.gz |binary 16 files changed, 1082 insertions , 2 deletions
        Hide
        rguo Guo Ruijing added a comment -

        question: how to generate release tar-ball?

        Show
        rguo Guo Ruijing added a comment - question: how to generate release tar-ball?
        Hide
        cos Konstantin Boudnik added a comment -

        Fair enough - these bits will have to be excluded from the release packaging before the next release.
        If there's no objection, I will make this a subtask of BIGTOP-1463, which is a blocker for 0.9

        Show
        cos Konstantin Boudnik added a comment - Fair enough - these bits will have to be excluded from the release packaging before the next release. If there's no objection, I will make this a subtask of BIGTOP-1463 , which is a blocker for 0.9
        Hide
        cos Konstantin Boudnik added a comment -

        question: how to generate release tar-ball?

        You should be able to get this by running mvn -Prelease package assembly:assembly, according to How To Release page

        Show
        cos Konstantin Boudnik added a comment - question: how to generate release tar-ball? You should be able to get this by running mvn -Prelease package assembly:assembly , according to How To Release page
        Hide
        cos Konstantin Boudnik added a comment -

        We'll have a discussion about this issue during the coming hackathon on 10/29/14 and if PMC feels like we need to re-spin the release - I will do it.

        Show
        cos Konstantin Boudnik added a comment - We'll have a discussion about this issue during the coming hackathon on 10/29/14 and if PMC feels like we need to re-spin the release - I will do it.
        Hide
        cos Konstantin Boudnik added a comment -

        I have looked up at re-publishing the release tarball. It is not a hard thing to do, however, it requires a deployment of the artifacts into repositories.apache.org Nexus server. And I think releases are immutable, so we can't just push another 0.8.0 release out there.

        Considering that the issue with the content is really minor and isn't affecting the source code per se, I'd suggest to punt on this and address the underlying issue in the next release. Any objections?

        Show
        cos Konstantin Boudnik added a comment - I have looked up at re-publishing the release tarball. It is not a hard thing to do, however, it requires a deployment of the artifacts into repositories.apache.org Nexus server. And I think releases are immutable, so we can't just push another 0.8.0 release out there. Considering that the issue with the content is really minor and isn't affecting the source code per se, I'd suggest to punt on this and address the underlying issue in the next release. Any objections?
        Hide
        rguo Guo Ruijing added a comment -

        Agree. We should not republish 0.8.0 release since release is immutable.
        We can fix it in 0.9.0.

        Show
        rguo Guo Ruijing added a comment - Agree. We should not republish 0.8.0 release since release is immutable. We can fix it in 0.9.0.
        Hide
        cos Konstantin Boudnik added a comment -

        There are two way of addressing this issue:

        1. Forcing fresh shallow clone of the repo to create a release from
        2. Add maven assembly so we only release what we need to release, namely the source code.

        The former is easier, the latter is a bit more robust, IMO. Thoughts?

        Show
        cos Konstantin Boudnik added a comment - There are two way of addressing this issue: Forcing fresh shallow clone of the repo to create a release from Add maven assembly so we only release what we need to release, namely the source code. The former is easier, the latter is a bit more robust, IMO. Thoughts?
        Hide
        cos Konstantin Boudnik added a comment -

        I have rolled out bits for 1.0.0 release and looks like #1 above worked just fine.

        Show
        cos Konstantin Boudnik added a comment - I have rolled out bits for 1.0.0 release and looks like #1 above worked just fine.
        Hide
        cos Konstantin Boudnik added a comment -

        Correction - there's still some garbage getting it, hence a second respin is required now. Looks like it'd make sense to invest to #2, after all.

        Show
        cos Konstantin Boudnik added a comment - Correction - there's still some garbage getting it, hence a second respin is required now. Looks like it'd make sense to invest to #2, after all.
        Hide
        cos Konstantin Boudnik added a comment -

        I have introduced the release assembly to control closely what's going into the release artifact. It looks that this also solves BIGTOP-1959, but I need to do some more tests.

        Would appreciate a second or third pair of eyes on this one. Thanks!

        Show
        cos Konstantin Boudnik added a comment - I have introduced the release assembly to control closely what's going into the release artifact. It looks that this also solves BIGTOP-1959 , but I need to do some more tests. Would appreciate a second or third pair of eyes on this one. Thanks!
        Hide
        apurtell Andrew Purtell added a comment -

        I eyeballed the patch and it looks good to me except for a small nit.
        Please remove:

                <!--<exclude>**/**</exclude>-->
        

        Let me try building from a release tarball - will report back.

        Show
        apurtell Andrew Purtell added a comment - I eyeballed the patch and it looks good to me except for a small nit. Please remove: <!--<exclude>**/**</exclude>--> Let me try building from a release tarball - will report back.
        Hide
        cos Konstantin Boudnik added a comment -

        Ah, of course... Thanks!

        Show
        cos Konstantin Boudnik added a comment - Ah, of course... Thanks!
        Hide
        cos Konstantin Boudnik added a comment -

        I am going to commit the patch and if anything is found later, we'll get it fixed. So far I am quite satisfied with how the release assembly is coming out

        Show
        cos Konstantin Boudnik added a comment - I am going to commit the patch and if anything is found later, we'll get it fixed. So far I am quite satisfied with how the release assembly is coming out
        Hide
        cos Konstantin Boudnik added a comment -

        Missing the maintainers and building text files.

        Show
        cos Konstantin Boudnik added a comment - Missing the maintainers and building text files.
        Hide
        cos Konstantin Boudnik added a comment -

        Pushed to the master; the same functionality will later needs to be moved to gradle build, once we're ready to get rid of mvn completely.

        Show
        cos Konstantin Boudnik added a comment - Pushed to the master; the same functionality will later needs to be moved to gradle build, once we're ready to get rid of mvn completely.
        Hide
        apurtell Andrew Purtell added a comment -

        Ok, +1

        Show
        apurtell Andrew Purtell added a comment - Ok, +1

          People

          • Assignee:
            cos Konstantin Boudnik
            Reporter:
            rguo Guo Ruijing
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development