Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: debian, rpm, tests
    • Labels:
      None
    1. BIGTOP-2179.2.patch
      34 kB
      Yeongeon KIM
    2. BIGTOP-2179.3.patch
      34 kB
      Yeongeon KIM
    3. BIGTOP-2179.patch
      32 kB
      Yeongeon KIM

      Activity

      Hide
      githubbot ASF GitHub Bot added a comment -

      GitHub user yeongeon opened a pull request:

      https://github.com/apache/bigtop/pull/66

      BIGTOP-2179: Add packaging scripts for Apache Tajo

      You can merge this pull request into a Git repository by running:

      $ git pull https://github.com/yeongeon/bigtop BIGTOP-2179

      Alternatively you can review and apply these changes as the patch at:

      https://github.com/apache/bigtop/pull/66.patch

      To close this pull request, make a commit to your master/trunk branch
      with (at least) the following in the commit message:

      This closes #66


      commit aeb4581587de5516110dfa10ef20261b6c723615
      Author: yeongeon <yeongeon@gmail.com>
      Date: 2015-12-16T10:50:58Z

      BIGTOP-2179: Add packaging scripts for Apache Tajo


      Show
      githubbot ASF GitHub Bot added a comment - GitHub user yeongeon opened a pull request: https://github.com/apache/bigtop/pull/66 BIGTOP-2179 : Add packaging scripts for Apache Tajo You can merge this pull request into a Git repository by running: $ git pull https://github.com/yeongeon/bigtop BIGTOP-2179 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/66.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #66 commit aeb4581587de5516110dfa10ef20261b6c723615 Author: yeongeon <yeongeon@gmail.com> Date: 2015-12-16T10:50:58Z BIGTOP-2179 : Add packaging scripts for Apache Tajo
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user jayunit100 commented on the pull request:

      https://github.com/apache/bigtop/pull/66#issuecomment-165106862

      This looks promising.
      In general puppet deployment and smoke tests are also good to have.

      Are you building those as well? You can usually just copy examples from something like pig or spark puppet recipes in our repo to get started .

      Smoke tests are also quite easy just a groovy file which you can dump in smoke-tests.

      The vagrant recipes can then be used to build and test from scratch

      Show
      githubbot ASF GitHub Bot added a comment - Github user jayunit100 commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-165106862 This looks promising. In general puppet deployment and smoke tests are also good to have. Are you building those as well? You can usually just copy examples from something like pig or spark puppet recipes in our repo to get started . Smoke tests are also quite easy just a groovy file which you can dump in smoke-tests. The vagrant recipes can then be used to build and test from scratch
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user yeongeon commented on the pull request:

      https://github.com/apache/bigtop/pull/66#issuecomment-165137389

      I just have tested rpm/deb as a standalone on centos/ubuntu.
      The smoke-tests and the deploy will be added step by step.

      Show
      githubbot ASF GitHub Bot added a comment - Github user yeongeon commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-165137389 I just have tested rpm/deb as a standalone on centos/ubuntu. The smoke-tests and the deploy will be added step by step.
      Hide
      jayunit100 jay vyas added a comment -

      Im +1 to commit this patch even though I've done only a very cursory review.

      Yeongeon KIM are you going to be available to maintain any issues this creates if i commit it ?

      Are others ok with me commiting this?

      cc Olaf Flebbe YoungWoo Kim Evans Ye ?

      Show
      jayunit100 jay vyas added a comment - Im +1 to commit this patch even though I've done only a very cursory review. Yeongeon KIM are you going to be available to maintain any issues this creates if i commit it ? Are others ok with me commiting this? cc Olaf Flebbe YoungWoo Kim Evans Ye ?
      Hide
      evans_ye Evans Ye added a comment -

      LGTM although I'm not an expert in packaging.
      It would be great to further have puppet and a simple smoke test so that we can setup end-to-end on our CI.

      Show
      evans_ye Evans Ye added a comment - LGTM although I'm not an expert in packaging. It would be great to further have puppet and a simple smoke test so that we can setup end-to-end on our CI.
      Hide
      cos Konstantin Boudnik added a comment -

      I have quickly reviewed the packaging PR - there is a couple of files, that don't need to be checked in. Please address the comments and I think it will be ready to go in.

      As Evans suggested, please also open a ticket to do the puppet recipes for the deployment of the component. Should be simple enough.

      Also, please make sure to add package tests like explained in here BIGTOP-2125

      Show
      cos Konstantin Boudnik added a comment - I have quickly reviewed the packaging PR - there is a couple of files, that don't need to be checked in. Please address the comments and I think it will be ready to go in. As Evans suggested, please also open a ticket to do the puppet recipes for the deployment of the component. Should be simple enough. Also, please make sure to add package tests like explained in here BIGTOP-2125
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user youngwookim commented on the pull request:

      https://github.com/apache/bigtop/pull/66#issuecomment-166591249

      The patch does not apply. Please rebase it.

      Show
      githubbot ASF GitHub Bot added a comment - Github user youngwookim commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-166591249 The patch does not apply. Please rebase it.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user youngwookim commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/66#discussion_r48245913

      — Diff: bigtop-packages/src/common/tajo/do-component-build —
      @@ -0,0 +1,28 @@
      +#!/bin/bash
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +set -ex
      +
      +# recognize for TAJO_VERSION, HADOOP_VERSION
      +. `dirname $0`/bigtop.bom
      +
      +rm -rf build
      +mkdir build
      +
      +mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=$HADOOP_VERSION "$@"
      — End diff –

      I believe, you should provide the appropriate hive.version for Hive integration

      Show
      githubbot ASF GitHub Bot added a comment - Github user youngwookim commented on a diff in the pull request: https://github.com/apache/bigtop/pull/66#discussion_r48245913 — Diff: bigtop-packages/src/common/tajo/do-component-build — @@ -0,0 +1,28 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -ex + +# recognize for TAJO_VERSION, HADOOP_VERSION +. `dirname $0`/bigtop.bom + +rm -rf build +mkdir build + +mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=$HADOOP_VERSION "$@" — End diff – I believe, you should provide the appropriate hive.version for Hive integration
      Show
      githubbot ASF GitHub Bot added a comment - Github user youngwookim commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-166593793 'compat' file is missing for deb? See https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Packaging Like this, https://github.com/apache/bigtop/blob/master/bigtop-packages/src/deb/hive/compat
      Hide
      oflebbe Olaf Flebbe added a comment -

      Yes the compat file is needed. It describes what version of your debian helper scripts you are expecting to use.

      Show
      oflebbe Olaf Flebbe added a comment - Yes the compat file is needed. It describes what version of your debian helper scripts you are expecting to use.
      Hide
      oflebbe Olaf Flebbe added a comment -

      Added Yeongeon KIM to Contributors and assigned this ticket .

      Show
      oflebbe Olaf Flebbe added a comment - Added Yeongeon KIM to Contributors and assigned this ticket .
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user yeongeon commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/66#discussion_r48430949

      — Diff: bigtop-packages/src/common/tajo/do-component-build —
      @@ -0,0 +1,28 @@
      +#!/bin/bash
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +set -ex
      +
      +# recognize for TAJO_VERSION, HADOOP_VERSION
      +. `dirname $0`/bigtop.bom
      +
      +rm -rf build
      +mkdir build
      +
      +mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=$HADOOP_VERSION "$@"
      — End diff –

      @youngwookim Your mention seems to be related with TAJO-1442 which remove hcatalog dependency. I think we first should make a new issue and then deal with it in this.

      Show
      githubbot ASF GitHub Bot added a comment - Github user yeongeon commented on a diff in the pull request: https://github.com/apache/bigtop/pull/66#discussion_r48430949 — Diff: bigtop-packages/src/common/tajo/do-component-build — @@ -0,0 +1,28 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -ex + +# recognize for TAJO_VERSION, HADOOP_VERSION +. `dirname $0`/bigtop.bom + +rm -rf build +mkdir build + +mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=$HADOOP_VERSION "$@" — End diff – @youngwookim Your mention seems to be related with TAJO-1442 which remove hcatalog dependency. I think we first should make a new issue and then deal with it in this.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user yeongeon commented on the pull request:

      https://github.com/apache/bigtop/pull/66#issuecomment-167192143

      Thanks guys. I have pushed three fixed.

      Show
      githubbot ASF GitHub Bot added a comment - Github user yeongeon commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-167192143 Thanks guys. I have pushed three fixed.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user yeongeon commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/66#discussion_r48447672

      — Diff: bigtop-packages/src/common/tajo/do-component-build —
      @@ -0,0 +1,28 @@
      +#!/bin/bash
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +set -ex
      +
      +# recognize for TAJO_VERSION, HADOOP_VERSION
      +. `dirname $0`/bigtop.bom
      +
      +rm -rf build
      +mkdir build
      +
      +mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=$HADOOP_VERSION "$@"
      — End diff –

      @youngwookim I had some problems while rebase but solved it using merge now.

      Show
      githubbot ASF GitHub Bot added a comment - Github user yeongeon commented on a diff in the pull request: https://github.com/apache/bigtop/pull/66#discussion_r48447672 — Diff: bigtop-packages/src/common/tajo/do-component-build — @@ -0,0 +1,28 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -ex + +# recognize for TAJO_VERSION, HADOOP_VERSION +. `dirname $0`/bigtop.bom + +rm -rf build +mkdir build + +mvn clean package -DskipTests -Pdist -Dtar -Dhadoop.version=$HADOOP_VERSION "$@" — End diff – @youngwookim I had some problems while rebase but solved it using merge now.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user yeongeon commented on the pull request:

      https://github.com/apache/bigtop/pull/66#issuecomment-168581062

      Let me know if this PR have any issues more for merging.
      (Also, PR/67 was related with this.)

      Show
      githubbot ASF GitHub Bot added a comment - Github user yeongeon commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-168581062 Let me know if this PR have any issues more for merging. (Also, PR/67 was related with this.)
      Hide
      oflebbe Olaf Flebbe added a comment -

      It compiles fine for me too.

      One last thing to get it committed: Please squash the contents of your branch into a single commit .
      Bigtop conventions are: One JIRA, one commit.

      (You can upload this commit to JIRA, too)

      Show
      oflebbe Olaf Flebbe added a comment - It compiles fine for me too. One last thing to get it committed: Please squash the contents of your branch into a single commit . Bigtop conventions are: One JIRA, one commit. (You can upload this commit to JIRA, too)
      Hide
      yeongeon Yeongeon KIM added a comment -

      I pushed squashing commit as you mentioned and attached a patch file at jira too.

      Show
      yeongeon Yeongeon KIM added a comment - I pushed squashing commit as you mentioned and attached a patch file at jira too.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user yeongeon commented on the pull request:

      https://github.com/apache/bigtop/pull/66#issuecomment-177692476

      I think this PR was updated as you guys are mentioned.
      Also, patch file was added on the issue.

      Show
      githubbot ASF GitHub Bot added a comment - Github user yeongeon commented on the pull request: https://github.com/apache/bigtop/pull/66#issuecomment-177692476 I think this PR was updated as you guys are mentioned. Also, patch file was added on the issue.
      Hide
      warwithin YoungWoo Kim added a comment - - edited

      +1, LGTM!

      Yeongeon KIM, Can you re-attach the patch using git format-patch ?(ref. "How to generate patches", https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute) Will commit then.

      Show
      warwithin YoungWoo Kim added a comment - - edited +1, LGTM! Yeongeon KIM , Can you re-attach the patch using git format-patch ?(ref. "How to generate patches", https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute ) Will commit then.
      Hide
      warwithin YoungWoo Kim added a comment -

      The patch from PR does not apply. Yeongeon KIM, please rebase to current master and then let us know. Thanks!

      Show
      warwithin YoungWoo Kim added a comment - The patch from PR does not apply. Yeongeon KIM , please rebase to current master and then let us know. Thanks!
      Hide
      yeongeon Yeongeon KIM added a comment -

      YoungWoo Kim Added it again. Thanks.

      Show
      yeongeon Yeongeon KIM added a comment - YoungWoo Kim Added it again. Thanks.
      Hide
      warwithin YoungWoo Kim added a comment - - edited

      Yeongeon KIM, Got the same error with BIGTOP-2179.2.patch:

      $ git am --signoff < BIGTOP-2179.2.patch
      ......
      
      error: patch failed: bigtop.bom:90
      error: bigtop.bom: patch does not apply
      Patch failed at 0001 BIGTOP-2179: Add packaging scripts for Apache Tajo
      The copy of the patch that failed is found in:
         /home/ywkim/workspace/bigtop/.git/rebase-apply/patch
      When you have resolved this problem, run "git am --continue".
      If you prefer to skip this patch, run "git am --skip" instead.
      To restore the original branch and stop patching, run "git am --abort".
      
      
      Show
      warwithin YoungWoo Kim added a comment - - edited Yeongeon KIM , Got the same error with BIGTOP-2179 .2.patch: $ git am --signoff < BIGTOP-2179.2.patch ...... error: patch failed: bigtop.bom:90 error: bigtop.bom: patch does not apply Patch failed at 0001 BIGTOP-2179: Add packaging scripts for Apache Tajo The copy of the patch that failed is found in: /home/ywkim/workspace/bigtop/.git/rebase-apply/patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
      Hide
      yeongeon Yeongeon KIM added a comment -

      YoungWoo Kim It had some problems. A new patch file was passed by command as 'git apply --check'. Thanks~

      Show
      yeongeon Yeongeon KIM added a comment - YoungWoo Kim It had some problems. A new patch file was passed by command as 'git apply --check'. Thanks~
      Hide
      warwithin YoungWoo Kim added a comment -

      Committed. Thank you Yeongeon KIM

      Show
      warwithin YoungWoo Kim added a comment - Committed. Thank you Yeongeon KIM

        People

        • Assignee:
          yeongeon Yeongeon KIM
          Reporter:
          yeongeon Yeongeon KIM
        • Votes:
          0 Vote for this issue
          Watchers:
          7 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development