Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: debian, rpm, toolchain
    • Labels:

      Activity

      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Jonathan Kelly I've submitted a patch (https://github.com/apache/bigtop/pull/98) to fix this. Thanks!

      Show
      fsareshwala Faraaz Sareshwala added a comment - Jonathan Kelly I've submitted a patch ( https://github.com/apache/bigtop/pull/98 ) to fix this. Thanks!
      Hide
      jonathak Jonathan Kelly added a comment -

      Thanks a lot!

      Show
      jonathak Jonathan Kelly added a comment - Thanks a lot!
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      I've created BIGTOP-2364 to track this. Patch coming up.

      Show
      fsareshwala Faraaz Sareshwala added a comment - I've created BIGTOP-2364 to track this. Patch coming up.
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Thanks for the note! Should I be doing that?

      Show
      fsareshwala Faraaz Sareshwala added a comment - Thanks for the note! Should I be doing that?
      Hide
      jonathak Jonathan Kelly added a comment -

      Oh, I see. It's because you are not sourcing the bigtop.bom file in qfs's do-component-build or install_qfs.sh scripts, as is done for other apps like Spark.

      Show
      jonathak Jonathan Kelly added a comment - Oh, I see. It's because you are not sourcing the bigtop.bom file in qfs's do-component-build or install_qfs.sh scripts, as is done for other apps like Spark.
      Hide
      jonathak Jonathan Kelly added a comment -

      I'm confused about why HADOOP_VERSION is explicitly passed through to qfs's do-component-build and install_qfs.sh scripts, when the bigtop.bom already provides it via the HADOOP_VERSION environment variable? For instance, note how the spark's do-component-build and install_spark.sh scripts reference $HADOOP_VERSION without having to pass it via the command line args. Also, you are setting HADOOP_VERSION to the Hadoop package version rather than the base version, which is normally OK (since they are usually the same value), but it can cause a problem if they are different.

      Show
      jonathak Jonathan Kelly added a comment - I'm confused about why HADOOP_VERSION is explicitly passed through to qfs's do-component-build and install_qfs.sh scripts, when the bigtop.bom already provides it via the HADOOP_VERSION environment variable? For instance, note how the spark's do-component-build and install_spark.sh scripts reference $HADOOP_VERSION without having to pass it via the command line args. Also, you are setting HADOOP_VERSION to the Hadoop package version rather than the base version, which is normally OK (since they are usually the same value), but it can cause a problem if they are different.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala closed the pull request at:

      https://github.com/apache/bigtop/pull/83

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala closed the pull request at: https://github.com/apache/bigtop/pull/83
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala commented on the pull request:

      https://github.com/apache/bigtop/pull/83#issuecomment-192962740

      Olaf committed this code into bigtop: https://issues.apache.org/jira/browse/BIGTOP-2285

      I am going to close this pull request out.

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala commented on the pull request: https://github.com/apache/bigtop/pull/83#issuecomment-192962740 Olaf committed this code into bigtop: https://issues.apache.org/jira/browse/BIGTOP-2285 I am going to close this pull request out.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user c0s commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r55145881

      — Diff: bigtop-packages/src/rpm/qfs/SPECS/qfs.spec —
      @@ -0,0 +1,360 @@
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +%define qfs_name qfs
      +
      +%define bin_dir /usr/bin
      +%define lib_dir /usr/lib/qfs
      +%define etc_dir %{_sysconfdir}/qfs
      +%define include_dir /usr/include
      +%define data_dir /usr/share/qfs
      +%define var_dir /var
      +%define qfs_services chunkserver metaserver webui
      +
      +%global HADOOP_VERSION %

      {hadoop_version}

      +
      +%if %{?!HADOOP_VERSION:1}0
      — End diff –

      Hmm, my bad then. I though the up-level build.gradle will enforce it. I will fix it later, for now just disregard this comment. Thanks

      Show
      githubbot ASF GitHub Bot added a comment - Github user c0s commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r55145881 — Diff: bigtop-packages/src/rpm/qfs/SPECS/qfs.spec — @@ -0,0 +1,360 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +%define qfs_name qfs + +%define bin_dir /usr/bin +%define lib_dir /usr/lib/qfs +%define etc_dir %{_sysconfdir}/qfs +%define include_dir /usr/include +%define data_dir /usr/share/qfs +%define var_dir /var +%define qfs_services chunkserver metaserver webui + +%global HADOOP_VERSION % {hadoop_version} + +%if %{?!HADOOP_VERSION:1}0 — End diff – Hmm, my bad then. I though the up-level build.gradle will enforce it. I will fix it later, for now just disregard this comment. Thanks
      Hide
      oflebbe Olaf Flebbe added a comment -

      committed

      Show
      oflebbe Olaf Flebbe added a comment - committed
      Hide
      oflebbe Olaf Flebbe added a comment -

      There are a couple of fine points which can be solved later on, I think.

      Thank you very much for your contribution!

      Will commit current #83 to bigtop .

      Show
      oflebbe Olaf Flebbe added a comment - There are a couple of fine points which can be solved later on, I think. Thank you very much for your contribution! Will commit current #83 to bigtop .
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Sorry Olaf. I just realized that the master branch hadn't been updated with the latest code. Just the branches that were the sources of the pull request. I just updated the master branch so it should be good to review now. Sorry about that!

      Show
      fsareshwala Faraaz Sareshwala added a comment - Sorry Olaf. I just realized that the master branch hadn't been updated with the latest code. Just the branches that were the sources of the pull request. I just updated the master branch so it should be good to review now. Sorry about that!
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r54286010

      — Diff: bigtop-packages/src/rpm/qfs/SPECS/qfs.spec —
      @@ -0,0 +1,360 @@
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +%define qfs_name qfs
      +
      +%define bin_dir /usr/bin
      +%define lib_dir /usr/lib/qfs
      +%define etc_dir %{_sysconfdir}/qfs
      +%define include_dir /usr/include
      +%define data_dir /usr/share/qfs
      +%define var_dir /var
      +%define qfs_services chunkserver metaserver webui
      +
      +%global HADOOP_VERSION %

      {hadoop_version}

      +
      +%if %{?!HADOOP_VERSION:1}0
      — End diff –

      We do pass it through from the bigtop bom. See my changes to packages.gradle to see how this is handled and flows through.

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r54286010 — Diff: bigtop-packages/src/rpm/qfs/SPECS/qfs.spec — @@ -0,0 +1,360 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +%define qfs_name qfs + +%define bin_dir /usr/bin +%define lib_dir /usr/lib/qfs +%define etc_dir %{_sysconfdir}/qfs +%define include_dir /usr/include +%define data_dir /usr/share/qfs +%define var_dir /var +%define qfs_services chunkserver metaserver webui + +%global HADOOP_VERSION % {hadoop_version} + +%if %{?!HADOOP_VERSION:1}0 — End diff – We do pass it through from the bigtop bom. See my changes to packages.gradle to see how this is handled and flows through.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r54285917

      — Diff: bigtop-packages/src/deb/qfs/rules —
      @@ -0,0 +1,81 @@
      +#!/usr/bin/make -f
      +
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +#
      +# makefile
      +
      +# Uncomment this to turn on verbose mode.
      +export DH_VERBOSE=1
      +
      +# This has to be exported to make some magic below work.
      +export DH_OPTIONS
      +
      +# Build configuration – override with environment variables
      +SHELL := /bin/bash
      +HADOOP_VERSION ?= 1.0.4
      — End diff –

      We do pass it through in the packages.gradle. This is just a default in case it isn't passed through for some reason.

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r54285917 — Diff: bigtop-packages/src/deb/qfs/rules — @@ -0,0 +1,81 @@ +#!/usr/bin/make -f + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# makefile + +# Uncomment this to turn on verbose mode. +export DH_VERBOSE=1 + +# This has to be exported to make some magic below work. +export DH_OPTIONS + +# Build configuration – override with environment variables +SHELL := /bin/bash +HADOOP_VERSION ?= 1.0.4 — End diff – We do pass it through in the packages.gradle. This is just a default in case it isn't passed through for some reason.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r54285663

      — Diff: bigtop-packages/src/common/qfs/do-component-build —
      @@ -0,0 +1,133 @@
      +#!/bin/bash
      +
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +# For debugging:
      +# set -ex
      +
      +# Autodetect JAVA_HOME if not defined
      +source /usr/lib/bigtop-utils/bigtop-detect-javahome
      — End diff –

      If I remove this, then JAVA_HOME isn't passed on through the makefile for qfs and we can't locate jni and build the jni bindings. Should this not be the case?

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r54285663 — Diff: bigtop-packages/src/common/qfs/do-component-build — @@ -0,0 +1,133 @@ +#!/bin/bash + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# For debugging: +# set -ex + +# Autodetect JAVA_HOME if not defined +source /usr/lib/bigtop-utils/bigtop-detect-javahome — End diff – If I remove this, then JAVA_HOME isn't passed on through the makefile for qfs and we can't locate jni and build the jni bindings. Should this not be the case?
      Hide
      cos Konstantin Boudnik added a comment -

      It does support 1.0, but in 0.4.0 we moved to Hadoop 2.x, so 0.3.x line is for Hadoop 1.x. Which is long time abandoned. Besides, is there anyone who still cares about Hadoop 1.x?

      Show
      cos Konstantin Boudnik added a comment - It does support 1.0, but in 0.4.0 we moved to Hadoop 2.x, so 0.3.x line is for Hadoop 1.x. Which is long time abandoned. Besides, is there anyone who still cares about Hadoop 1.x?
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user youngwookim commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r54212968

      — Diff: bigtop-packages/src/rpm/qfs/SPECS/qfs.spec —
      @@ -0,0 +1,360 @@
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +%define qfs_name qfs
      +
      +%define bin_dir /usr/bin
      +%define lib_dir /usr/lib/qfs
      +%define etc_dir %{_sysconfdir}/qfs
      +%define include_dir /usr/include
      +%define data_dir /usr/share/qfs
      +%define var_dir /var
      +%define qfs_services chunkserver metaserver webui
      +
      +%global HADOOP_VERSION %

      {hadoop_version}

      +
      +%if %{?!HADOOP_VERSION:1}0
      — End diff –

      Would be good to have default Hadoop version from bigtop bom. Bigtop does not support Hadoop 1.x at this point.

      Show
      githubbot ASF GitHub Bot added a comment - Github user youngwookim commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r54212968 — Diff: bigtop-packages/src/rpm/qfs/SPECS/qfs.spec — @@ -0,0 +1,360 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +%define qfs_name qfs + +%define bin_dir /usr/bin +%define lib_dir /usr/lib/qfs +%define etc_dir %{_sysconfdir}/qfs +%define include_dir /usr/include +%define data_dir /usr/share/qfs +%define var_dir /var +%define qfs_services chunkserver metaserver webui + +%global HADOOP_VERSION % {hadoop_version} + +%if %{?!HADOOP_VERSION:1}0 — End diff – Would be good to have default Hadoop version from bigtop bom. Bigtop does not support Hadoop 1.x at this point.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user youngwookim commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r54212818

      — Diff: bigtop-packages/src/deb/qfs/rules —
      @@ -0,0 +1,81 @@
      +#!/usr/bin/make -f
      +
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +#
      +# makefile
      +
      +# Uncomment this to turn on verbose mode.
      +export DH_VERBOSE=1
      +
      +# This has to be exported to make some magic below work.
      +export DH_OPTIONS
      +
      +# Build configuration – override with environment variables
      +SHELL := /bin/bash
      +HADOOP_VERSION ?= 1.0.4
      — End diff –

      Bigtop defines HADOOP_VERSION in bom so, just pass the variable $

      {HADOOP_VERSION}

      or $

      {HADOOP_BASE_VERSION}

      . ref. https://github.com/apache/bigtop/blob/master/bigtop-packages/src/deb/zookeeper/rules#L39

      Show
      githubbot ASF GitHub Bot added a comment - Github user youngwookim commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r54212818 — Diff: bigtop-packages/src/deb/qfs/rules — @@ -0,0 +1,81 @@ +#!/usr/bin/make -f + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# makefile + +# Uncomment this to turn on verbose mode. +export DH_VERBOSE=1 + +# This has to be exported to make some magic below work. +export DH_OPTIONS + +# Build configuration – override with environment variables +SHELL := /bin/bash +HADOOP_VERSION ?= 1.0.4 — End diff – Bigtop defines HADOOP_VERSION in bom so, just pass the variable $ {HADOOP_VERSION} or $ {HADOOP_BASE_VERSION} . ref. https://github.com/apache/bigtop/blob/master/bigtop-packages/src/deb/zookeeper/rules#L39
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user youngwookim commented on a diff in the pull request:

      https://github.com/apache/bigtop/pull/83#discussion_r54212291

      — Diff: bigtop-packages/src/common/qfs/do-component-build —
      @@ -0,0 +1,133 @@
      +#!/bin/bash
      +
      +# Licensed to the Apache Software Foundation (ASF) under one or more
      +# contributor license agreements. See the NOTICE file distributed with
      +# this work for additional information regarding copyright ownership.
      +# The ASF licenses this file to You under the Apache License, Version 2.0
      +# (the "License"); you may not use this file except in compliance with
      +# the License. You may obtain a copy of the License at
      +#
      +# http://www.apache.org/licenses/LICENSE-2.0
      +#
      +# Unless required by applicable law or agreed to in writing, software
      +# distributed under the License is distributed on an "AS IS" BASIS,
      +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      +# See the License for the specific language governing permissions and
      +# limitations under the License.
      +
      +# For debugging:
      +# set -ex
      +
      +# Autodetect JAVA_HOME if not defined
      +source /usr/lib/bigtop-utils/bigtop-detect-javahome
      — End diff –

      I believe JAVA_HOME detection is not necessary at build stage because we have bigtop toolchain. So, please remove this bits.

      Show
      githubbot ASF GitHub Bot added a comment - Github user youngwookim commented on a diff in the pull request: https://github.com/apache/bigtop/pull/83#discussion_r54212291 — Diff: bigtop-packages/src/common/qfs/do-component-build — @@ -0,0 +1,133 @@ +#!/bin/bash + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# For debugging: +# set -ex + +# Autodetect JAVA_HOME if not defined +source /usr/lib/bigtop-utils/bigtop-detect-javahome — End diff – I believe JAVA_HOME detection is not necessary at build stage because we have bigtop toolchain. So, please remove this bits.
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Thanks for taking another look Olaf. I've updated the code to fix all the issues you mentioned. I'm still ignoring the binary-without-manpage lintian warning since I am not able to provide manpages at the moment. I will do that in a future release. I took care of various other errors so the lintian overrides are significantly less now. Some are proper to ignore (e.g. new-package-should-close-itp-bug).

      Please let me know what you think about the latest code.

      Show
      fsareshwala Faraaz Sareshwala added a comment - Thanks for taking another look Olaf. I've updated the code to fix all the issues you mentioned. I'm still ignoring the binary-without-manpage lintian warning since I am not able to provide manpages at the moment. I will do that in a future release. I took care of various other errors so the lintian overrides are significantly less now. Some are proper to ignore (e.g. new-package-should-close-itp-bug). Please let me know what you think about the latest code.
      Hide
      oflebbe Olaf Flebbe added a comment -

      I looked again at the debian package.

      Building is o.k. But I do not agree reducing the errors by overriding lintian defaults. My intention was that you fix the packages instead. If something has to be fixed upstream, o.k. But please remove the lintian overrides.

      Installation errors:
      There are conflicts between qfs-client and qfs-metaserver . Both own qfsfsck.

      Installing qfs-metaserver has errors while installing

      root@e534e7776385:/qfs# dpkg -i qfs-metaserver_1.1.4-1_amd64.deb 
      (Reading database ... 14261 files and directories currently installed.)
      Preparing to unpack qfs-metaserver_1.1.4-1_amd64.deb ...
      Unpacking qfs-metaserver (1.1.4-1) ...
      Setting up qfs-metaserver (1.1.4-1) ...
      chown: cannot access '/var/run/qfs': No such file or directory
      dpkg: error processing package qfs-metaserver (--install):
       subprocess installed post-installation script returned error exit status 1
      Processing triggers for systemd (215-17+deb8u2) ...
      Errors were encountered while processing:
       qfs-metaserver
      root@e534e7776385:/qfs#  cannot access '/var/run/qfs': No such file or directory^C
      

      This seems to be because /var/run aka /run is a temporary dir, you have to create directories in your start script.

      Show
      oflebbe Olaf Flebbe added a comment - I looked again at the debian package. Building is o.k. But I do not agree reducing the errors by overriding lintian defaults. My intention was that you fix the packages instead. If something has to be fixed upstream, o.k. But please remove the lintian overrides. Installation errors: There are conflicts between qfs-client and qfs-metaserver . Both own qfsfsck. Installing qfs-metaserver has errors while installing root@e534e7776385:/qfs# dpkg -i qfs-metaserver_1.1.4-1_amd64.deb (Reading database ... 14261 files and directories currently installed.) Preparing to unpack qfs-metaserver_1.1.4-1_amd64.deb ... Unpacking qfs-metaserver (1.1.4-1) ... Setting up qfs-metaserver (1.1.4-1) ... chown: cannot access '/ var /run/qfs': No such file or directory dpkg: error processing package qfs-metaserver (--install): subprocess installed post-installation script returned error exit status 1 Processing triggers for systemd (215-17+deb8u2) ... Errors were encountered while processing: qfs-metaserver root@e534e7776385:/qfs# cannot access '/ var /run/qfs': No such file or directory^C This seems to be because /var/run aka /run is a temporary dir, you have to create directories in your start script.
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Thanks for the reviews guys. Really appreciate it.

      Does anyone have any comments on the packaging structure? Are files where they should be to conform to the bigtop standards?

      Show
      fsareshwala Faraaz Sareshwala added a comment - Thanks for the reviews guys. Really appreciate it. Does anyone have any comments on the packaging structure? Are files where they should be to conform to the bigtop standards?
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Thanks Olaf. I've updated the code on the github pull request to eliminate all errors and have reduced the warnings by quite a bit. I've set a few warnings and errors as overridden and ignored (e.g. binary-without-manpage). Is that okay?

      Show
      fsareshwala Faraaz Sareshwala added a comment - Thanks Olaf. I've updated the code on the github pull request to eliminate all errors and have reduced the warnings by quite a bit. I've set a few warnings and errors as overridden and ignored (e.g. binary-without-manpage). Is that okay?
      Hide
      cos Konstantin Boudnik added a comment -

      Please reduce the size of the commit message. git log isn't to replace JIRA Thanks!

      Show
      cos Konstantin Boudnik added a comment - Please reduce the size of the commit message. git log isn't to replace JIRA Thanks!
      Hide
      cos Konstantin Boudnik added a comment -

      I think we can safely skip centos-6, as I believe we have decided not to support it anymore.

      Show
      cos Konstantin Boudnik added a comment - I think we can safely skip centos-6, as I believe we have decided not to support it anymore.
      Hide
      oflebbe Olaf Flebbe added a comment - - edited

      At least it builds now on debian, ubuntu, centos-7. (didn't try centos-6 and fedora right now)

      There are a couple of lintian errors (debian). I like to get Errors out and number of Warnings reduced. (forget about the force-reload Error)

      W: qfs-python: missing-depends-line
      E: qfs-metaserver: dir-or-file-in-var-run var/run/qfs/
      E: qfs-metaserver: postrm-does-not-call-updaterc.d-for-init.d-script etc/init.d/qfs-metaserver
      E: qfs-metaserver: postrm-contains-additional-updaterc.d-calls etc/init.d/qfs-chunkserver
      E: qfs-metaserver: init.d-script-does-not-implement-required-option etc/init.d/qfs-metaserver force-reload
      W: qfs-metaserver: binary-without-manpage usr/bin/filelister
      W: qfs-metaserver: binary-without-manpage usr/bin/logcompactor
      W: qfs-metaserver: binary-without-manpage usr/bin/metaserver
      W: qfs-metaserver: binary-without-manpage usr/bin/qfsfsck
      W: qfs-client: hardening-no-relro usr/lib/qfs/libJerasure.so
      W: qfs-client: hardening-no-relro usr/lib/qfs/libJerasure.so.2
      W: qfs-client: hardening-no-relro usr/lib/qfs/libJerasure.so.2.0.0
      W: qfs-client: binary-without-manpage usr/bin/cpfromqfs
      W: qfs-client: binary-without-manpage usr/bin/cptoqfs
      W: qfs-client: binary-without-manpage usr/bin/qfs
      W: qfs-client: binary-without-manpage usr/bin/qfs_backup
      W: qfs-client: binary-without-manpage usr/bin/qfs_checkpoint_prune.py
      W: qfs-client: binary-without-manpage usr/bin/qfs_log_prune.py
      W: qfs-client: binary-without-manpage usr/bin/qfsadmin
      W: qfs-client: binary-without-manpage usr/bin/qfscat
      W: qfs-client: binary-without-manpage usr/bin/qfsdataverify
      W: qfs-client: binary-without-manpage usr/bin/qfsfileenum
      W: qfs-client: binary-without-manpage usr/bin/qfsfsck
      W: qfs-client: binary-without-manpage usr/bin/qfshibernate
      W: qfs-client: binary-without-manpage usr/bin/qfsping
      W: qfs-client: binary-without-manpage usr/bin/qfsput
      W: qfs-client: binary-without-manpage usr/bin/qfssample
      W: qfs-client: binary-without-manpage usr/bin/qfsshell
      W: qfs-client: binary-without-manpage usr/bin/qfsstats
      W: qfs-client: binary-without-manpage usr/bin/qfstoggleworm
      E: qfs-client: python-script-but-no-python-dep usr/bin/qfs_backup
      E: qfs-client: python-script-but-no-python-dep usr/bin/qfs_checkpoint_prune.py
      E: qfs-client: python-script-but-no-python-dep usr/bin/qfs_log_prune.py
      W: qfs-client: postinst-has-useless-call-to-ldconfig
      W: qfs-client: postrm-has-useless-call-to-ldconfig
      E: qfs-webui: dir-or-file-in-var-run var/run/qfs/
      E: qfs-webui: postrm-does-not-call-updaterc.d-for-init.d-script etc/init.d/qfs-webui
      E: qfs-webui: postrm-contains-additional-updaterc.d-calls etc/init.d/qfs-chunkserver
      E: qfs-webui: init.d-script-does-not-implement-required-option etc/init.d/qfs-webui force-reload
      E: qfs-webui: python-script-but-no-python-dep usr/share/qfs/webui/qfsstatus.py
      W: qfs-webui: executable-not-elf-or-script usr/share/qfs/webui/files/sorttable/arrow-down.gif
      W: qfs-webui: executable-not-elf-or-script usr/share/qfs/webui/files/sorttable/arrow-none.gif
      W: qfs-webui: executable-not-elf-or-script usr/share/qfs/webui/files/sorttable/arrow-up.gif
      W: qfs-fuse: binary-without-manpage usr/bin/qfs_fuse
      E: qfs-chunkserver: dir-or-file-in-var-run var/run/qfs/
      E: qfs-chunkserver: init.d-script-does-not-implement-required-option etc/init.d/qfs-chunkserver force-reload
      W: qfs-chunkserver: binary-without-manpage usr/bin/chunkscrubber
      W: qfs-chunkserver: binary-without-manpage usr/bin/chunkserver
      W: qfs: empty-binary-package
      W: qfs-dev: binary-without-manpage usr/bin/checksum
      W: qfs-dev: binary-without-manpage usr/bin/dirtree_creator
      W: qfs-dev: binary-without-manpage usr/bin/dtokentest
      W: qfs-dev: binary-without-manpage usr/bin/qfslogger
      W: qfs-dev: binary-without-manpage usr/bin/rand-sfmt
      W: qfs-dev: binary-without-manpage usr/bin/rebalanceexecutor
      W: qfs-dev: binary-without-manpage usr/bin/rebalanceplanner
      W: qfs-dev: binary-without-manpage usr/bin/replicachecker
      W: qfs-dev: binary-without-manpage usr/bin/requestparser
      W: qfs-dev: binary-without-manpage usr/bin/sortedhash
      W: qfs-dev: binary-without-manpage usr/bin/sslfiltertest
      W: qfs-dev: binary-without-manpage usr/bin/stlset
      
      Show
      oflebbe Olaf Flebbe added a comment - - edited At least it builds now on debian, ubuntu, centos-7. (didn't try centos-6 and fedora right now) There are a couple of lintian errors (debian). I like to get Errors out and number of Warnings reduced. (forget about the force-reload Error) W: qfs-python: missing-depends-line E: qfs-metaserver: dir-or-file-in- var -run var /run/qfs/ E: qfs-metaserver: postrm-does-not-call-updaterc.d- for -init.d-script etc/init.d/qfs-metaserver E: qfs-metaserver: postrm-contains-additional-updaterc.d-calls etc/init.d/qfs-chunkserver E: qfs-metaserver: init.d-script-does-not-implement-required-option etc/init.d/qfs-metaserver force-reload W: qfs-metaserver: binary-without-manpage usr/bin/filelister W: qfs-metaserver: binary-without-manpage usr/bin/logcompactor W: qfs-metaserver: binary-without-manpage usr/bin/metaserver W: qfs-metaserver: binary-without-manpage usr/bin/qfsfsck W: qfs-client: hardening-no-relro usr/lib/qfs/libJerasure.so W: qfs-client: hardening-no-relro usr/lib/qfs/libJerasure.so.2 W: qfs-client: hardening-no-relro usr/lib/qfs/libJerasure.so.2.0.0 W: qfs-client: binary-without-manpage usr/bin/cpfromqfs W: qfs-client: binary-without-manpage usr/bin/cptoqfs W: qfs-client: binary-without-manpage usr/bin/qfs W: qfs-client: binary-without-manpage usr/bin/qfs_backup W: qfs-client: binary-without-manpage usr/bin/qfs_checkpoint_prune.py W: qfs-client: binary-without-manpage usr/bin/qfs_log_prune.py W: qfs-client: binary-without-manpage usr/bin/qfsadmin W: qfs-client: binary-without-manpage usr/bin/qfscat W: qfs-client: binary-without-manpage usr/bin/qfsdataverify W: qfs-client: binary-without-manpage usr/bin/qfsfileenum W: qfs-client: binary-without-manpage usr/bin/qfsfsck W: qfs-client: binary-without-manpage usr/bin/qfshibernate W: qfs-client: binary-without-manpage usr/bin/qfsping W: qfs-client: binary-without-manpage usr/bin/qfsput W: qfs-client: binary-without-manpage usr/bin/qfssample W: qfs-client: binary-without-manpage usr/bin/qfsshell W: qfs-client: binary-without-manpage usr/bin/qfsstats W: qfs-client: binary-without-manpage usr/bin/qfstoggleworm E: qfs-client: python-script-but-no-python-dep usr/bin/qfs_backup E: qfs-client: python-script-but-no-python-dep usr/bin/qfs_checkpoint_prune.py E: qfs-client: python-script-but-no-python-dep usr/bin/qfs_log_prune.py W: qfs-client: postinst-has-useless-call-to-ldconfig W: qfs-client: postrm-has-useless-call-to-ldconfig E: qfs-webui: dir-or-file-in- var -run var /run/qfs/ E: qfs-webui: postrm-does-not-call-updaterc.d- for -init.d-script etc/init.d/qfs-webui E: qfs-webui: postrm-contains-additional-updaterc.d-calls etc/init.d/qfs-chunkserver E: qfs-webui: init.d-script-does-not-implement-required-option etc/init.d/qfs-webui force-reload E: qfs-webui: python-script-but-no-python-dep usr/share/qfs/webui/qfsstatus.py W: qfs-webui: executable-not-elf-or-script usr/share/qfs/webui/files/sorttable/arrow-down.gif W: qfs-webui: executable-not-elf-or-script usr/share/qfs/webui/files/sorttable/arrow-none.gif W: qfs-webui: executable-not-elf-or-script usr/share/qfs/webui/files/sorttable/arrow-up.gif W: qfs-fuse: binary-without-manpage usr/bin/qfs_fuse E: qfs-chunkserver: dir-or-file-in- var -run var /run/qfs/ E: qfs-chunkserver: init.d-script-does-not-implement-required-option etc/init.d/qfs-chunkserver force-reload W: qfs-chunkserver: binary-without-manpage usr/bin/chunkscrubber W: qfs-chunkserver: binary-without-manpage usr/bin/chunkserver W: qfs: empty-binary- package W: qfs-dev: binary-without-manpage usr/bin/checksum W: qfs-dev: binary-without-manpage usr/bin/dirtree_creator W: qfs-dev: binary-without-manpage usr/bin/dtokentest W: qfs-dev: binary-without-manpage usr/bin/qfslogger W: qfs-dev: binary-without-manpage usr/bin/rand-sfmt W: qfs-dev: binary-without-manpage usr/bin/rebalanceexecutor W: qfs-dev: binary-without-manpage usr/bin/rebalanceplanner W: qfs-dev: binary-without-manpage usr/bin/replicachecker W: qfs-dev: binary-without-manpage usr/bin/requestparser W: qfs-dev: binary-without-manpage usr/bin/sortedhash W: qfs-dev: binary-without-manpage usr/bin/sslfiltertest W: qfs-dev: binary-without-manpage usr/bin/stlset
      Hide
      oflebbe Olaf Flebbe added a comment -

      opensuse compile fails because krb5-config ist not found. You have to manually add /usr/lib/mit/bin/ into your PATH. That's one of the peculiarities of SuSE distributions.

      You should be able to download a suitable machine (w/o your additions) with

      docker pull bigtop/slaves:trunk-opensuse-13.2
      
      Show
      oflebbe Olaf Flebbe added a comment - opensuse compile fails because krb5-config ist not found. You have to manually add /usr/lib/mit/bin/ into your PATH. That's one of the peculiarities of SuSE distributions. You should be able to download a suitable machine (w/o your additions) with docker pull bigtop/slaves:trunk-opensuse-13.2
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Thanks for testing this Olaf. It looks like the vm I was using to test the packaging already had maven installed. I've fixed up the code and tested in using the bigtop docker provisioner to make sure it all works. I've also updated the pull request with the latest patch.

      I tried to set up opensuse under docker to test this myself but I couldn't get things quite working. I did manage to test it under centos and ubuntu so I believe that things should be building correctly now for both build types. Could you take a look again?

      Show
      fsareshwala Faraaz Sareshwala added a comment - Thanks for testing this Olaf. It looks like the vm I was using to test the packaging already had maven installed. I've fixed up the code and tested in using the bigtop docker provisioner to make sure it all works. I've also updated the pull request with the latest patch. I tried to set up opensuse under docker to test this myself but I couldn't get things quite working. I did manage to test it under centos and ubuntu so I believe that things should be building correctly now for both build types. Could you take a look again?
      Hide
      oflebbe Olaf Flebbe added a comment -

      The debian and ubuntu packages do not compile because of the unmet build requirement "maven" ,
      since we do not install the "maven" package from debian/ubuntu (for a purpose) .

      opensuse fails at

      running build_ext
      building 'qfs' extension
      creating build
      creating build/temp.linux-x86_64-2.7
      creating build/temp.linux-x86_64-2.7/ws
      creating build/temp.linux-x86_64-2.7/ws/build
      creating build/temp.linux-x86_64-2.7/ws/build/qfs
      creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm
      creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD
      creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4
      creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src
      creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc
      creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access
      gcc -pthread -fno-strict-aliasing -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DNDEBUG -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DOPENSSL_LOAD_CONF -fPIC -I/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc -I/usr/include/python2.7 -c /ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access/kfs_module_py.cc -o build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access/kfs_module_py.o
      creating build/lib.linux-x86_64-2.7
      g++ -pthread -shared build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access/kfs_module_py.o -Lsrc/cc/libclient -Lsrc/cc/common -Lsrc/cc/kfsio -Lsrc/cc/qcdio -Lsrc/cc/qcrs -L/usr/lib64 -lqfs_client -lqfs_common -lqfs_io -lqfs_qcdio -lqfs_qcrs -lboost_regex -lpython2.7 -o build/lib.linux-x86_64-2.7/qfs.so
      /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_client
      /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_common
      /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_io
      /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_qcdio
      /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_qcrs
      collect2: error: ld returned 1 exit status
      error: command 'g++' failed with exit status 1
      

      Could you please handle at least the ubuntu/debian issue?

      Show
      oflebbe Olaf Flebbe added a comment - The debian and ubuntu packages do not compile because of the unmet build requirement "maven" , since we do not install the "maven" package from debian/ubuntu (for a purpose) . opensuse fails at running build_ext building 'qfs' extension creating build creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/ws creating build/temp.linux-x86_64-2.7/ws/build creating build/temp.linux-x86_64-2.7/ws/build/qfs creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4 creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc creating build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access gcc -pthread -fno-strict-aliasing -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DNDEBUG -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -DOPENSSL_LOAD_CONF -fPIC -I/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc -I/usr/include/python2.7 -c /ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access/kfs_module_py.cc -o build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access/kfs_module_py.o creating build/lib.linux-x86_64-2.7 g++ -pthread -shared build/temp.linux-x86_64-2.7/ws/build/qfs/rpm/BUILD/qfs-1.1.4/src/cc/access/kfs_module_py.o -Lsrc/cc/libclient -Lsrc/cc/common -Lsrc/cc/kfsio -Lsrc/cc/qcdio -Lsrc/cc/qcrs -L/usr/lib64 -lqfs_client -lqfs_common -lqfs_io -lqfs_qcdio -lqfs_qcrs -lboost_regex -lpython2.7 -o build/lib.linux-x86_64-2.7/qfs.so /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_client /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_common /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_io /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_qcdio /usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -lqfs_qcrs collect2: error: ld returned 1 exit status error: command 'g++' failed with exit status 1 Could you please handle at least the ubuntu/debian issue?
      Hide
      fsareshwala Faraaz Sareshwala added a comment -

      Thanks Olaf! Let me know if you need any help with anything. I'm available on IRC as well as email if you want a more realtime communication mechanism.

      Show
      fsareshwala Faraaz Sareshwala added a comment - Thanks Olaf! Let me know if you need any help with anything. I'm available on IRC as well as email if you want a more realtime communication mechanism.
      Hide
      oflebbe Olaf Flebbe added a comment -

      Thank you very much for your contribution! Testing it right now, if it compiles in my environment.

      Show
      oflebbe Olaf Flebbe added a comment - Thank you very much for your contribution! Testing it right now, if it compiles in my environment.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala commented on the pull request:

      https://github.com/apache/bigtop/pull/77#issuecomment-181656131

      New packaging pull request: https://github.com/apache/bigtop/pull/83

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala commented on the pull request: https://github.com/apache/bigtop/pull/77#issuecomment-181656131 New packaging pull request: https://github.com/apache/bigtop/pull/83
      Hide
      githubbot ASF GitHub Bot added a comment -

      GitHub user fsareshwala opened a pull request:

      https://github.com/apache/bigtop/pull/83

      BIGTOP-2285. add qfs rpm and debian packaging code

      This change adds code to bigtop to package qfs for debian and rpm systems. The
      packages produced are specific to their functions, e.g. a separate package for
      the metaserver, chunkserver, webui, client tools, development headers and
      libraries, etc. This way, users can select which packages they wish to install
      on which system in their infrastructure. One caveat worth mentioning is that the
      packages themselves don't contain configuration files for the services in
      question. The configuration files are provided by the puppet deploy scripts that
      come with the bigtop repository. Those will be submitted in a separate change.

      In order to build qfs and its respective packages, various dependencies were
      added to the toolchain puppet tasks (e.g. boost-devel). These will be installed
      along with the other dependencies that get installed during the toolchain gradle
      task. In doing so, users should be presented with a seamless setup process to
      get qfs packages building.

      I have tested to make sure that packaging is correct by using the qfs-rpm and
      qfs-deb grandle tasks to generate the respective rpm and debian packages. I then
      manually inspected package contents to ensure that files were being installed
      into the correct location.

      You can merge this pull request into a Git repository by running:

      $ git pull https://github.com/fsareshwala/bigtop qfs-packaging

      Alternatively you can review and apply these changes as the patch at:

      https://github.com/apache/bigtop/pull/83.patch

      To close this pull request, make a commit to your master/trunk branch
      with (at least) the following in the commit message:

      This closes #83


      commit c5f45c8f4b6e43e06f439a0cdb89844be65571fc
      Author: Faraaz Sareshwala <fsareshwala@quantcast.com>
      Date: 2016-01-13T23:29:46Z

      BIGTOP-2285. add qfs rpm and debian packaging code

      This change adds code to bigtop to package qfs for debian and rpm systems. The
      packages produced are specific to their functions, e.g. a separate package for
      the metaserver, chunkserver, webui, client tools, development headers and
      libraries, etc. This way, users can select which packages they wish to install
      on which system in their infrastructure. One caveat worth mentioning is that the
      packages themselves don't contain configuration files for the services in
      question. The configuration files are provided by the puppet deploy scripts that
      come with the bigtop repository. Those will be submitted in a separate change.

      In order to build qfs and its respective packages, various dependencies were
      added to the toolchain puppet tasks (e.g. boost-devel). These will be installed
      along with the other dependencies that get installed during the toolchain gradle
      task. In doing so, users should be presented with a seamless setup process to
      get qfs packages building.

      I have tested to make sure that packaging is correct by using the qfs-rpm and
      qfs-deb grandle tasks to generate the respective rpm and debian packages. I then
      manually inspected package contents to ensure that files were being installed
      into the correct location.


      Show
      githubbot ASF GitHub Bot added a comment - GitHub user fsareshwala opened a pull request: https://github.com/apache/bigtop/pull/83 BIGTOP-2285 . add qfs rpm and debian packaging code This change adds code to bigtop to package qfs for debian and rpm systems. The packages produced are specific to their functions, e.g. a separate package for the metaserver, chunkserver, webui, client tools, development headers and libraries, etc. This way, users can select which packages they wish to install on which system in their infrastructure. One caveat worth mentioning is that the packages themselves don't contain configuration files for the services in question. The configuration files are provided by the puppet deploy scripts that come with the bigtop repository. Those will be submitted in a separate change. In order to build qfs and its respective packages, various dependencies were added to the toolchain puppet tasks (e.g. boost-devel). These will be installed along with the other dependencies that get installed during the toolchain gradle task. In doing so, users should be presented with a seamless setup process to get qfs packages building. I have tested to make sure that packaging is correct by using the qfs-rpm and qfs-deb grandle tasks to generate the respective rpm and debian packages. I then manually inspected package contents to ensure that files were being installed into the correct location. You can merge this pull request into a Git repository by running: $ git pull https://github.com/fsareshwala/bigtop qfs-packaging Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/83.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #83 commit c5f45c8f4b6e43e06f439a0cdb89844be65571fc Author: Faraaz Sareshwala <fsareshwala@quantcast.com> Date: 2016-01-13T23:29:46Z BIGTOP-2285 . add qfs rpm and debian packaging code This change adds code to bigtop to package qfs for debian and rpm systems. The packages produced are specific to their functions, e.g. a separate package for the metaserver, chunkserver, webui, client tools, development headers and libraries, etc. This way, users can select which packages they wish to install on which system in their infrastructure. One caveat worth mentioning is that the packages themselves don't contain configuration files for the services in question. The configuration files are provided by the puppet deploy scripts that come with the bigtop repository. Those will be submitted in a separate change. In order to build qfs and its respective packages, various dependencies were added to the toolchain puppet tasks (e.g. boost-devel). These will be installed along with the other dependencies that get installed during the toolchain gradle task. In doing so, users should be presented with a seamless setup process to get qfs packages building. I have tested to make sure that packaging is correct by using the qfs-rpm and qfs-deb grandle tasks to generate the respective rpm and debian packages. I then manually inspected package contents to ensure that files were being installed into the correct location.
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala closed the pull request at:

      https://github.com/apache/bigtop/pull/77

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala closed the pull request at: https://github.com/apache/bigtop/pull/77
      Hide
      githubbot ASF GitHub Bot added a comment -

      Github user fsareshwala commented on the pull request:

      https://github.com/apache/bigtop/pull/77#issuecomment-181655944

      Closing this pull request out so that I can fix the source branch.

      Show
      githubbot ASF GitHub Bot added a comment - Github user fsareshwala commented on the pull request: https://github.com/apache/bigtop/pull/77#issuecomment-181655944 Closing this pull request out so that I can fix the source branch.
      Hide
      oflebbe Olaf Flebbe added a comment -

      Added Faraaz Sareshwala to contributors and assigned ticket.

      Show
      oflebbe Olaf Flebbe added a comment - Added Faraaz Sareshwala to contributors and assigned ticket.
      Show
      fsareshwala Faraaz Sareshwala added a comment - https://github.com/apache/bigtop/pull/77
      Hide
      githubbot ASF GitHub Bot added a comment -

      GitHub user fsareshwala opened a pull request:

      https://github.com/apache/bigtop/pull/77

      BIGTOP-2285. add qfs rpm and debian packaging code

      This change adds code to bigtop to package qfs for debian and rpm systems. The
      packages produced are specific to their functions, e.g. a separate package for
      the metaserver, chunkserver, webui, client tools, development headers and
      libraries, etc. This way, users can select which packages they wish to install
      on which system in their infrastructure.

      In order to build qfs and its respective packages, various dependencies were
      added to the toolchain puppet tasks (e.g. boost-devel). These will be installed
      along with the other dependencies that get installed during the toolchain gradle
      task. In doing so, users should be presented with a seamless setup process to
      get qfs packages building.

      I have tested to make sure that packaging is correct by using the qfs-rpm and
      qfs-deb grandle tasks to generate the respective rpm and debian packages. I then
      manually inspected package contents to ensure that files were being installed
      into the correct location.

      You can merge this pull request into a Git repository by running:

      $ git pull https://github.com/fsareshwala/bigtop master

      Alternatively you can review and apply these changes as the patch at:

      https://github.com/apache/bigtop/pull/77.patch

      To close this pull request, make a commit to your master/trunk branch
      with (at least) the following in the commit message:

      This closes #77


      commit dd33286309ef111d2ac23bf2ab1df327b64c971a
      Author: Faraaz Sareshwala <fsareshwala@quantcast.com>
      Date: 2016-01-13T23:29:46Z

      BIGTOP-2285. add qfs rpm and debian packaging code

      This change adds code to bigtop to package qfs for debian and rpm systems. The
      packages produced are specific to their functions, e.g. a separate package for
      the metaserver, chunkserver, webui, client tools, development headers and
      libraries, etc. This way, users can select which packages they wish to install
      on which system in their infrastructure.

      In order to build qfs and its respective packages, various dependencies were
      added to the toolchain puppet tasks (e.g. boost-devel). These will be installed
      along with the other dependencies that get installed during the toolchain gradle
      task. In doing so, users should be presented with a seamless setup process to
      get qfs packages building.

      I have tested to make sure that packaging is correct by using the qfs-rpm and
      qfs-deb grandle tasks to generate the respective rpm and debian packages. I then
      manually inspected package contents to ensure that files were being installed
      into the correct location.


      Show
      githubbot ASF GitHub Bot added a comment - GitHub user fsareshwala opened a pull request: https://github.com/apache/bigtop/pull/77 BIGTOP-2285 . add qfs rpm and debian packaging code This change adds code to bigtop to package qfs for debian and rpm systems. The packages produced are specific to their functions, e.g. a separate package for the metaserver, chunkserver, webui, client tools, development headers and libraries, etc. This way, users can select which packages they wish to install on which system in their infrastructure. In order to build qfs and its respective packages, various dependencies were added to the toolchain puppet tasks (e.g. boost-devel). These will be installed along with the other dependencies that get installed during the toolchain gradle task. In doing so, users should be presented with a seamless setup process to get qfs packages building. I have tested to make sure that packaging is correct by using the qfs-rpm and qfs-deb grandle tasks to generate the respective rpm and debian packages. I then manually inspected package contents to ensure that files were being installed into the correct location. You can merge this pull request into a Git repository by running: $ git pull https://github.com/fsareshwala/bigtop master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/77.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #77 commit dd33286309ef111d2ac23bf2ab1df327b64c971a Author: Faraaz Sareshwala <fsareshwala@quantcast.com> Date: 2016-01-13T23:29:46Z BIGTOP-2285 . add qfs rpm and debian packaging code This change adds code to bigtop to package qfs for debian and rpm systems. The packages produced are specific to their functions, e.g. a separate package for the metaserver, chunkserver, webui, client tools, development headers and libraries, etc. This way, users can select which packages they wish to install on which system in their infrastructure. In order to build qfs and its respective packages, various dependencies were added to the toolchain puppet tasks (e.g. boost-devel). These will be installed along with the other dependencies that get installed during the toolchain gradle task. In doing so, users should be presented with a seamless setup process to get qfs packages building. I have tested to make sure that packaging is correct by using the qfs-rpm and qfs-deb grandle tasks to generate the respective rpm and debian packages. I then manually inspected package contents to ensure that files were being installed into the correct location.

        People

        • Assignee:
          fsareshwala Faraaz Sareshwala
          Reporter:
          fsareshwala Faraaz Sareshwala
        • Votes:
          0 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development