Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: debian, flink
    • Labels:
      None

      Description

      Right now, flink debian package comes with flink-jobmanager and flink-taskmanager installed and running. There should be separate packages for these two components, so an user can install only binaries (maybe in order to run flink jobs on yarn...).

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/bigtop/pull/132

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/bigtop/pull/132
          Hide
          rvs Roman Shaposhnik added a comment -

          Sorry for this taking a long time! Committed now!

          Show
          rvs Roman Shaposhnik added a comment - Sorry for this taking a long time! Committed now!
          Hide
          cos Konstantin Boudnik added a comment -

          David Moravek, Roman Shaposhnik: is this ready to go in? If so - let's pull this in as the release is getting closer.

          Show
          cos Konstantin Boudnik added a comment - David Moravek , Roman Shaposhnik : is this ready to go in? If so - let's pull this in as the release is getting closer.
          Hide
          davidmoravek David Moravek added a comment -

          fixed

          Show
          davidmoravek David Moravek added a comment - fixed
          Hide
          rvs Roman Shaposhnik added a comment -

          David Moravek I found two small issues with the RPM side. Please take a look and address those. The DEB looks great. Would love to commit both tomorrow (right in time for the Flink meetup )

          Show
          rvs Roman Shaposhnik added a comment - David Moravek I found two small issues with the RPM side. Please take a look and address those. The DEB looks great. Would love to commit both tomorrow (right in time for the Flink meetup )
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rvs commented on a diff in the pull request:

          https://github.com/apache/bigtop/pull/132#discussion_r71269745

          — Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec —
          @@ -53,6 +54,14 @@ Source6: bigtop.bom
          Requires: bigtop-utils >= 0.7
          Requires(preun): /sbin/service

          +%if %{?suse_version:1}0
          — End diff –

          This whole block needs to be removed. Once you add two of the subpackages there's no reason to have main package required these

          Show
          githubbot ASF GitHub Bot added a comment - Github user rvs commented on a diff in the pull request: https://github.com/apache/bigtop/pull/132#discussion_r71269745 — Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec — @@ -53,6 +54,14 @@ Source6: bigtop.bom Requires: bigtop-utils >= 0.7 Requires(preun): /sbin/service +%if %{?suse_version:1}0 — End diff – This whole block needs to be removed. Once you add two of the subpackages there's no reason to have main package required these
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rvs commented on a diff in the pull request:

          https://github.com/apache/bigtop/pull/132#discussion_r71269543

          — Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec —
          @@ -106,29 +134,17 @@ bash $RPM_SOURCE_DIR/do-component-build

          sh -x %

          {SOURCE2}

          --prefix=$RPM_BUILD_ROOT --source-dir=$RPM_SOURCE_DIR --build-dir=`pwd`/build-target

          -
          -
          for service in %

          {flink_services}
          do
          # Install init script
          init_file=$RPM_BUILD_ROOT/%{initd_dir}/${service}
          bash %{SOURCE3} $RPM_SOURCE_DIR/${service}.svc rpm $init_file
          done

          -
          -%preun
          -for service in %{flink_services}

          ; do

          • /sbin/service $ {service} status > /dev/null 2>&1
            - if [ $? -eq 0 ]; then
            - /sbin/service ${service}

            stop > /dev/null 2>&1

          • fi
            -done
            -
            %pre
            getent group flink >/dev/null || groupadd -r flink
            getent passwd flink >/dev/null || useradd -c "Flink" -s /sbin/nologin -g flink -r -d % {lib_flink}

            flink 2> /dev/null || :

          -
          %post
          %

          {alternatives_cmd}

          --install %

          {config_flink} %{flink_name}-conf %{config_flink}

          .dist 30
          systemctl daemon-reload
          — End diff –

          I don't think this belongs here: systemctl is only available on systemd-based Linux distros, Bigtop currently supports init.d

          Show
          githubbot ASF GitHub Bot added a comment - Github user rvs commented on a diff in the pull request: https://github.com/apache/bigtop/pull/132#discussion_r71269543 — Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec — @@ -106,29 +134,17 @@ bash $RPM_SOURCE_DIR/do-component-build sh -x % {SOURCE2} --prefix=$RPM_BUILD_ROOT --source-dir=$RPM_SOURCE_DIR --build-dir=`pwd`/build-target - - for service in % {flink_services} do # Install init script init_file=$RPM_BUILD_ROOT/%{initd_dir}/${service} bash %{SOURCE3} $RPM_SOURCE_DIR/${service}.svc rpm $init_file done - -%preun -for service in %{flink_services} ; do /sbin/service $ {service} status > /dev/null 2>&1 - if [ $? -eq 0 ]; then - /sbin/service ${service} stop > /dev/null 2>&1 fi -done - %pre getent group flink >/dev/null || groupadd -r flink getent passwd flink >/dev/null || useradd -c "Flink" -s /sbin/nologin -g flink -r -d % {lib_flink} flink 2> /dev/null || : - %post % {alternatives_cmd} --install % {config_flink} %{flink_name}-conf %{config_flink} .dist 30 systemctl daemon-reload — End diff – I don't think this belongs here: systemctl is only available on systemd-based Linux distros, Bigtop currently supports init.d
          Hide
          rvs Roman Shaposhnik added a comment -

          David Moravek awesome! It looks good to me now. I'll do some additional testing tonight and will either commit or come back with the feedback still.

          Great job, tho! Thanks for the patch!

          Show
          rvs Roman Shaposhnik added a comment - David Moravek awesome! It looks good to me now. I'll do some additional testing tonight and will either commit or come back with the feedback still. Great job, tho! Thanks for the patch!
          Hide
          davidmoravek David Moravek added a comment -

          Changed according to your suggestions. Thanks

          Show
          davidmoravek David Moravek added a comment - Changed according to your suggestions. Thanks
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rvs commented on a diff in the pull request:

          https://github.com/apache/bigtop/pull/132#discussion_r70703555

          — Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec —
          @@ -72,19 +81,38 @@ Some of the key features of Apache Flink includes.

          • Fault-tolerance via Lightweight Distributed Snapshots
          • Hadoop-native YARN & HDFS implementation
            1. Additions for master-worker configuration #
              +%package jobmanager
              +Summary: Provides the Apache Flink Job Manager service.
              • End diff –

          Its better to keep the description exactly the same between Debian and RPM packages

          Show
          githubbot ASF GitHub Bot added a comment - Github user rvs commented on a diff in the pull request: https://github.com/apache/bigtop/pull/132#discussion_r70703555 — Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec — @@ -72,19 +81,38 @@ Some of the key features of Apache Flink includes. Fault-tolerance via Lightweight Distributed Snapshots Hadoop-native YARN & HDFS implementation Additions for master-worker configuration # +%package jobmanager +Summary: Provides the Apache Flink Job Manager service. End diff – Its better to keep the description exactly the same between Debian and RPM packages
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rvs commented on a diff in the pull request:

          https://github.com/apache/bigtop/pull/132#discussion_r70702844

          — Diff: bigtop-packages/src/deb/flink/flink-jobmanager.postinst —
          @@ -0,0 +1,33 @@
          +#!/bin/bash
          +#
          +# Licensed to the Apache Software Foundation (ASF) under one or more
          +# contributor license agreements. See the NOTICE file distributed with
          +# this work for additional information regarding copyright ownership.
          +# The ASF licenses this file to You under the Apache License, Version 2.0
          +# (the "License"); you may not use this file except in compliance with
          +# the License. You may obtain a copy of the License at
          +#
          +# http://www.apache.org/licenses/LICENSE-2.0
          +#
          +# Unless required by applicable law or agreed to in writing, software
          +# distributed under the License is distributed on an "AS IS" BASIS,
          +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          +# See the License for the specific language governing permissions and
          +# limitations under the License.
          +
          +set -e
          +
          +case "$1" in
          + configure)
          — End diff –

          This is a good place to make sure you update the alternatives files for the /etc/flink/conf if needed

          Show
          githubbot ASF GitHub Bot added a comment - Github user rvs commented on a diff in the pull request: https://github.com/apache/bigtop/pull/132#discussion_r70702844 — Diff: bigtop-packages/src/deb/flink/flink-jobmanager.postinst — @@ -0,0 +1,33 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -e + +case "$1" in + configure) — End diff – This is a good place to make sure you update the alternatives files for the /etc/flink/conf if needed
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rvs commented on a diff in the pull request:

          https://github.com/apache/bigtop/pull/132#discussion_r70702283

          — Diff: bigtop-packages/src/deb/flink/control —
          @@ -27,3 +27,13 @@ Depends: bigtop-utils (>= 0.7), adduser
          Description: Flink’s core is a streaming dataflow engine that provides data
          distribution, communication, and fault tolerance for distributed
          computations over data streams.
          +
          +Package: flink-jobmanager
          +Architecture: all
          +Depends: flink
          +Description: Flink’s jobmanager.
          +
          +Package: flink-taskmanager
          +Architecture: all
          +Depends: flink
          — End diff –

          See above.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rvs commented on a diff in the pull request: https://github.com/apache/bigtop/pull/132#discussion_r70702283 — Diff: bigtop-packages/src/deb/flink/control — @@ -27,3 +27,13 @@ Depends: bigtop-utils (>= 0.7), adduser Description: Flink’s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams. + +Package: flink-jobmanager +Architecture: all +Depends: flink +Description: Flink’s jobmanager. + +Package: flink-taskmanager +Architecture: all +Depends: flink — End diff – See above.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rvs commented on a diff in the pull request:

          https://github.com/apache/bigtop/pull/132#discussion_r70702252

          — Diff: bigtop-packages/src/deb/flink/control —
          @@ -27,3 +27,13 @@ Depends: bigtop-utils (>= 0.7), adduser
          Description: Flink’s core is a streaming dataflow engine that provides data
          distribution, communication, and fault tolerance for distributed
          computations over data streams.
          +
          +Package: flink-jobmanager
          +Architecture: all
          +Depends: flink
          — End diff –

          you need
          Depends: flink (= $

          {source:Version}

          )
          here instead to make sure that there's 1-1 correspondence version-wise

          Show
          githubbot ASF GitHub Bot added a comment - Github user rvs commented on a diff in the pull request: https://github.com/apache/bigtop/pull/132#discussion_r70702252 — Diff: bigtop-packages/src/deb/flink/control — @@ -27,3 +27,13 @@ Depends: bigtop-utils (>= 0.7), adduser Description: Flink’s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams. + +Package: flink-jobmanager +Architecture: all +Depends: flink — End diff – you need Depends: flink (= $ {source:Version} ) here instead to make sure that there's 1-1 correspondence version-wise
          Hide
          davidmoravek David Moravek added a comment -

          Done, but it was my very first RPM package, so someone should take a closer look.

          Show
          davidmoravek David Moravek added a comment - Done, but it was my very first RPM package, so someone should take a closer look.
          Hide
          rvs Roman Shaposhnik added a comment -

          David Moravek this is a great suggestion and thanks a million for the patch! We have to, of course, keep RPM and DEB packaging in sync. Any chance you can update the patch to include relevant RPM changes as well?

          Show
          rvs Roman Shaposhnik added a comment - David Moravek this is a great suggestion and thanks a million for the patch! We have to, of course, keep RPM and DEB packaging in sync. Any chance you can update the patch to include relevant RPM changes as well?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user dmvk opened a pull request:

          https://github.com/apache/bigtop/pull/132

          BIGTOP-2492 Split flink debian packaging

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/dmvk/bigtop BIGTOP-2492

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/bigtop/pull/132.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #132


          commit 084d420298c9c1d7b8ee2b8659ca0949a4419bcb
          Author: David Moravek <david.moravek@firma.seznam.cz>
          Date: 2016-07-13T14:29:11Z

          BIGTOP-2492 Split flink debian packaging


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user dmvk opened a pull request: https://github.com/apache/bigtop/pull/132 BIGTOP-2492 Split flink debian packaging You can merge this pull request into a Git repository by running: $ git pull https://github.com/dmvk/bigtop BIGTOP-2492 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/132.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #132 commit 084d420298c9c1d7b8ee2b8659ca0949a4419bcb Author: David Moravek <david.moravek@firma.seznam.cz> Date: 2016-07-13T14:29:11Z BIGTOP-2492 Split flink debian packaging

            People

            • Assignee:
              davidmoravek David Moravek
              Reporter:
              davidmoravek David Moravek
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development