Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-4057

Docker Image is twice as large as necessary

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.3.0
    • Fix Version/s: 1.4.0
    • Component/s: Docker
    • Labels:
      None

      Description

      By calling chown as a secondary RUN command, you effectively double the size of image by creating a Docker layer of the same size as the extracted binary.

      See GitHub discussion: https://github.com/apache/nifi/pull/1372#issuecomment-307592287

      Expectation

      The resultant Docker image should be no larger than the Base image + the size required by extracting the Nifi binaries.

        Issue Links

          Activity

          Hide
          taftster Adam Taft added a comment -

          This is a good catch and will be super easy to fix.

          There are some other cleanup areas that might want to be considered in the same patch. I'm wondering about:

          • GID=50 already exists in openjdk:8 (staff). Why are we attempting to use and rename an existing group? Just create a new group and be done with it?
          • Consolidate the "Setup NiFi user" into a single layer. Combine any groupadd, useradd, mkdir, chown statements into a single RUN layer.
          • Update the deprecated MAINTAINER statement with "LABEL maintainer"
          Show
          taftster Adam Taft added a comment - This is a good catch and will be super easy to fix. There are some other cleanup areas that might want to be considered in the same patch. I'm wondering about: GID=50 already exists in openjdk:8 (staff). Why are we attempting to use and rename an existing group? Just create a new group and be done with it? Consolidate the "Setup NiFi user" into a single layer. Combine any groupadd, useradd, mkdir, chown statements into a single RUN layer. Update the deprecated MAINTAINER statement with "LABEL maintainer"
          Hide
          cricket007 Jordan Moore added a comment -

          Just want to mention, I'm already working on this, and have a resultant 1.29 GB image.

          GID=50 already exists in openjdk:8 (staff). Why are we attempting to use and rename an existing group? Just create a new group and be done with it?

          Completely agree. Open to suggestions for alternative numbers.

          Consolidate the "Setup NiFi user" into a single layer. Combine any groupadd, useradd, mkdir, chown statements into a single RUN layer.

          Did that.

          Update the deprecated MAINTAINER statement with "LABEL maintainer"

          Forgot about that, but will fix.

          Also want to point out that openjdk:8 is the JDK, whereas you only need the JRE.

          Show
          cricket007 Jordan Moore added a comment - Just want to mention, I'm already working on this, and have a resultant 1.29 GB image. GID=50 already exists in openjdk:8 (staff). Why are we attempting to use and rename an existing group? Just create a new group and be done with it? Completely agree. Open to suggestions for alternative numbers. Consolidate the "Setup NiFi user" into a single layer. Combine any groupadd, useradd, mkdir, chown statements into a single RUN layer. Did that. Update the deprecated MAINTAINER statement with "LABEL maintainer" Forgot about that, but will fix. Also want to point out that openjdk:8 is the JDK, whereas you only need the JRE.
          Hide
          taftster Adam Taft added a comment -

          Yes, you're right. Using the JRE only version might be a good idea as well. Maybe use openjdk:jre-8 as the base image?

          Are there any JSPs etc. that are dependent on the JDK compiler? I don't think so, but Matt Gilman would know for sure

          Since we're assigning UID=1000, how about GID=1000 to follow tradition? In fact, I would just recommend we don't bother with the group creation at all, because calling useradd will create a group with the same GID as the UID by default.

          Show
          taftster Adam Taft added a comment - Yes, you're right. Using the JRE only version might be a good idea as well. Maybe use openjdk:jre-8 as the base image? Are there any JSPs etc. that are dependent on the JDK compiler? I don't think so, but Matt Gilman would know for sure Since we're assigning UID=1000, how about GID=1000 to follow tradition? In fact, I would just recommend we don't bother with the group creation at all, because calling useradd will create a group with the same GID as the UID by default.
          Hide
          taftster Adam Taft added a comment -

          OK, indeed. JSP compilation is being handled by the Eclipse compiler. This is mentioned in the NOTICE file. I'm thinking JRE will be fine. Let's try it out!

          Show
          taftster Adam Taft added a comment - OK, indeed. JSP compilation is being handled by the Eclipse compiler. This is mentioned in the NOTICE file. I'm thinking JRE will be fine. Let's try it out!
          Hide
          cricket007 Jordan Moore added a comment -

          Cool, yeah the page loads with the JRE, so I assumed it would be fine, though I haven't extensively tested it.

          Also, I've tweaked some other things with my changes, but I wasn't sure if they should be separate issues.

          • Add ability to specify Apache Mirror for downloading (since you're limited to a daily 5 GB from archive.apache.org)
          • Enable the templates directory to be a VOLUME so you can export/import them easier (also actually create the templates directory since it isn't there when you extract the binary)
          Show
          cricket007 Jordan Moore added a comment - Cool, yeah the page loads with the JRE, so I assumed it would be fine, though I haven't extensively tested it. Also, I've tweaked some other things with my changes, but I wasn't sure if they should be separate issues. Add ability to specify Apache Mirror for downloading (since you're limited to a daily 5 GB from archive.apache.org) Enable the templates directory to be a VOLUME so you can export/import them easier (also actually create the templates directory since it isn't there when you extract the binary)
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user cricket007 opened a pull request:

          https://github.com/apache/nifi/pull/1910

          NIFI-4057 Docker Image is twice as large as necessary

          Resulting Docker image in now 1.29 GB instead of 2.58 GB

          *Detailed changes*

          • Merging unnecessary layers
          • MAINTAINER is deprecated
          • Using JRE as base since JDK is not necessary
          • Remove GID as a build-arg since useradd adds a group id for us
          • Add ability to specify Apache mirror site to reduce load on Apache Archive
          • Create a define templates directory as a Docker volume

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [X] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [X] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [X] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [X] Have you written or updated unit tests to verify your changes?
          • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
          • [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
          • [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. For documentation related changes:
          • [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/cricket007/nifi NIFI-4057

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/1910.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1910


          commit e4fdd3194a31356cc6597be1515f5597b19beaff
          Author: Jordan Moore <moorej@avalonconsult.com>
          Date: 2017-06-11T19:22:51Z

          NIFI-4057 Docker Image is twice as large as necessary

          • Merging unnecessary layers
          • MAINTAINER is deprecated
          • Using JRE as base since JDK is not necessary
          • Remove GID as a build-arg since useradd adds a group id for us
          • Add ability to specify Apache mirror site to reduce load on Apache Archive
          • Create a define templates directory as a Docker volume

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user cricket007 opened a pull request: https://github.com/apache/nifi/pull/1910 NIFI-4057 Docker Image is twice as large as necessary Resulting Docker image in now 1.29 GB instead of 2.58 GB * Detailed changes * Merging unnecessary layers MAINTAINER is deprecated Using JRE as base since JDK is not necessary Remove GID as a build-arg since useradd adds a group id for us Add ability to specify Apache mirror site to reduce load on Apache Archive Create a define templates directory as a Docker volume — Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [X] Has your PR been rebased against the latest commit within the target branch (typically master)? [X] Is your initial contribution a single, squashed commit? For code changes: [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [X] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [ ] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cricket007/nifi NIFI-4057 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1910.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1910 commit e4fdd3194a31356cc6597be1515f5597b19beaff Author: Jordan Moore <moorej@avalonconsult.com> Date: 2017-06-11T19:22:51Z NIFI-4057 Docker Image is twice as large as necessary Merging unnecessary layers MAINTAINER is deprecated Using JRE as base since JDK is not necessary Remove GID as a build-arg since useradd adds a group id for us Add ability to specify Apache mirror site to reduce load on Apache Archive Create a define templates directory as a Docker volume
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121291104

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
              • End diff –

          Your groupadd command leaves off the "-g $GID" from the previous commit. Since $GID is being specified and passed in from DockerBuild.sh, it's probably best that we continue to create the group with the specified $GID. If it's a valid configuration option, it needs to be used as the group id here.

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121291104 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ End diff – Your groupadd command leaves off the "-g $GID" from the previous commit. Since $GID is being specified and passed in from DockerBuild.sh, it's probably best that we continue to create the group with the specified $GID. If it's a valid configuration option, it needs to be used as the group id here.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121291213

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
            + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi
          1. Download, validate, and expand Apache NiFi binary.
            RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
            + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            && tar xvzf $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \
          • && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz
            + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \
            + && chown -R nifi:nifi $NIFI_HOME

          -RUN chown -R nifi:nifi $NIFI_HOME
          +# Web HTTP Port & Remote Site-to-Site Ports
          +EXPOSE 8080 8181

            1. Web HTTP Port
              -EXPOSE 8080
              +WORKDIR $NIFI_HOME
            1. Remote Site-To-Site Port
              -EXPOSE 8181
              -
              -USER nifi
              +VOLUME conf/templates

          — End diff –

          There are lots of files in conf/ that would likely be good candidates for VOLUME export (flow.xml.gz, nifi.properties, bootstrap.conf, etc.). Maybe this should just be: "VOLUME conf"

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121291213 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ && tar xvzf $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \ && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \ + && chown -R nifi:nifi $NIFI_HOME -RUN chown -R nifi:nifi $NIFI_HOME +# Web HTTP Port & Remote Site-to-Site Ports +EXPOSE 8080 8181 Web HTTP Port -EXPOSE 8080 +WORKDIR $NIFI_HOME Remote Site-To-Site Port -EXPOSE 8181 - -USER nifi +VOLUME conf/templates — End diff – There are lots of files in conf/ that would likely be good candidates for VOLUME export (flow.xml.gz, nifi.properties, bootstrap.conf, etc.). Maybe this should just be: "VOLUME conf"
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121290876

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
            + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi
          1. Download, validate, and expand Apache NiFi binary.
            RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
            + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
              • End diff –

          Shouldn't this inner sha256 curl also use $MIRROR?

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121290876 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ End diff – Shouldn't this inner sha256 curl also use $MIRROR?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on the issue:

          https://github.com/apache/nifi/pull/1910

          Changes to this Dockerfile should also be reciprocally to ../dockermaven/Dockerfile

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on the issue: https://github.com/apache/nifi/pull/1910 Changes to this Dockerfile should also be reciprocally to ../dockermaven/Dockerfile
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121292271

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
            + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi
          1. Download, validate, and expand Apache NiFi binary.
            RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
            + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
              • End diff –

          The mirrors don't include the sha

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121292271 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ End diff – The mirrors don't include the sha
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121292308

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
              • End diff –

          From my understanding of the JIRA comments, the GID could be removed since useradd makes the group. And GID 50 already existed, anyway

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121292308 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ End diff – From my understanding of the JIRA comments, the GID could be removed since useradd makes the group. And GID 50 already existed, anyway
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121292333

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
            + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi
          1. Download, validate, and expand Apache NiFi binary.
            RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
            + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
              • End diff –

          Ah ha! OK then!

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121292333 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ End diff – Ah ha! OK then!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121292352

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
              • End diff –

          I'm just saying, if we're going to remove $GID, we should remove it entirely. But if we're going to support it (i.e. by having an option in the Dockerbuild.sh), then we should support it entirely. Line 31 currently ignores $GID.

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121292352 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ End diff – I'm just saying, if we're going to remove $GID, we should remove it entirely. But if we're going to support it (i.e. by having an option in the Dockerbuild.sh), then we should support it entirely. Line 31 currently ignores $GID.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r121296054

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,37 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \
            + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi
          1. Download, validate, and expand Apache NiFi binary.
            RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
            + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
            && tar xvzf $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \
          • && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz
            + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \
            + && chown -R nifi:nifi $NIFI_HOME

          -RUN chown -R nifi:nifi $NIFI_HOME
          +# Web HTTP Port & Remote Site-to-Site Ports
          +EXPOSE 8080 8181

            1. Web HTTP Port
              -EXPOSE 8080
              +WORKDIR $NIFI_HOME
            1. Remote Site-To-Site Port
              -EXPOSE 8181
              -
              -USER nifi
              +VOLUME conf/templates

          — End diff –

          Can that be an extended discussion elsewhere? While I might agree, at least for nifi.properties, the primary goal of this PR was to cut down the image size.

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r121296054 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,37 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd nifi && useradd nifi --shell /bin/bash -u $UID -m -g nifi \ + && mkdir -p $NIFI_HOME/conf/templates && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ && tar xvzf $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \ && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \ + && chown -R nifi:nifi $NIFI_HOME -RUN chown -R nifi:nifi $NIFI_HOME +# Web HTTP Port & Remote Site-to-Site Ports +EXPOSE 8080 8181 Web HTTP Port -EXPOSE 8080 +WORKDIR $NIFI_HOME Remote Site-To-Site Port -EXPOSE 8181 - -USER nifi +VOLUME conf/templates — End diff – Can that be an extended discussion elsewhere? While I might agree, at least for nifi.properties, the primary goal of this PR was to cut down the image size.
          Hide
          mcgilman Matt Gilman added a comment -

          Adam Taft That is accurate. Only a JRE is required. The bundle dependencies handle any necessary JSP compiling.

          Show
          mcgilman Matt Gilman added a comment - Adam Taft That is accurate. Only a JRE is required. The bundle dependencies handle any necessary JSP compiling.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on the issue:

          https://github.com/apache/nifi/pull/1910

          @taftster I updated the Maven version as well, and I believe it built successfully after updating the `pom.xml` version.

          Was there anything else needing to be addressed?

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on the issue: https://github.com/apache/nifi/pull/1910 @taftster I updated the Maven version as well, and I believe it built successfully after updating the `pom.xml` version. Was there anything else needing to be addressed?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r123647376

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,38 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates \
              • End diff –

          Is this 'mkdir' leftover from the previous pull request? Specifically, didn't we say we'd address the exposed VOLUME directories in another PR? It's not hurting anything, of course, but I don't think it was in the original Dockerfile either.

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r123647376 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,38 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates \ End diff – Is this 'mkdir' leftover from the previous pull request? Specifically, didn't we say we'd address the exposed VOLUME directories in another PR? It's not hurting anything, of course, but I don't think it was in the original Dockerfile either.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r123647502

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,38 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates \
            + && chown -R nifi:nifi $NIFI_BASE_DIR
            1. Download, validate, and expand Apache NiFi binary.
              RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
          • && tar xvzf $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \
          • && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz
            -
            -RUN chown -R nifi:nifi $NIFI_HOME
            +USER nifi
            1. Web HTTP Port
              -EXPOSE 8080
              +# Download, validate, and expand Apache NiFi binary.
              +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
              + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
              + && tar xvzf $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \
              + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \
              + && chown -R nifi:nifi $NIFI_HOME
              • End diff –

          The chown here shouldn't be necessary? Since NIFI_BASE_DIR has already been chowned to the NIFI user, which we're using for the curl statement. Is that right?

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r123647502 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,38 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates \ + && chown -R nifi:nifi $NIFI_BASE_DIR Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ && tar xvzf $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \ && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz - -RUN chown -R nifi:nifi $NIFI_HOME +USER nifi Web HTTP Port -EXPOSE 8080 +# Download, validate, and expand Apache NiFi binary. +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ + && tar xvzf $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \ + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \ + && chown -R nifi:nifi $NIFI_HOME End diff – The chown here shouldn't be necessary? Since NIFI_BASE_DIR has already been chowned to the NIFI user, which we're using for the curl statement. Is that right?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r123796605

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,38 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates \
              • End diff –

          The `conf/templates` directory is not part of the binary, so as a result, I don't think you can save templates as part of running the Docker container.

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r123796605 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,38 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates \ End diff – The `conf/templates` directory is not part of the binary, so as a result, I don't think you can save templates as part of running the Docker container.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r123797661

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,38 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates \
            + && chown -R nifi:nifi $NIFI_BASE_DIR
            1. Download, validate, and expand Apache NiFi binary.
              RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
          • && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
          • && tar xvzf $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \
          • && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz
            -
            -RUN chown -R nifi:nifi $NIFI_HOME
            +USER nifi
            1. Web HTTP Port
              -EXPOSE 8080
              +# Download, validate, and expand Apache NiFi binary.
              +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz \
              + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \
              + && tar xvzf $NIFI_BASE_DIR/nifi$NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \
              + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \
              + && chown -R nifi:nifi $NIFI_HOME
              • End diff –

          Shouldn't be necessary, no. Doesn't seem to be hurting anything, though.
          I think I did it because I didn't trust the previous `chown -R` to hold after the `curl`, but since using `USER` beforehand, that does make it redundant.

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r123797661 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,38 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates \ + && chown -R nifi:nifi $NIFI_BASE_DIR Download, validate, and expand Apache NiFi binary. RUN curl -fSL $NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ && echo "$(curl $NIFI_BINARY_URL.sha256) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ && tar xvzf $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \ && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz - -RUN chown -R nifi:nifi $NIFI_HOME +USER nifi Web HTTP Port -EXPOSE 8080 +# Download, validate, and expand Apache NiFi binary. +RUN curl fSL $MIRROR/$NIFI_BINARY_URL -o $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz \ + && echo "$(curl https://archive.apache.org/dist/$NIFI_BINARY_URL.sha256 ) *$NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz" | sha256sum -c - \ + && tar xvzf $NIFI_BASE_DIR/nifi $NIFI_VERSION-bin.tar.gz -C $NIFI_BASE_DIR \ + && rm $NIFI_BASE_DIR/nifi-$NIFI_VERSION-bin.tar.gz \ + && chown -R nifi:nifi $NIFI_HOME End diff – Shouldn't be necessary, no. Doesn't seem to be hurting anything, though. I think I did it because I didn't trust the previous `chown -R` to hold after the `curl`, but since using `USER` beforehand, that does make it redundant.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user taftster commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r123810933

          — Diff: nifi-docker/dockerhub/Dockerfile —
          @@ -16,37 +16,38 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION=1.4.0
          +ARG MIRROR=https://archive.apache.org/dist

          -ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi$NIFI_VERSION
          -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz
          +ENV NIFI_BASE_DIR /opt/nifi
          +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \
          + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates \
              • End diff –

          OK, wow. Yeah, that would be a problem. Good catch! Definitely need this then.

          Show
          githubbot ASF GitHub Bot added a comment - Github user taftster commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r123810933 — Diff: nifi-docker/dockerhub/Dockerfile — @@ -16,37 +16,38 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION=1.4.0 +ARG MIRROR= https://archive.apache.org/dist -ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi $NIFI_VERSION -ENV NIFI_BINARY_URL https://archive.apache.org/dist/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz +ENV NIFI_BASE_DIR /opt/nifi +ENV NIFI_HOME=$NIFI_BASE_DIR/nifi-$NIFI_VERSION \ + NIFI_BINARY_URL=/nifi/$NIFI_VERSION/nifi-$NIFI_VERSION-bin.tar.gz Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates \ End diff – OK, wow. Yeah, that would be a problem. Good catch! Definitely need this then.
          Hide
          taftster Adam Taft added a comment -

          Aldrin Piri Can you please look this over and target this for 1.4.0? Since you've invested a lot in the NIFI docker image, I was hoping you could signoff on these changes? This will reduce the docker image size by half. We should get this in for the next minor release, if it checks out.

          Show
          taftster Adam Taft added a comment - Aldrin Piri Can you please look this over and target this for 1.4.0? Since you've invested a lot in the NIFI docker image, I was hoping you could signoff on these changes? This will reduce the docker image size by half. We should get this in for the next minor release, if it checks out.
          Hide
          aldrin Aldrin Piri added a comment -

          Adam Taft certainly. currently on vacation but will put it on my list of items when I return next week.

          Show
          aldrin Aldrin Piri added a comment - Adam Taft certainly. currently on vacation but will put it on my list of items when I return next week.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on the issue:

          https://github.com/apache/nifi/pull/1910

          reviewing

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1910 reviewing
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r127817762

          — Diff: nifi-docker/dockermaven/Dockerfile —
          @@ -16,32 +16,33 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION
          ARG NIFI_BINARY

          ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates
              • End diff –

          This needs a '\'

          Everything else looks pretty good and just verifying successful build. If so, I can adjust this on merge.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r127817762 — Diff: nifi-docker/dockermaven/Dockerfile — @@ -16,32 +16,33 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION ARG NIFI_BINARY ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates End diff – This needs a '\' Everything else looks pretty good and just verifying successful build. If so, I can adjust this on merge.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r127823749

          — Diff: nifi-docker/dockermaven/Dockerfile —
          @@ -16,32 +16,33 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION
          ARG NIFI_BINARY

          ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates
            + && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi

          ADD $NIFI_BINARY $NIFI_BASE_DIR
          RUN chown -R nifi:nifi $NIFI_HOME
          — End diff –

          Looks like the chown was an issue after user. Moving USER below the chown seems to work appropriately.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r127823749 — Diff: nifi-docker/dockermaven/Dockerfile — @@ -16,32 +16,33 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION ARG NIFI_BINARY ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates + && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi ADD $NIFI_BINARY $NIFI_BASE_DIR RUN chown -R nifi:nifi $NIFI_HOME — End diff – Looks like the chown was an issue after user. Moving USER below the chown seems to work appropriately.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r127824242

          — Diff: nifi-docker/dockermaven/Dockerfile —
          @@ -16,32 +16,33 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION
          ARG NIFI_BINARY

          ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates
            + && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi

          ADD $NIFI_BINARY $NIFI_BASE_DIR
          RUN chown -R nifi:nifi $NIFI_HOME
          — End diff –

          But this causes the duplicate layer issue again. Bit of a different environment as we are not able to add & chmod the files in the same sequence given the nature of the ADD command. May just have to be a concession we make for the local environment.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r127824242 — Diff: nifi-docker/dockermaven/Dockerfile — @@ -16,32 +16,33 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION ARG NIFI_BINARY ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates + && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi ADD $NIFI_BINARY $NIFI_BASE_DIR RUN chown -R nifi:nifi $NIFI_HOME — End diff – But this causes the duplicate layer issue again. Bit of a different environment as we are not able to add & chmod the files in the same sequence given the nature of the ADD command. May just have to be a concession we make for the local environment.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1910#discussion_r127826350

          — Diff: nifi-docker/dockermaven/Dockerfile —
          @@ -16,32 +16,33 @@

          1. under the License.
            #

          -FROM openjdk:8
          -MAINTAINER Apache NiFi <dev@nifi.apache.org>
          +FROM openjdk:8-jre
          +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"

          ARG UID=1000
          -ARG GID=50
          +ARG GID=1000
          ARG NIFI_VERSION
          ARG NIFI_BINARY

          ENV NIFI_BASE_DIR /opt/nifi
          ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION

          1. Setup NiFi user
            -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1`
            -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi
            -RUN mkdir -p $NIFI_HOME
            +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \
            + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \
            + && mkdir -p $NIFI_HOME/conf/templates
            + && chown -R nifi:nifi $NIFI_BASE_DIR
            +
            +USER nifi

          ADD $NIFI_BINARY $NIFI_BASE_DIR
          RUN chown -R nifi:nifi $NIFI_HOME
          — End diff –

          Going to merge this in as I don't believe we have a better way to work around this for the local case and presumably there is not another way without getting overly complex.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi/pull/1910#discussion_r127826350 — Diff: nifi-docker/dockermaven/Dockerfile — @@ -16,32 +16,33 @@ under the License. # -FROM openjdk:8 -MAINTAINER Apache NiFi <dev@nifi.apache.org> +FROM openjdk:8-jre +LABEL maintainer "Apache NiFi <dev@nifi.apache.org>" ARG UID=1000 -ARG GID=50 +ARG GID=1000 ARG NIFI_VERSION ARG NIFI_BINARY ENV NIFI_BASE_DIR /opt/nifi ENV NIFI_HOME $NIFI_BASE_DIR/nifi-$NIFI_VERSION Setup NiFi user -RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` -RUN useradd --shell /bin/bash -u $UID -g $GID -m nifi -RUN mkdir -p $NIFI_HOME +RUN groupadd -g $GID nifi || groupmod -n nifi `getent group $GID | cut -d: -f1` \ + && useradd --shell /bin/bash -u $UID -g $GID -m nifi \ + && mkdir -p $NIFI_HOME/conf/templates + && chown -R nifi:nifi $NIFI_BASE_DIR + +USER nifi ADD $NIFI_BINARY $NIFI_BASE_DIR RUN chown -R nifi:nifi $NIFI_HOME — End diff – Going to merge this in as I don't believe we have a better way to work around this for the local case and presumably there is not another way without getting overly complex.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 3da8b94dddc3b08ecbf10f368240dd1b3e992bbf in nifi's branch refs/heads/master from Jordan Moore
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3da8b94 ]

          NIFI-4057 Docker Image is twice as large as necessary

          • Merging unnecessary layers
          • MAINTAINER is deprecated
          • Using JRE as base since JDK is not necessary
          • Set GID=1000 since openjdk image already defines 50
          • Add ability to specify Apache mirror site to reduce load on Apache Archive
          • Created templates directory since this is not included in the binary

          This closes #1910.

          Signed-off-by: Aldrin Piri <aldrin@apache.org>

          Show
          jira-bot ASF subversion and git services added a comment - Commit 3da8b94dddc3b08ecbf10f368240dd1b3e992bbf in nifi's branch refs/heads/master from Jordan Moore [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3da8b94 ] NIFI-4057 Docker Image is twice as large as necessary Merging unnecessary layers MAINTAINER is deprecated Using JRE as base since JDK is not necessary Set GID=1000 since openjdk image already defines 50 Add ability to specify Apache mirror site to reduce load on Apache Archive Created templates directory since this is not included in the binary This closes #1910. Signed-off-by: Aldrin Piri <aldrin@apache.org>
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/1910

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1910
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on the issue:

          https://github.com/apache/nifi/pull/1910

          Is there anyway to re-publish the 1.2.0 and 1.3.0 images so that those aren't as large?

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on the issue: https://github.com/apache/nifi/pull/1910 Is there anyway to re-publish the 1.2.0 and 1.3.0 images so that those aren't as large?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on the issue:

          https://github.com/apache/nifi/pull/1910

          @cricket007 we cannot given ASF guidelines unless we do another release. If there is a patch release that is created, we can certainly feed this in as part of that effort.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1910 @cricket007 we cannot given ASF guidelines unless we do another release. If there is a patch release that is created, we can certainly feed this in as part of that effort.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on the issue:

          https://github.com/apache/nifi/pull/1910

          @apiri I understand that you cannot push code changes for a release < 1.4.0, but this isn't changing source code, only pushing a different Dockerfile to DockerHub.

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on the issue: https://github.com/apache/nifi/pull/1910 @apiri I understand that you cannot push code changes for a release < 1.4.0, but this isn't changing source code, only pushing a different Dockerfile to DockerHub.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on the issue:

          https://github.com/apache/nifi/pull/1910

          @cricket007 The Dockerfiles are part of the entire NiFi codebase. There has been some discussion about breaking the Docker components into their own repository which would free us from this situation and allow us to release more frequently/independently. However, in the current state, another release would be needed as the tagged release triggers Docker Hub to build.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1910 @cricket007 The Dockerfiles are part of the entire NiFi codebase. There has been some discussion about breaking the Docker components into their own repository which would free us from this situation and allow us to release more frequently/independently. However, in the current state, another release would be needed as the tagged release triggers Docker Hub to build.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cricket007 commented on the issue:

          https://github.com/apache/nifi/pull/1910

          I understand the release tagging and automated build hooks of DockerHub, but I don't see why whoever has permission to push to DockerHub cannot manually overwrite the current "bloated" images.

          Show
          githubbot ASF GitHub Bot added a comment - Github user cricket007 commented on the issue: https://github.com/apache/nifi/pull/1910 I understand the release tagging and automated build hooks of DockerHub, but I don't see why whoever has permission to push to DockerHub cannot manually overwrite the current "bloated" images.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user apiri commented on the issue:

          https://github.com/apache/nifi/pull/1910

          Those images are created automatically from release tags (e.g. rel/nifi-1.2.0) on the source code repository. Those tags are protected and immutable on the repository; once they are created they cannot be overwritten whatsoever. It is this standard that prevents us from replacing the images. As a project community, we have no permissions to directly interact with Apache Docker Hub account which is managed by the Infra team.

          As mentioned, if a patch version of either those comes out, we can port these changes to that release branch as well. If we were to break docker components out into their own repo, we could overwrite those tags on Docker Hub. Unfortunately, until the source repo of the Dockerfile changes, those versions in Docker Hub are locked to the associated commits that generated them.

          Show
          githubbot ASF GitHub Bot added a comment - Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1910 Those images are created automatically from release tags (e.g. rel/nifi-1.2.0) on the source code repository. Those tags are protected and immutable on the repository; once they are created they cannot be overwritten whatsoever. It is this standard that prevents us from replacing the images. As a project community, we have no permissions to directly interact with Apache Docker Hub account which is managed by the Infra team. As mentioned, if a patch version of either those comes out, we can port these changes to that release branch as well. If we were to break docker components out into their own repo, we could overwrite those tags on Docker Hub. Unfortunately, until the source repo of the Dockerfile changes, those versions in Docker Hub are locked to the associated commits that generated them.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jimzucker commented on the issue:

          https://github.com/apache/nifi/pull/1910

          When can we get a 1.3.1 with this fix, happy to help if that is needed but the current image is terribly large.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jimzucker commented on the issue: https://github.com/apache/nifi/pull/1910 When can we get a 1.3.1 with this fix, happy to help if that is needed but the current image is terribly large.

            People

            • Assignee:
              cricket007 Jordan Moore
              Reporter:
              cricket007 Jordan Moore
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development