Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-4650

Add Hadoop HttpCore property independent of the HttpClient property

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: None
    • Labels:
      None

      Description

      Apache HttpComponents versions HttpClient and HttpCore independently (e.g., 4.5.2 and 4.4.4 used in Hadoop 2.8.x) so overriding hadoop.http.client.version with certain versions of HttpClient or HttpCore isn't currently possible. We should add a separate hadoop.http.core.version to pom.xml and nifi-nar-bundles/nifi-hadoop-libraries-bundle/pom.xml.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/2308

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2308
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit a458a139b31d8526690471aafdb582dcb1885217 in nifi's branch refs/heads/master from Joey Frazee
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a458a13 ]

          NIFI-4650 This closes #2308. Added independent hadoop.http.core.version property

          This closes #2308

          Signed-off-by: joewitt <joewitt@apache.org>

          Show
          jira-bot ASF subversion and git services added a comment - Commit a458a139b31d8526690471aafdb582dcb1885217 in nifi's branch refs/heads/master from Joey Frazee [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a458a13 ] NIFI-4650 This closes #2308. Added independent hadoop.http.core.version property This closes #2308 Signed-off-by: joewitt <joewitt@apache.org>
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user joewitt commented on the issue:

          https://github.com/apache/nifi/pull/2308

          after running ($ mvn clean install -Dhadoop.version=2.8.2 -Dhadoop.guava.version=12.0.1 -Dhadoop.http.client.version=4.5.2 -Dhadoop.http.core.version=4.4.4 -Pcontrib-check
          ) command i see

          httpclient-4.5.2.jar
          httpcore-4.4.4.jar
          okhttp-2.4.0.jar

          And before it looks like it always did

          Looks correct. +1 merging to master

          Show
          githubbot ASF GitHub Bot added a comment - Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2308 after running ($ mvn clean install -Dhadoop.version=2.8.2 -Dhadoop.guava.version=12.0.1 -Dhadoop.http.client.version=4.5.2 -Dhadoop.http.core.version=4.4.4 -Pcontrib-check ) command i see httpclient-4.5.2.jar httpcore-4.4.4.jar okhttp-2.4.0.jar And before it looks like it always did Looks correct. +1 merging to master
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on the issue:

          https://github.com/apache/nifi/pull/2308

          Yes, except I tried to override it with the example Joey has and the versions still come out the same as if I don’t override

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2308 Yes, except I tried to override it with the example Joey has and the versions still come out the same as if I don’t override
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user joewitt commented on the issue:

          https://github.com/apache/nifi/pull/2308

          that looks like the output of the default command run without overloading. Will try the command locally as well. That said, since this is only splitting out that property and letting someone choose to have more control still seems ok to allow through -no?

          Show
          githubbot ASF GitHub Bot added a comment - Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2308 that looks like the output of the default command run without overloading. Will try the command locally as well. That said, since this is only splitting out that property and letting someone choose to have more control still seems ok to allow through -no?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on the issue:

          https://github.com/apache/nifi/pull/2308

          I ran the second command above then checked the versions, got this:

          ```
          mvn dependency:list | grep http
          [INFO] org.apache.httpcomponents:httpcore:jar:4.2.5:compile
          [INFO] org.apache.httpcomponents:httpclient:jar:4.2.5:compile
          [INFO] commons-httpclient:commons-httpclient:jar:3.1:compile
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2308 I ran the second command above then checked the versions, got this: ``` mvn dependency:list | grep http [INFO] org.apache.httpcomponents:httpcore:jar:4.2.5:compile [INFO] org.apache.httpcomponents:httpclient:jar:4.2.5:compile [INFO] commons-httpclient:commons-httpclient:jar:3.1:compile ```
          Hide
          joewitt Joseph Witt added a comment -

          +1 will merge to master shortly - waiting for contrib check build to complete.

          Show
          joewitt Joseph Witt added a comment - +1 will merge to master shortly - waiting for contrib check build to complete.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jfrazee commented on the issue:

          https://github.com/apache/nifi/pull/2308

          FWIW, the tests pass on the following build as well:

          ```sh
          $ mvn clean install -Dhadoop.version=2.8.2 -Dhadoop.guava.version=12.0.1 -Dhadoop.http.client.version=4.5.2 -Dhadoop.http.core.version=4.4.4 -Pcontrib-check
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/2308 FWIW, the tests pass on the following build as well: ```sh $ mvn clean install -Dhadoop.version=2.8.2 -Dhadoop.guava.version=12.0.1 -Dhadoop.http.client.version=4.5.2 -Dhadoop.http.core.version=4.4.4 -Pcontrib-check ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user jfrazee opened a pull request:

          https://github.com/apache/nifi/pull/2308

          NIFI-4650 Added independent hadoop.http.core.version property

          For later releases of HttpComponents HttpClient and HttpCore versioning diverges so these need to be separate properties to support overriding them, e.g., for builds against Hadoop 2.8.x, etc.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/jfrazee/nifi NIFI-4650

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/2308.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2308


          commit cd5dd2d18af1e70b2ceca34035eccc6533fbd21d
          Author: Joey Frazee <jfrazee@apache.org>
          Date: 2017-11-30T16:59:19Z

          NIFI-4650 Added independent hadoop.http.core.version property


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/2308 NIFI-4650 Added independent hadoop.http.core.version property For later releases of HttpComponents HttpClient and HttpCore versioning diverges so these need to be separate properties to support overriding them, e.g., for builds against Hadoop 2.8.x, etc. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-4650 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2308.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2308 commit cd5dd2d18af1e70b2ceca34035eccc6533fbd21d Author: Joey Frazee <jfrazee@apache.org> Date: 2017-11-30T16:59:19Z NIFI-4650 Added independent hadoop.http.core.version property

            People

            • Assignee:
              jfrazee Joey Frazee
              Reporter:
              jfrazee Joey Frazee
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development