Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3866

HttpFS POM should have property where to download tomcat from

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: build
    • Labels:
      None
    • Environment:

      CDH4 build on CentOS 6.2

    • Hadoop Flags:
      Reviewed

      Description

      When trying to enable a build of CDH4 in Jenkins, I got a build error due to an attempt to download Tomcat from the internet directly instead of via Maven and thus our internal Maven repository.

      The problem is due to this line in src/hadoop-hdfs-project/hadoop-hdfs-httpfs/target/antrun/build-main.xml:
      <get dest="downloads/tomcat.tar.gz" skipexisting="true" verbose="true" src="http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.32/bin/apache-tomcat-6.0.32.tar.gz"/>

      This build.xml is generated from src/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml:
      <get src="http://archive.apache.org/dist/tomcat/tomcat-6/v$

      {tomcat.version}/bin/apache-tomcat-${tomcat.version}

      .tar.gz" dest="downloads/tomcat.tar.gz" verbose="true" skipexisting="true"/>

      Instead of directly downloading from a hardcoded location, the Tomcat dependency should be managed by Maven. This would enable the use of a local repository for build machines without internet access.

      1. HDFS-3866.patch
        1 kB
        Plamen Jeliazkov

        Issue Links

          Activity

          Ryan Hennig created issue -
          Ryan Hennig made changes -
          Field Original Value New Value
          Link This issue relates to HDFS-3135 [ HDFS-3135 ]
          Hide
          Ryan Hennig added a comment -

          HDFS-3135 is a similar problem, but with a different user need and a different proposed solution.

          Show
          Ryan Hennig added a comment - HDFS-3135 is a similar problem, but with a different user need and a different proposed solution.
          Hide
          Alejandro Abdelnur added a comment -

          HttpFS uses a Tomcat server to run the service, not just Tomcat JARs. A Tomcat server is much more than just the JARs (available in Maven), it has a special layout, scripts and configuration files. Tomcat server TARBALLs are not available in Maven.

          To help build this in-house without downloading Tomcat TARBALL from the internet you could modify the property that sets the download URL to an internal web server where you stage the Tomcat TARBALL.

          One thing we could do as part of this JIRA is to make the download location to be a POM property so you can easily override it with -D or edit in the properties section.

          Show
          Alejandro Abdelnur added a comment - HttpFS uses a Tomcat server to run the service, not just Tomcat JARs. A Tomcat server is much more than just the JARs (available in Maven), it has a special layout, scripts and configuration files. Tomcat server TARBALLs are not available in Maven. To help build this in-house without downloading Tomcat TARBALL from the internet you could modify the property that sets the download URL to an internal web server where you stage the Tomcat TARBALL. One thing we could do as part of this JIRA is to make the download location to be a POM property so you can easily override it with -D or edit in the properties section.
          Plamen Jeliazkov made changes -
          Assignee Plamen Jeliazkov [ zero45 ]
          Plamen Jeliazkov made changes -
          Attachment HDFS-3866.patch [ 12543751 ]
          Hide
          Plamen Jeliazkov added a comment -

          Taking Alejandro's advice here by making it a POM property for easy overriding with -D or editing.

          Show
          Plamen Jeliazkov added a comment - Taking Alejandro's advice here by making it a POM property for easy overriding with -D or editing.
          Plamen Jeliazkov made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Alejandro Abdelnur added a comment -

          +1

          Show
          Alejandro Abdelnur added a comment - +1
          Hide
          Alejandro Abdelnur added a comment -

          changing summary of JIRA to reflect what the patch does.

          Show
          Alejandro Abdelnur added a comment - changing summary of JIRA to reflect what the patch does.
          Alejandro Abdelnur made changes -
          Summary HttpFS build should download Tomcat via Maven instead of directly HttpFS POM should have property where to download tomcat from
          Alejandro Abdelnur made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          Hide
          Alejandro Abdelnur added a comment -

          Thanks Plamen. Committed to trunk and branch-2.

          Show
          Alejandro Abdelnur added a comment - Thanks Plamen. Committed to trunk and branch-2.
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.2.0-alpha [ 12322472 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2677 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2677/)
          HDFS-3866. HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2677 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2677/ ) HDFS-3866 . HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2740 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2740/)
          HDFS-3866. HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2740 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2740/ ) HDFS-3866 . HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2703 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2703/)
          HDFS-3866. HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2703 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2703/ ) HDFS-3866 . HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1156 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1156/)
          HDFS-3866. HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1156 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1156/ ) HDFS-3866 . HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1187 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1187/)
          HDFS-3866. HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1187 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1187/ ) HDFS-3866 . HttpFS POM should have property where to download tomcat from (zero45 via tucu) (Revision 1380927) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380927 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Plamen Jeliazkov
              Reporter:
              Ryan Hennig
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development