Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1168

hue build fails (unresolvable maven dependencies)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:

      Description

      (i may be doing somethin' wrong) building current master and facing this problem:

      http://grokbase.com/t/cloudera/hue-user/13bnfjzm9g/compilation-error-due-to-unresolved-pom-dependency-for-hue-3-0-0-and-hue-2-5-1-in-mac

      Git-cloned hue 2.5.1 from cloudera repo - doesn't build either. Error:

      The project com.cloudera.hue:hue-parent:2.5.1-SNAPSHOT (/projects/cloudera/hue/maven/pom.xml) has 1 error
      [ERROR] Non-resolvable parent POM: Failure to find com.cloudera:parent:pom:1.0-SNAPSHOT in https://repository.cloudera.com/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of cloudera.snapshots.repo has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 22, column 11 -> [Help 2]

      The recommended fix by the hue guy was to remove parent section from the pom. So in case others observe the same, i'm attaching small patch with oneliner sed to make build pass.

      1. BIGTOP-1168.1.patch
        1 kB
        Mikhail Antonov
      2. BIGTOP-1168.2.patch
        0.7 kB
        Mikhail Antonov

        Activity

        Hide
        mantonov Mikhail Antonov added a comment -

        first version, simple sed added to do-comp-build to overcome build error

        Show
        mantonov Mikhail Antonov added a comment - first version, simple sed added to do-comp-build to overcome build error
        Hide
        cos Konstantin Boudnik added a comment -

        Please do not assign hue bug to me - I am not doing anything with Cloudera software.
        A general rule: if you have a patch for a problem - assign it to yourself.

        Show
        cos Konstantin Boudnik added a comment - Please do not assign hue bug to me - I am not doing anything with Cloudera software. A general rule: if you have a patch for a problem - assign it to yourself.
        Hide
        mantonov Mikhail Antonov added a comment -

        Ah, ok. Sure.

        Show
        mantonov Mikhail Antonov added a comment - Ah, ok. Sure.
        Hide
        romainr Romain Rigaux added a comment - - edited

        Something changed but in the repo but you should be able to fix it with this patch: https://github.com/cloudera/hue/commit/37f892c0d6c8da7f267c6614be592cc3e0f2d381 (pushed in Hue 3.5)

        We could do a 2.5.2 if it is blocking BigTop.

        Show
        romainr Romain Rigaux added a comment - - edited Something changed but in the repo but you should be able to fix it with this patch: https://github.com/cloudera/hue/commit/37f892c0d6c8da7f267c6614be592cc3e0f2d381 (pushed in Hue 3.5) We could do a 2.5.2 if it is blocking BigTop.
        Hide
        mantonov Mikhail Antonov added a comment -

        Romain - thanks for coming back quickly - yep, removing parent section from pom helps, I actually tried it with the attached stop-gap patch to Bigtop itself, which is sed one-liner, removing those lines, and it works.

        Definitely fixing it in Hue code is the right way of fixing that, I'd appreciate if you could do 2.5.2 with this fix.

        Show
        mantonov Mikhail Antonov added a comment - Romain - thanks for coming back quickly - yep, removing parent section from pom helps, I actually tried it with the attached stop-gap patch to Bigtop itself, which is sed one-liner, removing those lines, and it works. Definitely fixing it in Hue code is the right way of fixing that, I'd appreciate if you could do 2.5.2 with this fix.
        Hide
        mantonov Mikhail Antonov added a comment -

        If you do that, I'll do another patch for Bigtop, simply updating Hue version being used.

        Show
        mantonov Mikhail Antonov added a comment - If you do that, I'll do another patch for Bigtop, simply updating Hue version being used.
        Hide
        romainr Romain Rigaux added a comment -

        So we will do the Hue 2.5.2 before the end of the year.

        Show
        romainr Romain Rigaux added a comment - So we will do the Hue 2.5.2 before the end of the year.
        Hide
        mantonov Mikhail Antonov added a comment -

        Great! Thank you.

        Show
        mantonov Mikhail Antonov added a comment - Great! Thank you.
        Hide
        rvs Roman Shaposhnik added a comment -

        Romain, thanks for jumping on this so quickly – please let us know when the bugfix release is out.

        Show
        rvs Roman Shaposhnik added a comment - Romain, thanks for jumping on this so quickly – please let us know when the bugfix release is out.
        Hide
        romainr Romain Rigaux added a comment -

        Here is 2.5.2 with the pom fixed: https://github.com/cloudera/hue/tree/branch-2.5.2

        Let us know but it should work fine now!

        Show
        romainr Romain Rigaux added a comment - Here is 2.5.2 with the pom fixed: https://github.com/cloudera/hue/tree/branch-2.5.2 Let us know but it should work fine now!
        Hide
        mantonov Mikhail Antonov added a comment -

        Thanks for the fix! I'll update version of Hue used in Bigtop trunk and see how it works.

        Show
        mantonov Mikhail Antonov added a comment - Thanks for the fix! I'll update version of Hue used in Bigtop trunk and see how it works.
        Hide
        mantonov Mikhail Antonov added a comment -

        Looks good to me, thank you! Submitting bigtop patch.

        Show
        mantonov Mikhail Antonov added a comment - Looks good to me, thank you! Submitting bigtop patch.
        Hide
        mantonov Mikhail Antonov added a comment -

        second version, modified bigtop.mk, updated hue version

        Show
        mantonov Mikhail Antonov added a comment - second version, modified bigtop.mk, updated hue version
        Hide
        mantonov Mikhail Antonov added a comment -

        Upgraded Hue version to 2.5.2, cut off by Hue team with fix in maven pom. Please see second version of patch (leaving 1st version also just for reference).

        Show
        mantonov Mikhail Antonov added a comment - Upgraded Hue version to 2.5.2, cut off by Hue team with fix in maven pom. Please see second version of patch (leaving 1st version also just for reference).
        Hide
        plinnell Peter Linnell added a comment -

        +1 Thanks Romain for cutting the new release.

        Show
        plinnell Peter Linnell added a comment - +1 Thanks Romain for cutting the new release.
        Hide
        mantonov Mikhail Antonov added a comment - - edited

        Would appreciate if someone could take a look at patch (really trivial) and comment/commit.

        Show
        mantonov Mikhail Antonov added a comment - - edited Would appreciate if someone could take a look at patch (really trivial) and comment/commit.
        Hide
        bmahe Bruno Mahé added a comment -

        I am getting a build error on fedora 18:

        + cp /run/media/bruno/HDFS/freesoftware/bigtop-asf/build/hue//rpm//SOURCES/hue.init /run/media/bruno/HDFS/freesoftware/bigtop-asf/build/hue/rpm/BUILDROOT/hue-2.5.2-1.fc18.x86_64//etc/rc.d/init.d/hue
        + '[' '%{buildarch}' = noarch ']'
        + QA_CHECK_RPATHS=1
        + case "${QA_CHECK_RPATHS:-}" in
        + /usr/lib/rpm/check-rpaths
        *******************************************************************************
        *
        * WARNING: 'check-rpaths' detected a broken RPATH and will cause 'rpmbuild'
        *          to fail. To ignore these errors, you can set the '$QA_RPATHS'
        *          environment variable which is a bitmask allowing the values
        *          below. The current value of QA_RPATHS is 0x0000.
        *
        *    0x0001 ... standard RPATHs (e.g. /usr/lib); such RPATHs are a minor
        *               issue but are introducing redundant searchpaths without
        *               providing a benefit. They can also cause errors in multilib
        *               environments.
        *    0x0002 ... invalid RPATHs; these are RPATHs which are neither absolute
        *               nor relative filenames and can therefore be a SECURITY risk
        *    0x0004 ... insecure RPATHs; these are relative RPATHs which are a
        *               SECURITY risk
        *    0x0008 ... the special '$ORIGIN' RPATHs are appearing after other
        *               RPATHs; this is just a minor issue but usually unwanted
        *    0x0010 ... the RPATH is empty; there is no reason for such RPATHs
        *               and they cause unneeded work while loading libraries
        *    0x0020 ... an RPATH references '..' of an absolute path; this will break
        *               the functionality when the path before '..' is a symlink
        *          
        *
        * Examples:
        * - to ignore standard and empty RPATHs, execute 'rpmbuild' like
        *   $ QA_RPATHS=$[ 0x0001|0x0010 ] rpmbuild my-package.src.rpm
        * - to check existing files, set $RPM_BUILD_ROOT and execute check-rpaths like
        *   $ RPM_BUILD_ROOT=<top-dir> /usr/lib/rpm/check-rpaths
        *  
        *******************************************************************************
        ERROR   0002: file '/usr/lib/hue/build/env/lib/python2.7/site-packages/python_ldap-2.3.13-py2.7-linux-x86_64.egg/_ldap.so' contains an invalid rpath '/usr/local/openldap-2.3/lib' in [/usr/local/openldap-2.3/lib]
        error: Bad exit status from /var/tmp/rpm-tmp.K0sQj5 (%install)
        
        
        RPM build errors:
            Bad exit status from /var/tmp/rpm-tmp.K0sQj5 (%install)
        make: *** [/run/media/bruno/HDFS/freesoftware/bigtop-asf/build/hue//.rpm] Error 1
        
        Show
        bmahe Bruno Mahé added a comment - I am getting a build error on fedora 18: + cp /run/media/bruno/HDFS/freesoftware/bigtop-asf/build/hue//rpm//SOURCES/hue.init /run/media/bruno/HDFS/freesoftware/bigtop-asf/build/hue/rpm/BUILDROOT/hue-2.5.2-1.fc18.x86_64//etc/rc.d/init.d/hue + '[' '%{buildarch}' = noarch ']' + QA_CHECK_RPATHS=1 + case "${QA_CHECK_RPATHS:-}" in + /usr/lib/rpm/check-rpaths ******************************************************************************* * * WARNING: 'check-rpaths' detected a broken RPATH and will cause 'rpmbuild' * to fail. To ignore these errors, you can set the '$QA_RPATHS' * environment variable which is a bitmask allowing the values * below. The current value of QA_RPATHS is 0x0000. * * 0x0001 ... standard RPATHs (e.g. /usr/lib); such RPATHs are a minor * issue but are introducing redundant searchpaths without * providing a benefit. They can also cause errors in multilib * environments. * 0x0002 ... invalid RPATHs; these are RPATHs which are neither absolute * nor relative filenames and can therefore be a SECURITY risk * 0x0004 ... insecure RPATHs; these are relative RPATHs which are a * SECURITY risk * 0x0008 ... the special '$ORIGIN' RPATHs are appearing after other * RPATHs; this is just a minor issue but usually unwanted * 0x0010 ... the RPATH is empty; there is no reason for such RPATHs * and they cause unneeded work while loading libraries * 0x0020 ... an RPATH references '..' of an absolute path; this will break * the functionality when the path before '..' is a symlink * * * Examples: * - to ignore standard and empty RPATHs, execute 'rpmbuild' like * $ QA_RPATHS=$[ 0x0001|0x0010 ] rpmbuild my-package.src.rpm * - to check existing files, set $RPM_BUILD_ROOT and execute check-rpaths like * $ RPM_BUILD_ROOT=<top-dir> /usr/lib/rpm/check-rpaths * ******************************************************************************* ERROR 0002: file '/usr/lib/hue/build/env/lib/python2.7/site-packages/python_ldap-2.3.13-py2.7-linux-x86_64.egg/_ldap.so' contains an invalid rpath '/usr/local/openldap-2.3/lib' in [/usr/local/openldap-2.3/lib] error: Bad exit status from /var/tmp/rpm-tmp.K0sQj5 (%install) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.K0sQj5 (%install) make: *** [/run/media/bruno/HDFS/freesoftware/bigtop-asf/build/hue//.rpm] Error 1
        Hide
        bmahe Bruno Mahé added a comment -

        I maxed out my investigation time for today, but so far, I believe we should do something like:

        • pass --disable-rpath to the parameter extra_compile_args in ext-py/python-ldap-2.3.13/setup.cfg. Maybe we can do this when setup.py is being called
        • Initialize library_dirs to nothing in ext-py/python-ldap-2.3.13/setup.cfg
        Show
        bmahe Bruno Mahé added a comment - I maxed out my investigation time for today, but so far, I believe we should do something like: pass --disable-rpath to the parameter extra_compile_args in ext-py/python-ldap-2.3.13/setup.cfg. Maybe we can do this when setup.py is being called Initialize library_dirs to nothing in ext-py/python-ldap-2.3.13/setup.cfg
        Hide
        mantonov Mikhail Antonov added a comment -

        I think the last patch for this jira solves declared roadblock in hue build, but there are more of them ahead.

        -Sent from mobile, please excuse typos
        Mikhail

        Show
        mantonov Mikhail Antonov added a comment - I think the last patch for this jira solves declared roadblock in hue build, but there are more of them ahead. -Sent from mobile, please excuse typos Mikhail
        Hide
        mantonov Mikhail Antonov added a comment -

        Coping my email here:

        The info with rpm build failure looks unrelated to this particular jira (which is about wrong pom file), but may be similar to another jira I reported recently on hue rpm build is being broken (BIGTOP-1169). I had to temporarily comment out hue in bigtop.mk file locally to make full 'make realclean rpm' pass for me on Fedora 19 and Oracle JDK 1.6.45. I'm going to setup VM with FC 18 and try to see if I can get the same error.

        Show
        mantonov Mikhail Antonov added a comment - Coping my email here: The info with rpm build failure looks unrelated to this particular jira (which is about wrong pom file), but may be similar to another jira I reported recently on hue rpm build is being broken ( BIGTOP-1169 ). I had to temporarily comment out hue in bigtop.mk file locally to make full 'make realclean rpm' pass for me on Fedora 19 and Oracle JDK 1.6.45. I'm going to setup VM with FC 18 and try to see if I can get the same error.
        Hide
        bmahe Bruno Mahé added a comment -

        I was worried we would have to ask Hue to roll out a new version to fix this issue since we cannot patch Hue itself and the issue I got didn't seem to be fixable from outside of Hue (except forcing the build to ignore rpath issues, but that's wrong).

        I don't think we want to support Fedora 18 when 20 has already been out for a bit. I should upgrade first and you shouldn't have to test on an old Fedora.

        So let's do the following:

        • I will commit this patch right now and close this issue.
        • Since it works for you on later Fedora, I will upgrade my machine (it was overdue, just did not have much spare time)
        • We will keep an eye on jenkins to ensure everything is fine

        +1 to this patch

        Show
        bmahe Bruno Mahé added a comment - I was worried we would have to ask Hue to roll out a new version to fix this issue since we cannot patch Hue itself and the issue I got didn't seem to be fixable from outside of Hue (except forcing the build to ignore rpath issues, but that's wrong). I don't think we want to support Fedora 18 when 20 has already been out for a bit. I should upgrade first and you shouldn't have to test on an old Fedora. So let's do the following: I will commit this patch right now and close this issue. Since it works for you on later Fedora, I will upgrade my machine (it was overdue, just did not have much spare time) We will keep an eye on jenkins to ensure everything is fine +1 to this patch
        Hide
        bmahe Bruno Mahé added a comment -

        Note on patch formats:
        Could you use the following format:
        BIGTOP-1168. <ONE_LINER_DESCRIPTION>
        <EVERYTHING AFTER IS UP TO YOU>

        For instance:

        commit 715f0bf4fdb519517c62bfd659e4401ad7278a11
        Author: Sean Mackrory <mackrorysd@apache.org>
        Date:   Mon Dec 23 11:50:01 2013 -0700
        
            BIGTOP-1170. Add an easier tool for uploading / creating the sharelib
        

        It makes it easier to parse commit logs and associate a ticket.
        It's not a big issue, but it would be really nice to follow it for future patches.

        Show
        bmahe Bruno Mahé added a comment - Note on patch formats: Could you use the following format: BIGTOP-1168 . <ONE_LINER_DESCRIPTION> <EVERYTHING AFTER IS UP TO YOU> For instance: commit 715f0bf4fdb519517c62bfd659e4401ad7278a11 Author: Sean Mackrory <mackrorysd@apache.org> Date: Mon Dec 23 11:50:01 2013 -0700 BIGTOP-1170. Add an easier tool for uploading / creating the sharelib It makes it easier to parse commit logs and associate a ticket. It's not a big issue, but it would be really nice to follow it for future patches.
        Hide
        mantonov Mikhail Antonov added a comment -

        Sure, I will definitely follow that in future, thanks for the note.

        Show
        mantonov Mikhail Antonov added a comment - Sure, I will definitely follow that in future, thanks for the note.
        Hide
        mantonov Mikhail Antonov added a comment -

        last version of patch has been committed on master

        Show
        mantonov Mikhail Antonov added a comment - last version of patch has been committed on master
        Hide
        plinnell Peter Linnell added a comment -

        tested, it is fixed

        Show
        plinnell Peter Linnell added a comment - tested, it is fixed

          People

          • Assignee:
            mantonov Mikhail Antonov
            Reporter:
            mantonov Mikhail Antonov
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development