Xerces-C++
  1. Xerces-C++
  2. XERCESC-1882

Xerces-C++ 3.0.1 fails to build with ICU 4.2

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.1
    • Fix Version/s: 3.1.0
    • Component/s: Build
    • Labels:
      None
    • Environment:
      Affects any platform with ICU 4.2, but I reproduced on OS X.

      Description

      Xerces-C++ 3.0.1 fails to build with ICU 4.2 because of a dependency on pkgdata's command line that apparently broke.

      The error is "pkgdata no longer supports -M option".

      make all-recursive
      make[1]: Entering directory `/var/tmp/portage/dev-libs/xerces-c-3.0.1/work/xerces-c-3.0.1'
      Making all in src
      make[2]: Entering directory `/var/tmp/portage/dev-libs/xerces-c-3.0.1/work/xerces-c-3.0.1/src'
      Making all in xercesc/util/MsgLoaders/ICU/resources
      make[3]: Entering directory `/var/tmp/portage/dev-libs/xerces-c-3.0.1/work/xerces-c-3.0.1/src/xercesc/util/MsgLoaders/ICU/resources'
      /usr/bin/genrb root.txt
      /usr/bin/pkgdata -name xercesc_messages_3_0 --mode dll -d . -M '"CC=x86_64-pc-linux-gnu-gcc" "CXX=x86_64-pc-linux-gnu-g++" "AR=x86_64-pc-linux-gnu-ar" "RANLIB=x86_64-pc-linux-gnu-ranlib" "CPPFLAGS=" "CFLAGS=-march=core2 -pipe -O2" "CXXFLAGS=-march=core2 -pipe -O2 " "LDFLAGS=-Wl,-O1,as-needed,gc-sections,hash-style=gnu,-sort-common"' ./res-file-list.txt
      /usr/bin/pkgdata: error in command line argument "-M"
      Run '/usr/bin/pkgdata --help' for help.
      make[3]: *** [xercesc_messages.lo] Error 1
      make[3]: Leaving directory `/var/tmp/portage/dev-libs/xerces-c-3.0.1/work/xerces-c-3.0.1/src/xercesc/util/MsgLoaders/ICU/resources'
      make[2]: *** [all-recursive] Error 1
      make[2]: Leaving directory `/var/tmp/portage/dev-libs/xerces-c-3.0.1/work/xerces-c-3.0.1/src'
      make[1]: *** [all-recursive] Error 1
      make[1]: Leaving directory `/var/tmp/portage/dev-libs/xerces-c-3.0.1/work/xerces-c-3.0.1'
      make: *** [all] Error 2

      1. pkgdata.patch
        5 kB
        Jay Berkenbilt

        Activity

        Hide
        Steven R. Loomis added a comment -

        Boris,
        Your current patch calls gencmn but it doesn't use the correct libraries.
        It's better to just use pkgdata directly.

        You can call gencmn with the output of: icu-config --invoke=gencmn
        or, use the $(INVOKE) variable in Makefile.inc

        Show
        Steven R. Loomis added a comment - Boris, Your current patch calls gencmn but it doesn't use the correct libraries. It's better to just use pkgdata directly. You can call gencmn with the output of: icu-config --invoke=gencmn or, use the $(INVOKE) variable in Makefile.inc
        Hide
        Boris Kolpackov added a comment -

        Ok, I have completely redone building of the resources for the ICU message loader. Instead of using pkgdata and then snatching some of its by-products (object files), we now call genccode and gencmn to produce a pair of C files which are then compiled normally by automake. I tested this with ICU 3.8.1, 4.0.0, and 4.2.1 and everything works as expected. Let's hope ICU folks won't deprecate one of these tools in the next release...

        Show
        Boris Kolpackov added a comment - Ok, I have completely redone building of the resources for the ICU message loader. Instead of using pkgdata and then snatching some of its by-products (object files), we now call genccode and gencmn to produce a pair of C files which are then compiled normally by automake. I tested this with ICU 3.8.1, 4.0.0, and 4.2.1 and everything works as expected. Let's hope ICU folks won't deprecate one of these tools in the next release...
        Hide
        Bartosz Szreder added a comment -

        You might be interested in patch posted at already mentioned Gentoo's bugzilla:
        https://bugs.gentoo.org/show_bug.cgi?id=269659

        Show
        Bartosz Szreder added a comment - You might be interested in patch posted at already mentioned Gentoo's bugzilla: https://bugs.gentoo.org/show_bug.cgi?id=269659
        Hide
        Boris Kolpackov added a comment -

        Guys, thanks for your comments. I am planning to try to fix this some time before the 3.1.0 release. But if someone wants to investigate it further, that would be great.

        Show
        Boris Kolpackov added a comment - Guys, thanks for your comments. I am planning to try to fix this some time before the 3.1.0 release. But if someone wants to investigate it further, that would be great.
        Hide
        Scott Cantor added a comment -

        Nothing's changed as of ICU 4.3.1. I think you were confused by the fact that there are two build options related to ICU. The transcoder part builds. The msgloader is the one that doesn't.

        From my testing, the transcoder is certainly more important, so I have a workaround there, but the build should still be fixed or the msgloader option removed or configure-tested to prevent use of newer ICU versions.

        Show
        Scott Cantor added a comment - Nothing's changed as of ICU 4.3.1. I think you were confused by the fact that there are two build options related to ICU. The transcoder part builds. The msgloader is the one that doesn't. From my testing, the transcoder is certainly more important, so I have a workaround there, but the build should still be fixed or the msgloader option removed or configure-tested to prevent use of newer ICU versions.
        Hide
        Scott Cantor added a comment -

        It's definitely not platform specific, anybody using 4.2, at least, would be affected because they simply changed the parameters of a command and the Xerces build is relying on one they removed. If your patch or their changes to 4.2.1 added back the option, that should fix it. If not, I don't see how it could.

        But if 4.2.1 is released, I will try and get the macport updated and see what results I get from it. If that fixes it, I'll certainly close the bug.

        The root of the problem is that the ICU build is critical for the Mac for apps that combine Xerces with Apache, and that remains Apple's fault. I may be able to elevate that problem with Apple at some point soon via a customer.

        Show
        Scott Cantor added a comment - It's definitely not platform specific, anybody using 4.2, at least, would be affected because they simply changed the parameters of a command and the Xerces build is relying on one they removed. If your patch or their changes to 4.2.1 added back the option, that should fix it. If not, I don't see how it could. But if 4.2.1 is released, I will try and get the macport updated and see what results I get from it. If that fixes it, I'll certainly close the bug. The root of the problem is that the ICU build is critical for the Mac for apps that combine Xerces with Apache, and that remains Apple's fault. I may be able to elevate that problem with Apple at some point soon via a customer.
        Hide
        Jay Berkenbilt added a comment -

        Okay, maybe it is Linux related. Well, I'll just stop making comments about my comments since obviously my brain is too tired right now. But I did just test xerces-c-3.0.1 with icu-4.2.1 on debian today, and it builds fine for the debian packages.

        Show
        Jay Berkenbilt added a comment - Okay, maybe it is Linux related. Well, I'll just stop making comments about my comments since obviously my brain is too tired right now. But I did just test xerces-c-3.0.1 with icu-4.2.1 on debian today, and it builds fine for the debian packages.
        Hide
        Jay Berkenbilt added a comment -

        Reading this report more carefully, I'm not sure whether my patch or comments will be at all relevant, but maybe they will help shed some light on pkgdata. It was extensively rewritten in 4.2. All I know is what I learned investigating the debian problem. Anyway, this bug seems not to be Linux related, so sorry if my comments are irrelevant.

        Show
        Jay Berkenbilt added a comment - Reading this report more carefully, I'm not sure whether my patch or comments will be at all relevant, but maybe they will help shed some light on pkgdata. It was extensively rewritten in 4.2. All I know is what I learned investigating the debian problem. Anyway, this bug seems not to be Linux related, so sorry if my comments are irrelevant.
        Hide
        Jay Berkenbilt added a comment -

        This patch should apply cleanly with -p1 to icu 4.2.1.

        Show
        Jay Berkenbilt added a comment - This patch should apply cleanly with -p1 to icu 4.2.1.
        Hide
        Jay Berkenbilt added a comment -

        I notice that the debian xerces-c-3.0.1 package builds fine with the debian icu 4.2.1 package (both of which I maintain). There were a number of problems with 4.2 and 4.2.1 with pkgdata, and I have a small patch that fixes the problems in debian. Upstream has accepted most or all of the patch, but not in time for 4.2.1. I'll attach the patch to this bug report. If you'd like, you can see whether building icu with this patch solves the problem.

        pkgdata creates .o files based on descriptions of data. There are at least two problems fixed by this patch. One is that it failed to run ranlib on libraries it was creating, which messed up some platforms. Another is that it sometimes decided that it could directly write a .o file even when it couldn't. See http://bugs.icu-project.org/trac/ticket/6969 for details.

        Show
        Jay Berkenbilt added a comment - I notice that the debian xerces-c-3.0.1 package builds fine with the debian icu 4.2.1 package (both of which I maintain). There were a number of problems with 4.2 and 4.2.1 with pkgdata, and I have a small patch that fixes the problems in debian. Upstream has accepted most or all of the patch, but not in time for 4.2.1. I'll attach the patch to this bug report. If you'd like, you can see whether building icu with this patch solves the problem. pkgdata creates .o files based on descriptions of data. There are at least two problems fixed by this patch. One is that it failed to run ranlib on libraries it was creating, which messed up some platforms. Another is that it sometimes decided that it could directly write a .o file even when it couldn't. See http://bugs.icu-project.org/trac/ticket/6969 for details.
        Hide
        Scott Cantor added a comment -

        Removing the -M option appears to work for static mode but fails for the dll/shared mode.

        I don't know enough about the icu package or what pkgdata does, but I'll hunt around a bit more.


        make all-recursive
        Making all in src
        Making all in xercesc/util/MsgLoaders/ICU/resources
        /opt/shibboleth-sp/bin/pkgdata --name xercesc_messages_3_0 --mode dll -d . ./res-file-list.txt
        ld: in ./, can't map file, errno=22
        collect2: ld returned 1 exit status
        Error generating assembly code for data.
        mv xercesc_messages_3_0_dat.o xercesc_messages_3_0_dat-pic.o
        /opt/shibboleth-sp/bin/pkgdata --name xercesc_messages_3_0 --mode static -d . ./res-file-list.txt
        ar: creating archive ./libxercesc_messages_3_0.a

        Show
        Scott Cantor added a comment - Removing the -M option appears to work for static mode but fails for the dll/shared mode. I don't know enough about the icu package or what pkgdata does, but I'll hunt around a bit more. — make all-recursive Making all in src Making all in xercesc/util/MsgLoaders/ICU/resources /opt/shibboleth-sp/bin/pkgdata --name xercesc_messages_3_0 --mode dll -d . ./res-file-list.txt ld: in ./, can't map file, errno=22 collect2: ld returned 1 exit status Error generating assembly code for data. mv xercesc_messages_3_0_dat.o xercesc_messages_3_0_dat-pic.o /opt/shibboleth-sp/bin/pkgdata --name xercesc_messages_3_0 --mode static -d . ./res-file-list.txt ar: creating archive ./libxercesc_messages_3_0.a
        Hide
        Scott Cantor added a comment -

        I will, but the indications in that bug were that doing that fails for non-static builds, which I took to mean Xerces as a whole, and I only build shared. I figured somebody who understood the ICU piece might understand the issue better than me.

        If it does work, I'm guessing a configure test will be needed to conditionally include the flag.

        Show
        Scott Cantor added a comment - I will, but the indications in that bug were that doing that fails for non-static builds, which I took to mean Xerces as a whole, and I only build shared. I figured somebody who understood the ICU piece might understand the issue better than me. If it does work, I'm guessing a configure test will be needed to conditionally include the flag.
        Hide
        James Berry added a comment -

        Ideas: see the information here:

        http://bugs.gentoo.org/show_bug.cgi?id=269659#c5

        It says there that the -M option to pkgdata was removed from ICU 4.2, because package data no longer needs to call make. So a fix in our case might just be to remove the -M from the call.

        Can you try that scott?

        Show
        James Berry added a comment - Ideas: see the information here: http://bugs.gentoo.org/show_bug.cgi?id=269659#c5 It says there that the -M option to pkgdata was removed from ICU 4.2, because package data no longer needs to call make. So a fix in our case might just be to remove the -M from the call. Can you try that scott?

          People

          • Assignee:
            Boris Kolpackov
            Reporter:
            Scott Cantor
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development