Thrift
  1. Thrift
  2. THRIFT-874

Script and document Windows build process

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Build Process, Deployment
    • Labels:
      None
    • Environment:

      Target: MinGW + MSYS on Windows XP

      Description

      1. Document how to build the Thrift code from source on a stock MinGW+MSYS install on Windows
      2. Investigate scripting the build as a contrib/ item.
      3. If #2 is successful, provide a way to generate a .MSI that installs the thrift compiler and runtime libraries.

        Issue Links

          Activity

          Michael Lum created issue -
          Roger Meier made changes -
          Link This issue is cloned as THRIFT-871 [ THRIFT-871 ]
          Hide
          Roger Meier added a comment -

          Great!

          Work's fine for Windows XP, built with mingw on Debian Lenny!

          it probably makes sense to strip the binary within the contrib/mingw-cross-compile.sh script by doing a

          i586-mingw32msvc-strip compiler/cpp/thrift.exe
          

          bevore make it available as download.

          Show
          Roger Meier added a comment - Great! Work's fine for Windows XP, built with mingw on Debian Lenny! it probably makes sense to strip the binary within the contrib/mingw-cross-compile.sh script by doing a i586-mingw32msvc-strip compiler/cpp/thrift.exe bevore make it available as download.
          Hide
          David Reiss added a comment -

          Ah, thanks. I was thinking to strip it, but I didn't know how.

          I think leaving this open for Windows users who want to hack on the code generator is a good idea.

          Show
          David Reiss added a comment - Ah, thanks. I was thinking to strip it, but I didn't know how. I think leaving this open for Windows users who want to hack on the code generator is a good idea.
          David Reiss made changes -
          Hide
          David Reiss added a comment -

          Hrm. It looks like I switched BUILD and HOST. After correcting them, the binary appears to be automatically stripped. Not sure what happened there. Can you try out this new patch? I put a binary up at http://people.apache.org/~dreiss/thrift-e0db3ee-r2.exe , and it appears to work under Wine.

          Show
          David Reiss added a comment - Hrm. It looks like I switched BUILD and HOST. After correcting them, the binary appears to be automatically stripped. Not sure what happened there. Can you try out this new patch? I put a binary up at http://people.apache.org/~dreiss/thrift-e0db3ee-r2.exe , and it appears to work under Wine.
          Hide
          Roger Meier added a comment -

          Thanks David!
          your thrift-e0db3ee-r2.exe version seems to be broken, just a 14KB file...

          I've attached my current version of mingw-cross-compile.sh. => everything is working fine on Windows XP.

          It should be sufficient to set the host type to i586-mingw32msvc
          build host, CC and CXX is auto detected if cross compiler is installed correctly.

          strip is usually done on make install , in that case we probably have to do it from mingw-cross-compile.sh script

          Show
          Roger Meier added a comment - Thanks David! your thrift-e0db3ee-r2.exe version seems to be broken, just a 14KB file... I've attached my current version of mingw-cross-compile.sh . => everything is working fine on Windows XP. It should be sufficient to set the host type to i586-mingw32msvc build host, CC and CXX is auto detected if cross compiler is installed correctly. strip is usually done on make install , in that case we probably have to do it from mingw-cross-compile.sh script
          Roger Meier made changes -
          Attachment mingw-cross-compile.sh [ 12453391 ]
          Hide
          David Reiss added a comment -

          Hey Roger, your version isn't working for me. Two questions. (1) Do you have Wine installed? (2) What does configure output for you after "checking whether we are cross compiling...". For me the answers are "yes" and "no", respectively.

          On closer inspection, I think what's happening to me is that compiler/cpp/thrift.exe is a libtool wrapper script for compiler/cpp/.libs/thrift.exe .

          It doesn't look like "make install" did a strip for me. I'm attaching a version that pulls thrift.exe directly out of libtool's .libs dir. Can you let me know if it works for you? If not, I'll probably have to write an even hackier version that grabs .libs/thrift.exe if present and thrift.exe if not.

          Show
          David Reiss added a comment - Hey Roger, your version isn't working for me. Two questions. (1) Do you have Wine installed? (2) What does configure output for you after "checking whether we are cross compiling...". For me the answers are "yes" and "no", respectively. On closer inspection, I think what's happening to me is that compiler/cpp/thrift.exe is a libtool wrapper script for compiler/cpp/.libs/thrift.exe . It doesn't look like "make install" did a strip for me. I'm attaching a version that pulls thrift.exe directly out of libtool's .libs dir. Can you let me know if it works for you? If not, I'll probably have to write an even hackier version that grabs .libs/thrift.exe if present and thrift.exe if not.
          David Reiss made changes -
          Hide
          Roger Meier added a comment -

          Hi David, to your questions:
          (1) Do you have Wine installed? Yes
          (2) What does configure output for you after "checking whether we are cross compiling...". For me the answers are "yes" and "no", respectively. yes
          see http://www.bufferoverflow.ch/hudson/job/Thrift-mingw32/11/console for details (lines starting with a + are command executed on command line)

          I do not have a compiler/cpp/.libs/thrift.exe

          From my perspective the attached mingw-cross-compile.sh is also sufficient, as described above.
          Here is the build using that shell script: http://www.bufferoverflow.ch/hudson/job/Thrift-mingw32_mingw-cross-compile.sh/3/console

          Show
          Roger Meier added a comment - Hi David, to your questions: (1) Do you have Wine installed? Yes (2) What does configure output for you after "checking whether we are cross compiling...". For me the answers are "yes" and "no", respectively. yes see http://www.bufferoverflow.ch/hudson/job/Thrift-mingw32/11/console for details (lines starting with a + are command executed on command line) I do not have a compiler/cpp/.libs/thrift.exe From my perspective the attached mingw-cross-compile.sh is also sufficient, as described above. Here is the build using that shell script: http://www.bufferoverflow.ch/hudson/job/Thrift-mingw32_mingw-cross-compile.sh/3/console
          Hide
          David Reiss added a comment -

          Do you know if you have binfmt-misc set up for wine? When I run

          $ echo 'int main() { return 0; }' > conftest.c
          $ i586-mingw32msvc-gcc -o conftest.exe  -DMINGW  conftest.c
          $ ./conftest.exe
          

          conftest spits out some warning messages, but it exits successfully. That's why it thinks it is not a cross compile. What happens when you run those commands?

          Show
          David Reiss added a comment - Do you know if you have binfmt-misc set up for wine? When I run $ echo 'int main() { return 0; }' > conftest.c $ i586-mingw32msvc-gcc -o conftest.exe -DMINGW conftest.c $ ./conftest.exe conftest spits out some warning messages, but it exits successfully. That's why it thinks it is not a cross compile. What happens when you run those commands?
          Hide
          Roger Meier added a comment -
          $ echo 'int main() { return 0; }' > conftest.c
          $ i586-mingw32msvc-gcc -o conftest.exe  -DMINGW  conftest.c
          $ ./conftest.exe
          $ file conftest.exe 
          conftest.exe: PE32 executable for MS Windows (console) Intel 80386 32-bit
          $ wine conftest.exe
          

          just quiet...

          $ cat /proc/sys/fs/binfmt_misc/
          cli        jar        python2.4  python2.5  register   status     wine
          $ cat /proc/sys/fs/binfmt_misc/wine 
          enabled
          interpreter /usr/share/binfmt-support/run-detectors
          flags: 
          offset 0
          magic 4d5a
          

          that's all

          Show
          Roger Meier added a comment - $ echo ' int main() { return 0; }' > conftest.c $ i586-mingw32msvc-gcc -o conftest.exe -DMINGW conftest.c $ ./conftest.exe $ file conftest.exe conftest.exe: PE32 executable for MS Windows (console) Intel 80386 32-bit $ wine conftest.exe just quiet... $ cat /proc/sys/fs/binfmt_misc/ cli jar python2.4 python2.5 register status wine $ cat /proc/sys/fs/binfmt_misc/wine enabled interpreter /usr/share/binfmt-support/run-detectors flags: offset 0 magic 4d5a that's all
          Hide
          David Reiss added a comment -

          Okay, in that second job, autoconf does not detect that it's a cross-compile, just like for me.

          I think I found the problem with .libs:

          http://www.bufferoverflow.ch/hudson/job/Print%20software%20versions/1/console

          You're using libtool 1.5.26. I'm on 2.2.6b.

          $ libtoolize --version
          libtoolize (GNU libtool) 2.2.6b
          Written by Gary V. Vaughan <gary@gnu.org>, 2003
          
          Copyright (C) 2008 Free Software Foundation, Inc.
          This is free software; see the source for copying conditions.  There is NO
          warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
          $ which libtoolize
          /usr/bin/libtoolize
          $ dpkg -S /usr/bin/libto
          libtool     libtoolize  
          $ dpkg -S /usr/bin/libtoolize 
          libtool: /usr/bin/libtoolize
          $ apt-cache show libtool
          Package: libtool
          Priority: optional
          Section: devel
          Installed-Size: 1332
          Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
          Original-Maintainer: Kurt Roeckx <kurt@roeckx.be>
          Architecture: i386
          Version: 2.2.6b-2ubuntu1
          [...]
          

          I'm guessing that's it. It looks like Lenny has a much older version than Lucid. 2.2.6b is in Lenny backports, FWIW.

          I'll just make a version of the script that works with both the older and newer version.

          Show
          David Reiss added a comment - Okay, in that second job, autoconf does not detect that it's a cross-compile, just like for me. I think I found the problem with .libs: http://www.bufferoverflow.ch/hudson/job/Print%20software%20versions/1/console You're using libtool 1.5.26. I'm on 2.2.6b. $ libtoolize --version libtoolize (GNU libtool) 2.2.6b Written by Gary V. Vaughan <gary@gnu.org>, 2003 Copyright (C) 2008 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. $ which libtoolize /usr/bin/libtoolize $ dpkg -S /usr/bin/libto libtool libtoolize $ dpkg -S /usr/bin/libtoolize libtool: /usr/bin/libtoolize $ apt-cache show libtool Package: libtool Priority: optional Section: devel Installed-Size: 1332 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Original-Maintainer: Kurt Roeckx <kurt@roeckx.be> Architecture: i386 Version: 2.2.6b-2ubuntu1 [...] I'm guessing that's it. It looks like Lenny has a much older version than Lucid. 2.2.6b is in Lenny backports, FWIW. I'll just make a version of the script that works with both the older and newer version.
          David Reiss made changes -
          Show
          David Reiss added a comment - Okay, this version appears to work both for me and your Hudson setup. LG? http://www.bufferoverflow.ch/hudson/job/Thrift-mingw32-dreiss/2/console http://gitweb.thrift-rpc.org/?p=thrift.git;a=blob;f=contrib/mingw-cross-compile.sh;h=9e8e054;hb=pri/dreiss/mingw-test
          Hide
          Roger Meier added a comment -

          Great! I like solutions which are able to run in different environments.

          Show
          Roger Meier added a comment - Great! I like solutions which are able to run in different environments.
          Roger Meier made changes -
          Component/s Build Process [ 12313845 ]
          Component/s Deployment [ 12313846 ]
          Hide
          Jens Geyer added a comment -

          Right now found that via Roger's Blog. Great work, but why don't we link that from the Thrift Wiki? I remember that it took me a while to find out how to build Thrift on Windows, even with the Wiki info, but without this stuff here. Could have saved me some hours, if I only had found it earlier.

          Show
          Jens Geyer added a comment - Right now found that via Roger's Blog. Great work, but why don't we link that from the Thrift Wiki? I remember that it took me a while to find out how to build Thrift on Windows, even with the Wiki info, but without this stuff here. Could have saved me some hours, if I only had found it earlier.
          Hide
          Jake Farrell added a comment -

          We are trying to remove the wiki dependency and move everything into the main website to make it easier to find documentation, tutorials, etc. If there is external documentation that would be of value please consider adding a patch or new ticket under the website component with it in markdown format so the information is not lost or forgotten about. For more information on the work being done for the website please see the mailing list. Thanks

          Show
          Jake Farrell added a comment - We are trying to remove the wiki dependency and move everything into the main website to make it easier to find documentation, tutorials, etc. If there is external documentation that would be of value please consider adding a patch or new ticket under the website component with it in markdown format so the information is not lost or forgotten about. For more information on the work being done for the website please see the mailing list. Thanks
          Hide
          Roger Meier added a comment - - edited

          Sorry, for not mention that topic with Windows build earlier...I just forgot that this might be a good entry point

          The Wiki is editable, feel free to add the relevant information to the Wiki.
          And we will merge your stuff with the upcoming web site!

          Or seperate Ticket as Jake mentioned
          or as extension to a existing readme file such as lib/cpp/README_WINDOWS

          Thanks,
          Roger

          PS: see http://en.wikipedia.org/wiki/Markdown or http://daringfireball.net/projects/markdown/syntax for details on markdown syntax

          Show
          Roger Meier added a comment - - edited Sorry, for not mention that topic with Windows build earlier...I just forgot that this might be a good entry point The Wiki is editable, feel free to add the relevant information to the Wiki. And we will merge your stuff with the upcoming web site! Or seperate Ticket as Jake mentioned or as extension to a existing readme file such as lib/cpp/README_WINDOWS Thanks, Roger PS: see http://en.wikipedia.org/wiki/Markdown or http://daringfireball.net/projects/markdown/syntax for details on markdown syntax
          Roger Meier made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Michael Lum [ mlum ]
          Resolution Fixed [ 1 ]
          Jake Farrell made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Michael Lum
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development