Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-4033

Default build WITH_PLUGIN=ON for all builds results in packaging errors

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.10.0
    • 0.11.0
    • Build Process
    • None
    • Ubuntu 14.04.4 LTS is where I reproduced it.

    Description

      I built thrift-0.10.0 debian packages doing the following:

      1. Clone the thrift project from github
      2. Check out the 0.10.0 tag
      3. Create a docker image: docker build -t thrift build/docker/ubuntu (from https://github.com/apache/thrift/tree/master/build/docker)
      4. Load the image as a docker instance: docker run -v $(pwd):/thrift/src -it thrift /bin/bash
      5. Run this command to build all debian package: dpkg-buildpackage -d -tc

      Then I replaced thrift-0.8.0 on my Ubuntu 14.04 system with the packages that I just built and tried to get the version from the thrift compiler:

      jking@dvm61:~/thrift/packages/0.10.0/unpacked$ dpkg -l | grep thrift
      hi  libthrift-dev                                 0.8.0-0+svt12                           amd64        Thrift C++ library (development headers)
      ii  libthrift-java                                0.8.0-0+svt12                           all          Java bindings for Thrift
      hi  libthrift-perl                                0.8.0-0+svt12                           all          Perl bindings for Thrift
      hi  libthrift0                                    0.8.0-0+svt12                           amd64        Thrift C++ library
      hi  php5-thrift                                   0.8.0-0+svt12                           amd64        PHP bindings for Thrift
      ii  python-thrift                                 0.8.0-0+svt12                           amd64        Python bindings for Thrift
      ii  thrift-compiler                               0.8.0-0+svt12                           amd64        Compiler for Thrift definition files
      jking@dvm61:~/thrift/packages/0.10.0/unpacked$ sudo dpkg -i libthrift0_0.10.0_amd64.deb libthrift-dev_0.10.0_amd64.deb libthrift-java_0.10.0_all.deb libthrift-perl_0.10.0_all.deb php5-thrift_0.10.0_amd64.deb python-thrift_0.10.0_amd64.deb thrift-compiler_0.10.0_amd64.deb 
      (Reading database ... 232573 files and directories currently installed.)
      Preparing to unpack libthrift0_0.10.0_amd64.deb ...
      Unpacking libthrift0 (0.10.0) over (0.8.0-0+svt12) ...
      Preparing to unpack libthrift-dev_0.10.0_amd64.deb ...
      Unpacking libthrift-dev (0.10.0) over (0.8.0-0+svt12) ...
      Preparing to unpack libthrift-java_0.10.0_all.deb ...
      Unpacking libthrift-java (0.10.0) over (0.8.0-0+svt12) ...
      Preparing to unpack libthrift-perl_0.10.0_all.deb ...
      Unpacking libthrift-perl (0.10.0) over (0.8.0-0+svt12) ...
      Preparing to unpack php5-thrift_0.10.0_amd64.deb ...
      Unpacking php5-thrift (0.10.0) over (0.8.0-0+svt12) ...
      Preparing to unpack python-thrift_0.10.0_amd64.deb ...
      Unpacking python-thrift (0.10.0) over (0.8.0-0+svt12) ...
      Preparing to unpack thrift-compiler_0.10.0_amd64.deb ...
      Unpacking thrift-compiler (0.10.0) over (0.8.0-0+svt12) ...
      Setting up libthrift0 (0.10.0) ...
      Setting up libthrift-dev (0.10.0) ...
      Setting up libthrift-java (0.10.0) ...
      Setting up libthrift-perl (0.10.0) ...
      Setting up php5-thrift (0.10.0) ...
      Setting up python-thrift (0.10.0) ...
      Setting up thrift-compiler (0.10.0) ...
      Processing triggers for python-support (1.0.15) ...
      Processing triggers for libc-bin (2.19-0ubuntu6.7+svt1) ...
      jking@dvm61:~/thrift/packages/0.10.0/unpacked$ dpkg -l | grep thrift
      ii  libthrift-dev                                 0.10.0                                  amd64        Thrift C++ library (development headers)
      ii  libthrift-java                                0.10.0                                  all          Java bindings for Thrift
      ii  libthrift-perl                                0.10.0                                  all          Perl bindings for Thrift
      ii  libthrift0                                    0.10.0                                  amd64        Thrift C++ library
      ii  php5-thrift                                   0.10.0                                  amd64        PHP bindings for Thrift
      ii  python-thrift                                 0.10.0                                  amd64        Python bindings for Thrift (Python 2)
      ii  thrift-compiler                               0.10.0                                  amd64        Compiler for Thrift definition files
      

      So then I tried to run the thrift compiler:

      jking@dvm61:~/thrift/packages/0.10.0/unpacked$ thrift --version
      mkdir: cannot create directory ‘/usr/bin/.libs’: Permission denied
      /usr/bin/thrift: line 202: cd: /thrift/src/compiler/cpp: No such file or directory
      g++: error: src/thrift/thrift-main.o: No such file or directory
      g++: error: src/thrift/audit/thrift-t_audit.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_c_glib_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_cpp_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_java_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_json_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_as3_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_dart_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_haxe_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_csharp_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_py_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_rb_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_perl_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_php_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_erl_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_cocoa_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_swift_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_st_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_ocaml_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_hs_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_xsd_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_xml_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_html_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_js_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_javame_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_delphi_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_go_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_gv_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_d_generator.o: No such file or directory
      g++: error: src/thrift/generate/thrift-t_lua_generator.o: No such file or directory
      g++: error: src/thrift/libparse.a: No such file or directory
      g++: error: ./.libs/libthriftc.so: No such file or directory
      
      Something looks quite wrong here…  I am going to open a Jira ticket.  
      It looks like this release is DOA unless I build the thrift compiler without plug-in support.
      It looks like it is assuming I am root and I can modify /usr/bin/.libs, or it thinks things are at /thrift
      

      /usr/bin/thrift is a text file provided by thrift-compiler-0.10.0.
      It has a comment in it:

      # This wrapper script should never be moved out of the build directory.
      # If it is, it will not operate correctly.
      

      It looks like we took this file and distributed it, moving it from the build directory to /usr/bin/thrift. It doesn't look like this works.

      The root cause is that this was the result of building the thrift compiler with plug-in support. I submitted a pull request to resolve this, however jfarrell expressed a desire to make the default compiler build without plugin support so the packaging jobs we already have will continue to work.

      As such, the scope of this defect is to make the autoconf and cmake defaults set WITH_PLUGIN=OFF behavior.

      Attachments

        Issue Links

          Activity

            The dpkg build for thrift seems to depend on building a thrift compiler without plug-ins, but the default mode to build the compiler is with plug-ins. The thrift-compiler-0.10.0 package does not provide any of the missing .o files... it wants /usr/bin/thrift to have minimal dependencies. I think that when building with dpkg-buildpackage we need to tell thrift to build a non-plugin compiler.

            jking3 James E. King III added a comment - The dpkg build for thrift seems to depend on building a thrift compiler without plug-ins, but the default mode to build the compiler is with plug-ins. The thrift-compiler-0.10.0 package does not provide any of the missing .o files... it wants /usr/bin/thrift to have minimal dependencies. I think that when building with dpkg-buildpackage we need to tell thrift to build a non-plugin compiler.

            I have a temporary solution for this which is to modify configure.ac and change "have_plugin" to "no" then build the debian packages. This keeps everything enabled but it disables the plugin support, but it will have full language support:

            thrift 0.10.0
            
            Building Plugin Support ...... : no
            Building C++ Library ......... : yes
            Building C (GLib) Library .... : yes
            Building Java Library ........ : yes
            Building C# Library .......... : yes
            Building Python Library ...... : yes
            Building Ruby Library ........ : yes
            Building Haxe Library ........ : yes
            Building Haskell Library ..... : yes
            Building Perl Library ........ : yes
            Building PHP Library ......... : yes
            Building Dart Library ........ : yes
            Building Erlang Library ...... : yes
            Building Go Library .......... : yes
            Building D Library ........... : yes
            Building NodeJS Library ...... : yes
            Building Lua Library ......... : yes
            
            C++ Library:
               Build TZlibTransport ...... : yes
               Build TNonblockingServer .. : yes
               Build TQTcpServer (Qt4) .... : no
               Build TQTcpServer (Qt5) .... : yes
            
            Java Library:
               Using javac ............... : javac
               Using java ................ : java
               Using ant ................. : /usr/bin/ant
            
            C# Library:
               Using .NET 3.5 ............ : yes
            
            Python Library:
               Using Python .............. : /usr/bin/python
               Using Python3 ............. : /usr/bin/python3
               Using trial ............... : /usr/bin/trial
            
            PHP Library:
               Using php-config .......... : /usr/bin/php-config
            
            Dart Library:
               Using Dart ................ : /usr/lib/dart/bin/dart
               Using Pub ................. : /usr/lib/dart/bin/pub
            
            Ruby Library:
               Using Ruby ................ : /usr/bin/ruby
            
            Haskell Library:
               Using Haskell ............. : /usr/bin/runhaskell
               Using Cabal ............... : /usr/bin/cabal
            
            Haxe Library:
               Using Haxe ................ : /usr/bin/haxe
               Using Haxe version ........ : 3.2.0
            
            Perl Library:
               Using Perl ................ : /usr/bin/perl
            
            Erlang Library:
               Using erlc ................ : /usr/bin/erlc
               Using rebar ............... : /usr/bin/rebar
            
            Go Library:
               Using Go................... : /usr/local/go/bin/go
               Using Go version........... : go version go1.4.3 linux/amd64
            
            D Library:
               Using D Compiler .......... : dmd
               Building D libevent tests . : yes
               Building D SSL tests ...... : yes
            
            NodeJS Library:
               Using NodeJS .............. : /usr/bin/nodejs
               Using NodeJS version....... : v0.10.25
            
            Lua Library:
               Using Lua .............. : /usr/bin/lua
            

            jfarrell I think modifying debian/rules where it calls configure would also work here, but I tried "--without-plugin" and that didn't work.

            jking3 James E. King III added a comment - I have a temporary solution for this which is to modify configure.ac and change "have_plugin" to "no" then build the debian packages. This keeps everything enabled but it disables the plugin support, but it will have full language support: thrift 0.10.0 Building Plugin Support ...... : no Building C++ Library ......... : yes Building C (GLib) Library .... : yes Building Java Library ........ : yes Building C# Library .......... : yes Building Python Library ...... : yes Building Ruby Library ........ : yes Building Haxe Library ........ : yes Building Haskell Library ..... : yes Building Perl Library ........ : yes Building PHP Library ......... : yes Building Dart Library ........ : yes Building Erlang Library ...... : yes Building Go Library .......... : yes Building D Library ........... : yes Building NodeJS Library ...... : yes Building Lua Library ......... : yes C++ Library: Build TZlibTransport ...... : yes Build TNonblockingServer .. : yes Build TQTcpServer (Qt4) .... : no Build TQTcpServer (Qt5) .... : yes Java Library: Using javac ............... : javac Using java ................ : java Using ant ................. : /usr/bin/ant C# Library: Using .NET 3.5 ............ : yes Python Library: Using Python .............. : /usr/bin/python Using Python3 ............. : /usr/bin/python3 Using trial ............... : /usr/bin/trial PHP Library: Using php-config .......... : /usr/bin/php-config Dart Library: Using Dart ................ : /usr/lib/dart/bin/dart Using Pub ................. : /usr/lib/dart/bin/pub Ruby Library: Using Ruby ................ : /usr/bin/ruby Haskell Library: Using Haskell ............. : /usr/bin/runhaskell Using Cabal ............... : /usr/bin/cabal Haxe Library: Using Haxe ................ : /usr/bin/haxe Using Haxe version ........ : 3.2.0 Perl Library: Using Perl ................ : /usr/bin/perl Erlang Library: Using erlc ................ : /usr/bin/erlc Using rebar ............... : /usr/bin/rebar Go Library: Using Go................... : /usr/local/go/bin/go Using Go version........... : go version go1.4.3 linux/amd64 D Library: Using D Compiler .......... : dmd Building D libevent tests . : yes Building D SSL tests ...... : yes NodeJS Library: Using NodeJS .............. : /usr/bin/nodejs Using NodeJS version....... : v0.10.25 Lua Library: Using Lua .............. : /usr/bin/lua jfarrell I think modifying debian/rules where it calls configure would also work here, but I tried "--without-plugin" and that didn't work.
            jking3 James E. King III added a comment - - edited

            Changing line 32 of debian/rules to:

            $(CURDIR)/configure --prefix=/usr --enable-plugin=no
            

            I also had to move the compiler build before the cpp library build to make it work.

            Disables the plug-in stuff for the debian package build. I will submit a PR to fix this.
            In the meantime, jfarrell we need to decide what to do about the 0.10.0 release, since it's DOA on debian based systems using this build methodology.

            jking3 James E. King III added a comment - - edited Changing line 32 of debian/rules to: $(CURDIR)/configure --prefix=/usr --enable-plugin=no I also had to move the compiler build before the cpp library build to make it work. Disables the plug-in stuff for the debian package build. I will submit a PR to fix this. In the meantime, jfarrell we need to decide what to do about the 0.10.0 release, since it's DOA on debian based systems using this build methodology.
            githubbot ASF GitHub Bot added a comment -

            GitHub user jeking3 opened a pull request:

            https://github.com/apache/thrift/pull/1157

            THRIFT-4033 disable compiler plug-ins for standard debian build

            as the thrift-compiler packaging is not ready for it yet

            You can merge this pull request into a Git repository by running:

            $ git pull https://github.com/jeking3/thrift defect/THRIFT-4033

            Alternatively you can review and apply these changes as the patch at:

            https://github.com/apache/thrift/pull/1157.patch

            To close this pull request, make a commit to your master/trunk branch
            with (at least) the following in the commit message:

            This closes #1157


            commit 800d09a0e330223de13a73f6405f2bab3ddad5fe
            Author: James E. King, III <jim.king@simplivity.com>
            Date: 2017-01-16T20:49:35Z

            THRIFT-4033 disable compiler plug-ins for standard debian build, as the thrift-compiler packaging is not ready for it yet


            githubbot ASF GitHub Bot added a comment - GitHub user jeking3 opened a pull request: https://github.com/apache/thrift/pull/1157 THRIFT-4033 disable compiler plug-ins for standard debian build as the thrift-compiler packaging is not ready for it yet You can merge this pull request into a Git repository by running: $ git pull https://github.com/jeking3/thrift defect/ THRIFT-4033 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/thrift/pull/1157.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1157 commit 800d09a0e330223de13a73f6405f2bab3ddad5fe Author: James E. King, III <jim.king@simplivity.com> Date: 2017-01-16T20:49:35Z THRIFT-4033 disable compiler plug-ins for standard debian build, as the thrift-compiler packaging is not ready for it yet
            githubbot ASF GitHub Bot added a comment -

            Github user jeking3 commented on the issue:

            https://github.com/apache/thrift/pull/1157

            @jfarrell I'd like one review of this before I merge it. It did fix the issue I had with the stock 0.10.0 debian package for the compiler not working.

            githubbot ASF GitHub Bot added a comment - Github user jeking3 commented on the issue: https://github.com/apache/thrift/pull/1157 @jfarrell I'd like one review of this before I merge it. It did fix the issue I had with the stock 0.10.0 debian package for the compiler not working.
            githubbot ASF GitHub Bot added a comment -

            Github user jfarrell commented on the issue:

            https://github.com/apache/thrift/pull/1157

            We have had issues with plugin support being on by default in a couple build instances now, and this does not fix when we pull out autoconf that it wont reappear with cmake. I'd like to see us change the defaults here and make --with-plugin the optional to enable and having WITH_PLUGIN=no so it is disabled by default

            githubbot ASF GitHub Bot added a comment - Github user jfarrell commented on the issue: https://github.com/apache/thrift/pull/1157 We have had issues with plugin support being on by default in a couple build instances now, and this does not fix when we pull out autoconf that it wont reappear with cmake. I'd like to see us change the defaults here and make --with-plugin the optional to enable and having WITH_PLUGIN=no so it is disabled by default
            githubbot ASF GitHub Bot added a comment -

            Github user jeking3 commented on the issue:

            https://github.com/apache/thrift/pull/1157

            I'm okay making the default for plug-in support "OFF", and you have to enable it specifically. Would it be okay to merge this as it fixes the ticket as described, and open a new one to change the global behavior?

            githubbot ASF GitHub Bot added a comment - Github user jeking3 commented on the issue: https://github.com/apache/thrift/pull/1157 I'm okay making the default for plug-in support "OFF", and you have to enable it specifically. Would it be okay to merge this as it fixes the ticket as described, and open a new one to change the global behavior?
            githubbot ASF GitHub Bot added a comment -

            Github user jfarrell commented on the issue:

            https://github.com/apache/thrift/pull/1157

            sure, or modify the ticket description to 'disable compiler plug-ins by default for all builds'

            githubbot ASF GitHub Bot added a comment - Github user jfarrell commented on the issue: https://github.com/apache/thrift/pull/1157 sure, or modify the ticket description to 'disable compiler plug-ins by default for all builds'

            Looks like when I set the default to "off" in configure.ac, the debian packaging build failed.

            jking3 James E. King III added a comment - Looks like when I set the default to "off" in configure.ac, the debian packaging build failed.
            githubbot ASF GitHub Bot added a comment -

            Github user asfgit closed the pull request at:

            https://github.com/apache/thrift/pull/1157

            githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/thrift/pull/1157

            People

              jking3 James E. King III
              jking3 James E. King III
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: