Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-5145

protobuf vendored but its depencencies are not

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Accepted
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build
    • Labels:

      Description

      Updating protobuf from 2.5 to 2.6.1 has caused Mesos builds to fail if pypi.python.org is unreachable. Protobuf-2.6.1 requires google-apputils and if it's not available the build process will attempt to download it from pypi.

      Prior to this change it was possible to build Mesos without Internet access. If the build process reaches out to arbitrary things on the Internet it's impossible to guarantee build reproducibility.

      snippet from setup.py in protobuf-2.6.1.tar.gz
        setup(name = 'protobuf',
              version = '2.6.1',
      ...
              setup_requires = ['google-apputils'],
      ...
              )
      
      snippet from build log
      08:20:49 DEBUG: Building protobuf Python egg ...
      08:20:49 DEBUG: cd ../3rdparty/libprocess/3rdparty/protobuf-2.6.1/python &&			\
      08:20:49 DEBUG: 	  CC="gcc"					\
      08:20:49 DEBUG: 	  CXX="g++"					\
      08:20:49 DEBUG: 	  CFLAGS="-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wno-unused-local-typedefs"				\
      08:20:49 DEBUG: 	  CXXFLAGS="-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wno-unused-local-typedefs -Wno-maybe-uninitialized -std=c++11"			\
      08:20:49 DEBUG: 	  PYTHONPATH=/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26	\
      08:20:49 DEBUG: 	  /usr/bin/python2.7 setup.py build bdist_egg
      08:20:49 DEBUG: Download error on http://pypi.python.org/simple/google-apputils/: [Errno 111] Connection refused -- Some packages may not be found!
      08:20:49 DEBUG: Download error on http://pypi.python.org/simple/google-apputils/: [Errno 111] Connection refused -- Some packages may not be found!
      08:20:49 DEBUG: Couldn't find index page for 'google-apputils' (maybe misspelled?)
      08:20:49 DEBUG: Download error on http://pypi.python.org/simple/: [Errno 111] Connection refused -- Some packages may not be found!
      08:20:49 DEBUG: No local packages or download links found for google-apputils
      08:20:49 DEBUG: Traceback (most recent call last):
      08:20:49 DEBUG:   File "setup.py", line 200, in <module>
      08:20:49 DEBUG:     "Protocol Buffers are Google's data interchange format.",
      08:20:49 DEBUG:   File "/usr/lib64/python2.7/distutils/core.py", line 111, in setup
      08:20:49 DEBUG:     _setup_distribution = dist = klass(attrs)
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/setuptools/dist.py", line 221, in __init__
      08:20:49 DEBUG:     self.fetch_build_eggs(attrs.pop('setup_requires'))
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/setuptools/dist.py", line 245, in fetch_build_eggs
      08:20:49 DEBUG:     parse_requirements(requires), installer=self.fetch_build_egg
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/pkg_resources.py", line 580, in resolve
      08:20:49 DEBUG:     dist = best[req.key] = env.best_match(req, self, installer)
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/pkg_resources.py", line 825, in best_match
      08:20:49 DEBUG:     return self.obtain(req, installer) # try and download/install
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/pkg_resources.py", line 837, in obtain
      08:20:49 DEBUG:     return installer(requirement)
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/setuptools/dist.py", line 294, in fetch_build_egg
      08:20:49 DEBUG:     return cmd.easy_install(req)
      08:20:49 DEBUG:   File "/builddir/build/BUILD/mesos-0.29.0/3rdparty/distribute-0.6.26/setuptools/command/easy_install.py", line 584, in easy_install
      08:20:49 DEBUG:     raise DistutilsError(msg)
      08:20:49 DEBUG: distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('google-apputils')
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                drobinson Daniel Robinson
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated: