3.2 working version will not install on Mac OS X (10.3.7)

• Key: MODPYTHON-65
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Unassigned
• Reporter: Graham Dumpleton
• Component/s: core
• Affects Version/s: 3.2.7
• Fix Version/s: 3.2.7

Description

Something is wrong with configure or setup.py file.

/usr/bin/python setup.py build
running build
running build_py
running build_ext
building 'mod_python_so' extension
gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/mod_python.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/_apachemodule.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/connobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/filterobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/hlist.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/hlistobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/requestobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/serverobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/tableobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/util.o -L -lapr-0 -laprutil-0 -o build/lib.darwin-7.7.0-Power_Macintosh-2.3/mod_python_so.so
ld: -L: directory name missing
error: command 'gcc' failed with exit status 1

More later when I have a chance to work out what is wrong.

Comments

[Graham Dumpleton]

To get past this, the following patch needs to be applied to dist/setup.py.in:

Index: dist/setup.py.in
===================================================================
--- dist/setup.py.in (revision 220194)
+++ dist/setup.py.in (working copy)
@@ -85,7 +85,7 @@
     """returns apache lib directory"""
     apache_srcdir = getapache_srcdir()
     if apache_srcdir is None:
- return ""
+ return getapxs_option("LIBDIR")
     else:
         return os.path.join(apache_srcdir, "lib")
 


This just means we get to the next problem though.

gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/mod_python.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/_apachemodule.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/connobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/filterobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/hlist.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/hlistobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/requestobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/serverobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/tableobject.o build/temp.darwin-7.7.0-Power_Macintosh-2.3/Users/grahamd/Workspaces/mod_python/src/util.o -L/usr/local/apache-2.0/lib -lapr-0 -laprutil-0 -o build/lib.darwin-7.7.0-Power_Macintosh-2.3/mod_python_so.so
ld: Undefined symbols:
_ap_add_cgi_vars
_ap_add_version_component
_ap_get_brigade
_ap_getword
_ap_getword_white
_ap_hook_access_checker
_ap_hook_auth_checker
_ap_hook_check_user_id
_ap_hook_child_init
_ap_hook_fixups
_ap_hook_handler
_ap_hook_header_parser
_ap_hook_log_transaction
_ap_hook_post_config
_ap_hook_post_read_request
_ap_hook_process_connection
_ap_hook_translate_name
_ap_hook_type_checker
_ap_is_directory
_ap_log_error
_ap_log_rerror
_ap_make_dirstr_parent
_ap_mpm_query
_ap_note_basic_auth_failure
_ap_pass_brigade
_ap_register_input_filter
_ap_register_output_filter
_unixd_config
_unixd_set_global_mutex_perms
_ap_conftree
_ap_server_root
_ap_unescape_url
_ap_add_common_vars
_ap_allow_methods
_ap_document_root
_ap_get_basic_auth_pw
_ap_get_client_block
_ap_get_remote_host
_ap_internal_redirect
_ap_meets_conditions
_ap_requires
_ap_rflush
_ap_rwrite
_ap_send_fd
_ap_set_content_length
_ap_set_content_type
_ap_setup_client_block
_ap_should_client_block
_ap_my_generation
_ap_scoreboard_image
_ap_loaded_modules
error: command 'gcc' failed with exit status 1

I am assuming for the moment that these are symbols that are within
the Apache executable itself, as they don't appear in libapr or libaprutil
libraries.

I can see that this move to setup.py away from libtool might cause
various problems on Mac OS X initially because of the quite
different way that it does creation of shared objects. :-(

[Graham Dumpleton]

The problem here is that on Mac OS X, distutils compiles Python
loadable modules in a way that expects all symbols to be resolvable
at the time the module is created. That is, all symbols must be present
in either the objects being linked into the module or in libraries which
are explicitly linked with the module.

In a scenario like mod_python where there will be undefined references
to symbols which are only defined within the executable running the
interpreter into which the module is loaded, this will fail.

Previously this wasn't a problem because distutils wasn't used to
create the module and instead the makefile use libtool to do it. One of
the options to libtool was "--silent". This option (at least I think so, it may
be a different option), would underneath pass appropriate options to
the linker for a platform to disable undefined symbols causing an error.

Since distutils is now used, there is going to have to be platform specific
hacks to add additional options to the link line when creating a module
to add these platform specific options. This however is not as simple as
defining extra_link_flags to the distutils extension object as they will only
be placed at the end. These magic options tend to be positional
dependent and need to be included at a specific spot early in the
link line. The distutils package does not give you this level of control
through its public APIs and thus a hack is required which delves into
its private parts. If distutils changes, like it did from 2.2 to 2.3, this hack
will break and no longer work.

This problem may affect other platforms besides MacOSX as there are
other platforms that also require specific options to ignore undefined
symbols.

Anyway, the hack required to get this to work for Mac OS X with Python 2.3
is as follows. If you know of a better way, by all means let me know. :-)

Index: dist/setup.py.in
===================================================================
--- dist/setup.py.in (revision 220194)
+++ dist/setup.py.in (working copy)
@@ -85,7 +85,7 @@
     """returns apache lib directory"""
     apache_srcdir = getapache_srcdir()
     if apache_srcdir is None:
- return ""
+ return getapxs_option("LIBDIR")
     else:
         return os.path.join(apache_srcdir, "lib")
 
@@ -153,6 +153,17 @@
     scripts = []
     data_files = []
 
+import string
+from distutils import sysconfig
+
+if sys.platform == "darwin":
+ sysconfig._config_vars["LDSHARED"] = \
+ string.replace(sysconfig.get_config_var("LDSHARED"), \
+ " -bundle "," -bundle -flat_namespace -undefined suppress ")
+ sysconfig._config_vars["BLDSHARED"] = \
+ string.replace(sysconfig.get_config_var("BLDSHARED"), \
+ " -bundle "," -bundle -flat_namespace -undefined suppress ")
+
 setup(name="mod_python",
       version=VER,
       description="Apache/Python Integration",

[Graham Dumpleton]

Attached supplementary patch to only do this workaround when required,
ie., when version of Python doesn't have required option already.

Copy and paste from mailing list included below.

On 19/08/2005, at 7:57 AM, Graham Dumpleton wrote:

>
> On 19/08/2005, at 2:59 AM, Jim Gallacher wrote:
>
>
>> Ron Reisor wrote:
>>
>>> Hello,
>>> I ran into a problem with the loader on MacOSX.
>>> MaxOSX 1.4.2
>>> python 2.4.1
>>> apache 2.0.54
>>> The loader seems to not like the "-undefined suppress" arguments
>>> in the final load.
>>> I modified dist/setup.py by removing the two "-undefined
>>> suppress" and ran configure and make again and the new mod_python
>>> does build and seems to work ok.
>>>
>>
>> The bit of code which includes "-undefined suppress" came from the
>> patch Graham submitted for MODPYTHON-65. See http://
>> issues.apache.org/jira/browse/MODPYTHON-65 for details.
>>
>> Not being a Mac person I can't comment further. Feel free to talk
>> among yourselves. ;)
>>
>
> Yep, my fault. It will not build on Mac OS X 10.3.9 without that
> option if
> you are using the standard version of GCC shipped with the operating
> system.
>
> I'll get onto my new Tiger laptop and try it there (haven't yet),
> but can
> you tell me which GCC you are using? I know the GCC version shipped
> on the
> box with Tiger, but are you using that, or are you using one
> supplied with
> Fink? Was worried that the change might not apply to the Fink
> version of GCC.

The missing bit of information here is that Python on Tiger (10.4) is
correctly
putting in the config/Makefile used by distutils the required
options. Ie.,

   LDSHARED= $(CC) $(LDFLAGS) -bundle -undefined dynamic_lookup
   BLDSHARED= $(CC) $(LDFLAGS) -bundle -undefined dynamic_lookup

The duplicated definition of the -undefined option causes grief when
setup.py
adds it as well.

Thus, setup.py will thus need to check first whether the -undefined
option
already exists. Thus, setup.py should use:

if sys.platform == "darwin":
         if not '-bundle' in sysconfig.get_config_var
("LDSHARED").split():
             sysconfig._config_vars["LDSHARED"] = \
                     string.replace(sysconfig.get_config_var
("LDSHARED"), \
                     " -bundle "," -bundle -flat_namespace -undefined
suppress ")
             sysconfig._config_vars["BLDSHARED"] = \
                     string.replace(sysconfig.get_config_var
("BLDSHARED"), \
                     " -bundle "," -bundle -flat_namespace -undefined
suppress ")

In Tiger it actually uses '-undefined dynamic_lookup' instead of what
I had
used which is '-undefined suppress'. If though you try and use '-
undefined
dynamic_lookup' on Panther (10.3) or earlier you will run up against a
further problem in that '-undefined dynamic_lookup' conflicts with the
default compilation mode of being compatible with Mac OS X 10.1. Thus,
still probably best to use '-undefined suppress', but the check means
that
it will only be used on older versions of Mac OS X anyway, or at
least where
the Python config isn't correct.

I'll attach a patch to the JIRA report on the original problem shortly.

Graham

[Jim Gallacher]

Graham's setup.py.in-2.diff patch has been applied.