Kickaha
2007-10-04, 14:15
Alright, I'm going bonkers. In the middle of porting a Linux app to MacOS X 10.4, and I've run into what seems to be a very common problem: converting the build system for dynamic libraries. There is no end of 'helpful' advice running around, 99.9% of which seems to boil down to "Try everything" or "Use this magic sequence, it works for some people, nobody knows why". The last .1% seems to be posts of "Got it working! Thanks!" *with no explanation of how*. Bastards.
The build system I'm struggling with is... byzantine. Multiple layers of Makefiles with embedded Perl, custom scripts that don't really *do* much that I can see, other than add obfuscation, etc, etc.
In other words, a pretty normal OSS project. :)
So here's the setup:
1) .o files being archived into .a libs.
2) .a libs being gathered into .dylibs.
3) binaries being built on dylibs.
Simple, no?
The gcc/ld link flags for the linux build are:
CFLAGS throughout are the same: -pipe -Wreturn-type -march=i686 -fPIC -Wunused -Wformat -Wall -Wno-parentheses -Wno-switch -Wno-sign-compare -Wno-unused-parameter
.c%.o:
g++ $(CFLAGS) -o foo.o foo.c
.o%.a:
ar cr foo.a *.o
.a%.dylib:
g++ -L/usr/local/lib -L. -lfoo -o libbar.so -Wl,--enable-new-dtags -shared -Wl,--whole-archive ../lib/[.a libs here] -Wl,--no-whole-archive
No, I don't know why they switch back to --no-whole-archive, I just report what they do.
.dylib%a.out:
g++ $(CFLAGS) -L$(DLLDIR) -Wl,-rpath-link,$(DLLDIR) -L. -o my_executable ../main.c -lMyLib1 -lMyLib2
--------
The analogous (I thought) flags for OS X's build chain are: (first two are the same)
.c%.o:
g++ $(CFLAGS) -o foo.o foo.c
.o%.a:
ar cr foo.a *.o
.a%.dylib:
g++ -L/usr/local/lib -L. -lfoo -o libbar.dylib -dynamiclib -all_load -single_module -flat_namespace -force_flat_namespace -bind_at_load ../lib/[.a libs here]
-shared -> -dynamiclib
--whole-archive -> -all_load
because it's not a MacOS X app -> -single_module -flat_namespace -force_flat_namespace
who the hell knows, see 99.9% reference above -> -bind_at_load
.dylib%a.out:
g++ $(CFLAGS) -fvisibility=default -L$(DLLDIR) -L. -o my_executable ../main.c -lMyLib1 -lMyLib2
The -fvisibility is to make sure that 'older style' auto-export of symbols in dylibs is done.
Of course, this doesn't work, it can't find the second level dependent libraries...
/usr/bin/ld: warning can't open dynamic library: libMyLib3.dylib referenced from: ../dll/libMyLib1.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2)
/usr/bin/ld: Undefined symbols:
_FirstUndefSym referenced from libMyLib1 expected to be defined in libMyLib3.dylib
I did an 'export DYLD_LIBRARY_PATH = $(DLLDIR)', *and* set it in the shell. Neither worked. I need a replacement for that -rpath up in the Linux build line, apparently.
Looking through the Makefiles, ranlib is never being invoked directly, but shouldn't that be handled by passing everything through gcc??
I'm getting another 'undefined symbol' error from a .a library that obviously *HAS* the symbol in it (two .o files, one defines, one uses, apparently the one using is bitching... in the *same library* - grr), but let's take this one step at a time.
On writing this up, I'm wondering if the -fvisibility flag should be in the .a%.dylib command...
The build system I'm struggling with is... byzantine. Multiple layers of Makefiles with embedded Perl, custom scripts that don't really *do* much that I can see, other than add obfuscation, etc, etc.
In other words, a pretty normal OSS project. :)
So here's the setup:
1) .o files being archived into .a libs.
2) .a libs being gathered into .dylibs.
3) binaries being built on dylibs.
Simple, no?
The gcc/ld link flags for the linux build are:
CFLAGS throughout are the same: -pipe -Wreturn-type -march=i686 -fPIC -Wunused -Wformat -Wall -Wno-parentheses -Wno-switch -Wno-sign-compare -Wno-unused-parameter
.c%.o:
g++ $(CFLAGS) -o foo.o foo.c
.o%.a:
ar cr foo.a *.o
.a%.dylib:
g++ -L/usr/local/lib -L. -lfoo -o libbar.so -Wl,--enable-new-dtags -shared -Wl,--whole-archive ../lib/[.a libs here] -Wl,--no-whole-archive
No, I don't know why they switch back to --no-whole-archive, I just report what they do.
.dylib%a.out:
g++ $(CFLAGS) -L$(DLLDIR) -Wl,-rpath-link,$(DLLDIR) -L. -o my_executable ../main.c -lMyLib1 -lMyLib2
--------
The analogous (I thought) flags for OS X's build chain are: (first two are the same)
.c%.o:
g++ $(CFLAGS) -o foo.o foo.c
.o%.a:
ar cr foo.a *.o
.a%.dylib:
g++ -L/usr/local/lib -L. -lfoo -o libbar.dylib -dynamiclib -all_load -single_module -flat_namespace -force_flat_namespace -bind_at_load ../lib/[.a libs here]
-shared -> -dynamiclib
--whole-archive -> -all_load
because it's not a MacOS X app -> -single_module -flat_namespace -force_flat_namespace
who the hell knows, see 99.9% reference above -> -bind_at_load
.dylib%a.out:
g++ $(CFLAGS) -fvisibility=default -L$(DLLDIR) -L. -o my_executable ../main.c -lMyLib1 -lMyLib2
The -fvisibility is to make sure that 'older style' auto-export of symbols in dylibs is done.
Of course, this doesn't work, it can't find the second level dependent libraries...
/usr/bin/ld: warning can't open dynamic library: libMyLib3.dylib referenced from: ../dll/libMyLib1.dylib (checking for undefined symbols may be affected) (No such file or directory, errno = 2)
/usr/bin/ld: Undefined symbols:
_FirstUndefSym referenced from libMyLib1 expected to be defined in libMyLib3.dylib
I did an 'export DYLD_LIBRARY_PATH = $(DLLDIR)', *and* set it in the shell. Neither worked. I need a replacement for that -rpath up in the Linux build line, apparently.
Looking through the Makefiles, ranlib is never being invoked directly, but shouldn't that be handled by passing everything through gcc??
I'm getting another 'undefined symbol' error from a .a library that obviously *HAS* the symbol in it (two .o files, one defines, one uses, apparently the one using is bitching... in the *same library* - grr), but let's take this one step at a time.
On writing this up, I'm wondering if the -fvisibility flag should be in the .a%.dylib command...