building quad-fat binaries and endianess issues

NeXT Computer, Inc. -> Porting New Software

Title: building quad-fat binaries and endianess issues
Post by: linmag7 on February 18, 2009, 04:02:41 PM
The NS-3.3 dev kit can produce binaries for four different platforms. The compiler can generate quadfat object files but linker and "ar" tools can only work with non-fat libs/binaries. The workaround seems to be to do four different builds and use lipo to glue it all together, or  am I missing something here?

In order to cross-compile one can supply the -arch xxx flag to the compiler/linker with CFLAGS and LDFLAGS environment variables (or edit makefiles). Some sourcetrees support cross-compiling with the --host xxx-xxx-xxxx argument to the ./configure script. However some source trees are set up to configure the code for little-endian/big-endian architecture and they do the tests on the build platform rather than on the target platform.
How does one generally deal with building source for a big-endian taget platform, on a little-endian machine? Seems like its necessary to go through the code checking for endianess problems before building? This seemes like alot more work than doing a native build, or is it rarely an issue at all?
Title: Re: building quad-fat binaries and endianess issues
Post by: kb7sqi on February 19, 2009, 12:09:04 AM
Quote from: "linmag7"The NS-3.3 dev kit can produce binaries for four different platforms. The compiler can generate quadfat object files but linker and "ar" tools can only work with non-fat libs/binaries. The workaround seems to be to do four different builds and use lipo to glue it all together, or  am I missing something here?

In order to cross-compile one can supply the -arch xxx flag to the compiler/linker with CFLAGS and LDFLAGS environment variables (or edit makefiles). Some sourcetrees support cross-compiling with the --host xxx-xxx-xxxx argument to the ./configure script. However some source trees are set up to configure the code for little-endian/big-endian architecture and they do the tests on the build platform rather than on the target platform.
How does one generally deal with building source for a big-endian taget platform, on a little-endian machine? Seems like its necessary to go through the code checking for endianess problems before building? This seemes like alot more work than doing a native build, or is it rarely an issue at all?

Hey linmag,
      The biggest problem I've had w/ needing to "lipo" something together is when compiling OpenSSL.  I haven't needed to do it for anything else.  What are you trying to compile?  
     Here's an example to fix things so they compile when dealing w/ endiness.

/* Do not trust WORDS_BIGENDIAN from configure since -arch compiler flag may result in a different endian.  Instead trust __BIG_ENDIAN__ and
__LITTLE_ENDIAN__ which are set correctly by -arch. */

#undef WORDS_BIGENDIAN
#ifdef __BIG_ENDIAN__
#define WORDS_BIGENDIAN
#endif

or

/* Define to 1 if your processor stores words with the most significant byte
  first (like Motorola and SPARC, unlike Intel and VAX). */
/* #undef WORDS_BIGENDIAN */

#undef WORDS_BIGENDIAN
#ifdef __BIG_ENDIAN__
#define WORDS_BIGENDIAN 1
#endif


Some programs won't have WORDS_BIGENDIAN.  They'll have something similar/close.  So after I run configure, I'll usually check the config files & set it manually like above.  Hope that helps.  It's worked so far for me. :-)
Take care.
Title: endianess
Post by: linmag7 on February 19, 2009, 01:38:37 AM
I'm trying to build BerkleyDB-3.3 which has some platform dependant assembly code for mutex/semaphores. But maybe it can be done with some clever #defines in the config files. I have however had problems with the "ar" utility that doesn't want to create fat-libraries from fat object files.
Title: Re: endianess
Post by: kb7sqi on February 19, 2009, 02:44:01 AM
Quote from: "linmag7"I'm trying to build BerkleyDB-3.3 which has some platform dependant assembly code for mutex/semaphores. But maybe it can be done with some clever #defines in the config files. I have however had problems with the "ar" utility that doesn't want to create fat-libraries from fat object files.

Assembly Code is the another problem when compiling quad-fat.  I forgot about that.  You might be able to do that, or you can try to disable the asm features.
Title: assembly code and fat binaries
Post by: linmag7 on February 19, 2009, 03:16:42 PM
I thinkt that the compiler defines __sparc__ or __hppa__ or __i386__ or __m68k__ for the appropriate architecture, this can be used by adding #ifdef __xxx__ around asm code.

I have a question regarding fat libraries, the compiler can generate fat object files by supplying -arch xxx argument, but how do I put together a static library from multiple fat object files? do I use ar? When I use "ar cru" and check the resulting library with lipo I get: "lipo: archive member libyyy.a(xxx.o) is a fat file (not allowed in an archive)"

Any suggestions?
Title: Re: assembly code and fat binaries
Post by: kb7sqi on February 19, 2009, 06:52:16 PM
Quote from: "linmag7"I thinkt that the compiler defines __sparc__ or __hppa__ or __i386__ or __m68k__ for the appropriate architecture, this can be used by adding #ifdef __xxx__ around asm code.

I have a question regarding fat libraries, the compiler can generate fat object files by supplying -arch xxx argument, but how do I put together a static library from multiple fat object files? do I use ar? When I use "ar cru" and check the resulting library with lipo I get: "lipo: archive member libyyy.a(xxx.o) is a fat file (not allowed in an archive)"

Any suggestions?

Hey linmag7,
      Sounds like you're running into the same problem as compiling OpenSSL.  I never found a workaround for it.  I had to compile for each -arch seperately & then lipo everything together.  Quick question though, are you using the stock "cc" that comes w/ the NS3.3 developer or are you using the "updated" gcc-2.7.2 for the Sparc?  There's some quirks I know when using that to compile stuff for NS3.3/OS4.2 just for the Sparc.  Take care.
Title: compiler
Post by: linmag7 on February 20, 2009, 01:38:21 AM
I'm using the stock compiler that comes with NS3.3 dev kit, but I have applied the patch for the dev-kit. cc -v says  "version cc-437.2.6, gcc version 2.5.8"
Title: Re: assembly code and fat binaries
Post by: oneNeXT on February 21, 2009, 01:48:51 PM
[quote="kb7sqi" I never found a workaround for it.  I had to compile for each -arch seperately & then lipo everything together.[/quote]

Am i wrong or it is the way that ProjectBuilder use to make Library ?
Title: Re: assembly code and fat binaries
Post by: kb7sqi on February 21, 2009, 09:07:04 PM
Quote from: "oneNeXT"[quote="kb7sqi" I never found a workaround for it.  I had to compile for each -arch seperately & then lipo everything together.

Am i wrong or it is the way that ProjectBuilder use to make Library ?[/quote]

Hi oneNeXT,
   We're discussing using straight cli to compile regular *nix software quad-fat. :-)  You don't have to use ProjectBuilder to build that kind of software.  Normally your "cc" line would look like something like this:
cc -O2 -c *.c -o *.c

Where * is any part of the code.  When compiling "quad-fat" it'd be something like this:

cc -arch i386 -arch m68k -arch hppa -arch sparc -O2 -c *.c -o *.o

Linmag7 has started using a dual-develop system so that way he can compile stuff for NEXTSTEP 3.3 as well as OPENSTEP 4.2.  His build system is a Sparc running OPENSTEP 4.2.  The only differences in our systems is my primary dual-development system is an x86 box running OPENSTEP 4.2.  He's working on trying to get bdb-3.x.x to compile quad-fat & I've had similar problems when trying to compile OpenSSL quad fat.  The only work around I've ever found was compiling everything for single arch's & then lipo the results together to produce a "quad-fat" binary to produce a package that's installable for everyone.   It can be done, it's just more of a pain compared to compiling everything like the above using the -arch flags, everything is compiled & lipo'ed together like it's supposed to be.   There's some software that appears to give the NeXT version of ar/ld some issues.  Hope that clears everything up for you.  :-)

On a side note, Linmag7 I started wondering if trying to get a newer version of "cctools" might help.  I'll start going thru my old archives.  I remember it was possible to build a newer version of cctools from Darwin on x86/m68k.  But I'm not sure on sparc/hppa.  Take care.

Go to top  Forum index