Greetings,
I've had way too much "free" time on my hands lately and one of my projects was playing with some new compiles...
Ultimately I managed to build gcc-3.2.3 on m68k under NS3.3 through all three bootstrap stages, and the compiler works and compares stage2 to stage3 just fine. Note that to build three stages of c, c++, objc and f77 takes almost 13 hours on my turbo color :-)
Unfortunately things somewhat fall apart when it comes to building the libstdc++ libraries. In fact I coaxed the libraries to build with a really brute force approach, but linking c++ fails due to undefined symbols, specifically access(), strxfrm(), strdup() and write(), most of which I believe are in the NeXT posix library. I also get ___gxx_personality_sj0 and ___cxa_call_unexpected as undefined when trying to link the abi_check testsuite program.
Now, I am not a developer or even a programmer. I only used to do some rudimentary C work years in the past when I was working on my mechanical engineering degree. So I am a bit stuck.
It would be great if a real developer or anyone familiar with GCC in general could assist to make this happen. Based on what I have seen I think even newer GCC 3 branches would be possible if we could find a workaround for the broken NeXT libraries.
t-rexky
I'm not a gcc specialist but can you make public what you've already done ?
Maybe i can give a little help !
Thank you for your response! I can most certainly summarize what I have done. Here is a list of the critical items - you will notice that getting the compilers to build is almost trivial, but the runtime libraries are somewhat more demanding.
[list=1]
- The biggest stumbling block was the new pre-processor that the gcc-3 branch introduced. This pre-processor seems to break pretty hard on the NeXT headers, specifically on the ARCH_INCLUDE() macro. Considering that the chance of more architectures being introduced to NEXTSTEP is pretty slim :-), I edited all of the header files to disable the ARCH_INCLUDE() macro and instead explicitly list all architecture. As an example, my ansi/math.h header now looks like this:
/* HISTORY
*
* 2011-11-28 t-rexky
* Disabled the NeXT ARCH_INCLUDE() macro.
*/
#ifndef _ANSI_MACHINE_MATH_H
#define _ANSI_MACHINE_MATH_H
/*
* ARCH_INCLUDE.h provides a facility for including architecture
* specific files without enumerating every possible architecture
* in this file.
*
* #include <architecture/ARCH_INCLUDE.h>
*
* #include ARCH_INCLUDE(ansi/, math.h)
*
* Disable the NeXT ARCH_INCLUDE() macro and instead enumerate
* every possible architecture.
*/
#ifdef m68k
#include <ansi/m68k/math.h>
#elif defined i386
#include <ansi/i386/math.h>
#elif defined hppa
#include <ansi/hppa/math.h>
#elif defined sparc
#include <ansi/sparc/math.h>
#endif /* m68k */
#endif /* _ANSI_MACHINE_MATH_H */
This change to all the affected headers allows the gcc-3 pre-processor to process all the headers. I would be more than happy to supply a TAR of the modified /NextDeveloper/Headers for NS3.3 to save you time.
- As a starting point and to practice I built and installed gcc-2.95.3 (with the headers modified per the above). I also installed most of the kb7sqi's packages, particularly bash, sed, bison, flex and gmake.
- Using bash, I specified the following environment variables:
SHELL=/usr/local/bin/bash
export SHELL
CONFIG_SHELL=/usr/local/bin/bash
export CONFIG_SHELL
- I configured the build from a separate directory as follows:
.../gcc-3.2.3/configure --prefix=/usr/local \
--enable-languages='c,objc,c++,f77' \
--disable-shared --disable-threads --enable-cpp \
--enable-obsolete m68k-next-nextstep3
- If you were to bootstrap the compiler now it would fail while processing the crtstuff.c file into crtend.o because our architecture does not support section attributes. I therefore edited the .../gcc-3.2.3/gcc/crtstuff.c by mixing in some code from an older gcc-3 version to remove the section attribute. This is a complete hack, BTW, but I hope it will do the job:
603,606c603,623
< STATIC int __FRAME_END__[]
< __attribute__ ((unused, mode(SI), section(EH_FRAME_SECTION_NAME),
< aligned(4)))
< = { 0 };
---
>
> /*
> * Revert to previous version of the code to avoid use
> * of section attribute and allow compilation on NeXT
> *
> * STATIC int __FRAME_END__[]
> * __attribute__ ((unused, mode(SI), section(EH_FRAME_SECTION_NAME),
> * aligned(4)))
> * = { 0 };
> */
>
> #ifndef EH_FRAME_SECTION_ASM_OP
> #define EH_FRAME_SECTION_ASM_OP "\t.section __TEXT,__eh_frame,regular"
> #endif /* EH_FRAME_SECTION_ASM_OP */
>
> /* Force cc1 to switch to .data section. */
> static void * force_to_data[1] __attribute__ ((__unused__)) = { };
>
> typedef unsigned int ui32 __attribute__ ((mode (SI)));
> asm (EH_FRAME_SECTION_ASM_OP);
> static ui32 __FRAME_END__[] __attribute__ ((__unused__)) = { 0 };
- Now we are finally ready for the bootstrap, with a few provisos. There are subsequent issues with building the libstdc++ library that probably require some additional tweaks to the gcc configuration for the NeXT, but I do not know enough about gcc. So for now I continued hacking along:
gmake CFLAGS='-O' LIBCFLAGS='-g -O2' \
LIBCXXFLAGS='-g -O2 -fno-implicit-templates' bootstrap
Believe it or not, this will now successfully complete and compare all three stages of the build! But be prepared to wait for 12+ hours if you are doing this on m68k like I was.[/list:o]
Beyond this it gets a bit hairy once cc1plus starts building the runtime library. Now, please be gentle with me, since as I mentioned before I have never written a single line of C++ code. I used to "play" a bit only with C for my engineering work, mostly because I hated Fortran with a passion 8)... Here are my rough notes on what I have done to complete the libstdc++ build. They are in the format of [error_message_excerpt followed by (my_braindead_hack):
../../../../gcc-3.2.3/libstdc++-v3/libmath/signbit.c: In function `__signbit':
../../../../gcc-3.2.3/libstdc++-v3/libmath/signbit.c:39: `ieee_double_shape_type' undeclared (first use in this function)
(brute force #define BIG_ENDIAN 4321 & #define BYTE_ORDER BIG_ENDIAN in mathconf.h)
In file included from ../../../../gcc-3.2.3/libstdc++-v3/libsupc++/eh_alloc.cc:34:
/Storage2/gcc-3.2.3-bld/m68k-next-nextstep3/libstdc++-v3/include/cstring:89: `strxfrm' not declared
(brute force declare size_t strxfrm(char *s1, const char *s2, size_t n); in cstring)
../../../../gcc-3.2.3/libstdc++-v3/libsupc++/pure.cc:49: `write' undeclared (first use this function)
(brute force typedef int ssize_t; typedef unsigned long size_t; extern ssize_t write(int fildes, const void *buf, size_t nbyte); in pure.cc)
/Storage2/gcc-3.2.3-bld/m68k-next-nextstep3/libstdc++-v3/include/cstdio:99: syntax error before `;' token
(brute force #ifdef FILE #undef FILE #endif typedef struct _iobuf FILE; in cstdio)
This next one was fun to figure out and I almost threw in the towel! I could not understand why the compiler was producing Motorola syntax while it was configured with MIT syntax and there was no single occurrence of 'jbne' string in cc1plus! An hour or two later I discovered that the assembler operator was defined in atomicity.h which is oblivious to the Motorola/MIT syntax differences. This is the change that will likely need to be implemented somewhere in the gcc/config NeXT specific files for a permanent fix:
complex_io.cc
/usr/tmp/ccdowAQh.s:1208:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:1520:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:1642:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:1872:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:2425:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:2639:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:3153:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:3367:"Unknown operator" -- Statement 'jbne 1b' ignored
/usr/tmp/ccdowAQh.s:3533:"Unknown operator" -- Statement 'jbne 1b' ignored
(temporary hack - add #ifndef __mc68040__ #define __mc68040__ #endif to ./gcc-3.2.3-bld/m68k-next-nextstep3/libstdc++-v3/include/m68k-next-nextstep3/bits/atomicity.h)
This one I also could not figure out, so I tried another brute force technique and it worked:
In file included from /Storage2/gcc-3.2.3-bld/m68k-next-nextstep3/libstdc++-v3/include/ext/rope:60, from ../../../../gcc-3.2.3/libstdc++-v3/src/ext-inst.cc:34:
/Storage2/gcc-3.2.3-bld/m68k-next-nextstep3/libstdc++-v3/include/ext/stl_rope.h:909:18: macro "index" requires 2 arguments, but only 1 given
(brute force #ifdef index #undef index #endif in stl_rope.h jus ahead of index())
../../../../gcc-3.2.3/libstdc++-v3/src/locale.cc:203: `strdup' undeclared (first use this function)
(replace with _partial_ locale.cc from GCC branch 3.3 CVS#76124 that is not using strdup())
NextDeveloper/Headers/ansi/string.h:52: too few arguments to function `size_t strcoll(char*, long unsigned int, const char*)'
collate_members.cc:46: at this point in file
(real hack!!! replace strcoll() by strcmp() in collate_members.cc)
time_members.cc: In member function `void std::__timepunct<_CharT>::_M_put(_CharT*, long unsigned int, const _CharT*, const tm*) const [with _CharT = char]':
time_members.cc:47: `strdup' undeclared (first use this function)
(added extern char * strdup(char *s); to time_members.cc)
c++locale.cc: In function `void std::__convert_to_v(const char*, _Tv&, std::_Ios_Iostate&, int* const&, int) [with _Tv = float]':
c++locale.cc:123: `strdup' undeclared (first use this function)
(added extern char * strdup(char *s); to c++locale.cc)
../../../../gcc-3.2.3/libstdc++-v3/testsuite/abi_check.cc: In function `int main(int, char**)':
../../../../gcc-3.2.3/libstdc++-v3/testsuite/abi_check.cc:295: `R_OK' undeclared (first use this function)
(brute force include #define R_OK 4 and extern int access(const char *path, int amode); in abi_check.cc)
All of the above hacks will conclude the build of the libstdc++ library, but of course linking anything to it will fail because we are missing access(), strxfrm(), strdup() and write() in the NeXT libsys_p library:
/bin/ld: Undefined symbols:
access(char const*, int)
___gxx_personality_sj0
___cxa_call_unexpected
strxfrm(char*, char const*, unsigned long)
strdup(char*)
write(int, void const*, unsigned long)
_strdup
And this is where I run out of ideas. Obviously we need to create an additional compatibility library that includes these functions, but I have no idea how to merge this with the gcc project to make it work seamlessly. I am also a bit concerned with the linker complaining about __gxx_personality_sj0 and __cxa_call_unexpected being undefined.
I also considered posting on the gcc-help mailing list to see if anyone there can help with how to merge such changes with an existing gcc configuration. Personally I don't care if the gcc source is hacked and not portable after these changes, as long as the resultant compilers all work...
Oh, and why am I doing this all? I hope that availability of a more modern compiler for NEXTSTEP reinvigorates the community a bit to allow porting of some more recent tools. I probably will not be doing this as I do not know enough about software development. But I hope that others might benefit from this.
Thank you!
Quote from: "oneNeXT"I'm not a gcc specialist but can you make public what you've already done ?
Maybe i can give a little help !
Quote from: "t-rexky"
It would be great if a real developer or anyone familiar with GCC in general could assist to make this happen. Based on what I have seen I think even newer GCC 3 branches would be possible if we could find a workaround for the broken NeXT libraries.
I can't answer anything, but maybe Rex Dieter's work is s omething usable? AFAIK egcs is a gcc fork ~2.7, ~2,8 and has later merged back to gcc.
http://www.math.unl.edu/~rdieter1/OpenStep/Developer/
Thank you for the pointer Andreas. I reviewed Rex Dieter's site and also spent a considerable amount of time looking for related info on the net...
I reconfigured the 3.2.3 build to only supply the C compiler, and I have to report that it works fine. In fact, I used it to build a number of other GNU tools and everything seems to be OK. The one thing that I have not yet tested is full support for the (broken) POSIX...
Regarding geting the C++ portion working, I am planning on posting something on the GCC listservers, unless I find some documentation that explains how to include additional functions in the C++ library to satisfy the missing NeXT dependencies.
Unfortunately I have to report that 3.2.3 is likely the latest GCC branch that we can build without a significant effort. I learnt the hard way that GCC 3.3 introduced a completely revamped method for hooking into the system specific stuff... Coincidentally GCC 3.3 removed a huge list of obsoleted systems from its support - no surprise. So if anyone would want to put more effort into building later versions of GCC, we might as well concentrate on GCC4.x.
Now, if we could only fix the POSIX implementation in 3.3, things would be a whole lot simpler to port.
In the meantime I also built some support tools such as gawk, sed, bash, etc. I am going to submit what I built to the file archives for others to use. Everything was built as m68k binaries only though. One day I might try to reconstruct my efforts on OS42 Intel...
Quote from: "Andreas"Quote from: "t-rexky"
It would be great if a real developer or anyone familiar with GCC in general could assist to make this happen. Based on what I have seen I think even newer GCC 3 branches would be possible if we could find a workaround for the broken NeXT libraries.
I can't answer anything, but maybe Rex Dieter's work is s omething usable? AFAIK egcs is a gcc fork ~2.7, ~2,8 and has later merged back to gcc.
http://www.math.unl.edu/~rdieter1/OpenStep/Developer/
Perhaps porting lcc would be worth the effort? It's usually a lot faster and less resource hungry than gcc.
This suggestion sounds quite intriguing... I had a quick look at the lcc site and noticed that it supports the i386 architecture, although with NASM as opposed to GAS. We would be out of luck for the m68k, which the purist in me is currently focusing on :-).
DISCLAIMER: I know absolutely nothing about compiler technologies - just returned to tinkering with code hobby-style...
Quote from: "gtnicol"Perhaps porting lcc would be worth the effort? It's usually a lot faster and less resource hungry than gcc.
Quote from: "t-rexky"Everything was built as m68k binaries only though. One day I might try to reconstruct my efforts on OS42 Intel...
I've started working on intel release...
Awesome news! I've been plugging away at the m68k build, but I am still far away from a full gcc suite. However, as I mentioned already I have been successfully using C only 3.2.3 to build quite a few recent GNU tools, such as bash, bison, flex, m4, gawk, etc...
I am not sure how far along you are, but I would suggest using the info on rdieter's site as an excellent guideline and starting point (thank you Andreas). One other stumbling block might be lacking features of the gas assembler on OS. I noticed that on OS42 Intel, once the compiler starts using the stage1 tools it chokes on cfg.c with the following error message:
/usr/tmp/ccyIsQNm.s:457:Unknown pseudo-op: .quad
Looking at the Apple open source site, .quad was implemented in cctools-590.32, way past OS. So, we might need a franken-gas to go with the new compiler. This issue does not appear on m68k, by the way.
Speaking of which, has anyone ever successfully built the NeXT (g)as from sources? I noticed that some library sources have not been supplied by NeXT, but are present in the Rhapsody sources...
Quote from: "oneNeXT"I've started working on intel release...
Never mind... I just managed to manually build the OS42 'as' from the NeXT supplied sources. The supplied libstuff sources, although incomplete, satisfy all of the i386 'as' dependencies. Hopefully it is not too difficult to roll-in the newer 'as' sources from a recent OS X revision...
Quote from: "t-rexky"Speaking of which, has anyone ever successfully built the NeXT (g)as from sources? I noticed that some library sources have not been supplied by NeXT, but are present in the Rhapsody sources...
I actually accidentally came across some m68k port of lcc yesterday... Now, I have to search through my browser history file to see where it was...
Quote from: "t-rexky"This suggestion sounds quite intriguing... I had a quick look at the lcc site and noticed that it supports the i386 architecture, although with NASM as opposed to GAS. We would be out of luck for the m68k, which the purist in me is currently focusing on :-).
Quote from: "t-rexky"I actually accidentally came across some m68k port of lcc yesterday... Now, I have to search through my browser history file to see where it was...
Quote from: "t-rexky"This suggestion sounds quite intriguing... I had a quick look at the lcc site and noticed that it supports the i386 architecture, although with NASM as opposed to GAS. We would be out of luck for the m68k, which the purist in me is currently focusing on :-).
It wouldn't be this would it?
http://ftp.gwdg.de/pub/misc/languages/lcc/contrib/
From memory there was another project, ldb, which was a retargetable debugger too.
It was actually an early Amiga port that I found originally. Perhaps it is based on the info that you found?
http://ftp.back2roots.org/geekgadgets/amiga/m68k/alpha/lcc/00readmeQuote from: "Noth"
It wouldn't be this would it? http://ftp.gwdg.de/pub/misc/languages/lcc/contrib/
Just for the sake of sharing the knowledge...
After going through a few chainsaws and sledge hammers I managed to build a new franken-as that uses code from Apple's cctools-622.9. And it seems to work fine in building the GCC 3.2.3 binaries. The approach I took was to compare the differences, use all the new code except the write_object module since the new object format would not be recognized by the OPENSTEP linker. I also eliminated all the references to architectures that are irrelevant for us, such as PPC, etc., in order to allow a successful compile.
It's really just an ugly proof of concept rather then a port, but it shows that it can be done.
Quote from: "t-rexky"Never mind... I just managed to manually build the OS42 'as' from the NeXT supplied sources. The supplied libstuff sources, although incomplete, satisfy all of the i386 'as' dependencies. Hopefully it is not too difficult to roll-in the newer 'as' sources from a recent OS X revision...
Quote from: "t-rexky"Speaking of which, has anyone ever successfully built the NeXT (g)as from sources? I noticed that some library sources have not been supplied by NeXT, but are present in the Rhapsody sources...
Please, excuse my ignorance on the matter, but is't it easier to build a cross-compiler that targets NeXTSTEP, rather than porting GCC and all dependencies. OK, maybe not easier, but definitely less time consuming.
I am pretty ignorant on this subject as well, but trying to learn... From what I understand, the challenge with a cross-compiler is that one needs the "binary tools" to create the native object files and executable files. So we would need full sources for 'as' and 'ld' to make executables that run in the cross-compile environment to produce native NS/OS code.
Normally this is not an issue because the GNU binutils are used for the cross-compile environment. However, the NS/OS as well as the current OS X use heavily modified or completely custom 'as' and 'ld'. The original NS and OS Developer distributions included source for 'as' as it was based on gas-1.38, but no source whatsoever for 'ld'. Apple started supplying the source for 'ld' some time in the Darwin era. At this point the executable format and object file format was changed and will not be compatible on NS/OS.
Now, perhaps it would be possible to glue together a linker that is compatible with NS/OS from the more modern Apple 'ld' sources, but it would probably be much more of a challenge than getting an updated assembler that I was able to glue together.
What I am really itching to do is port a much more modern GCC compiler and perhaps the glibc libraries to allow porting of modern software. But of course this is way beyond my skill level. However, since I now managed to create working executables for gcc-3.2.3 on m68k and i386, it can obviously be done by someone who is more familiar with gcc.
I am currently side-tracked into compiling recent versions of bash, bison, sed, gawk and binutils. I have also been struggling to understand why I can build Perl 5.6.2 that passes all but 3 tests fine in the build directory, but fails dynamic loading once installed into /usr/local. (I have the same problem with the kb7sqi's Perl binary package on my m68k NS33 and i386 OS42).
However, once the above is done (or I give up on it), I plan on getting back to gcc-3.2.3 to get the C++ libraries ported.
Quote from: "iDork"Please, excuse my ignorance on the matter, but is't it easier to build a cross-compiler that targets NeXTSTEP, rather than porting GCC and all dependencies. OK, maybe not easier, but definitely less time consuming.
Quote from: "t-rexky"
I reconfigured the 3.2.3 build to only supply the C compiler, and I have to report that it works fine. In fact, I used it to build a number of other GNU tools and everything seems to be OK. The one thing that I have not yet tested is full support for the (broken) POSIX...
This is really great news! a more recent compiler is really the key to building a bunch of new stuff. How m68k specific is this? is it a big step to get gcc-3.2.3 working for sparc/intel as well?
A while back I worked on some ports of unix software to NextSTEP I put together a "libnext" with a collection of posix functions that were missing or faulty in nextstep, it wasn't much but fixed a lot of unresolved symbols when linking standard unix utilities...
I cannot say about sparc, but I managed to hack an Intel build almost as far as m68k. There was a fundamental challenge with the Intel build because the assembly code generated by gcc-3.2.3 was breaking the OPENSTEP supplied 'as'. I managed however to hack together a new 'as' by combining the OPENSTEP 'as' sources with Apple's cctools-622.9. And shockingly it works and produces good code. So I was able to build and successfully compare gcc-3.2.3 binaries for C only. But I stopped there since
oneNeXT offered to help with the Intel port. I will submit some of what I have done once I have access.
I did follow everything here from you and also from kb7sqi - thank you for all your efforts! I also spent an incredible amount of time going through the old Usenet posts, which provide a wealth of information. In fact it would be great to produce an indexed archive of all NeXT Usenet info with the Librarian, but that's a task for a different time. I am not a programmer, but I used to do a bit of hacking in the past and I enjoy this kind of puzzles :-). I managed to build a few fairly recent GNU tools and they pass most if not all of the supplied tests. I also almost finished TCL 8.4.19 that at this point passes all but 11 tests! And I almost have latest expect working as well, all in an effort to try to get dejagnu working...
I found that once I have gone through a few ports things got much easier, as the issues are very similar. The biggest challenge now is that things compile and run, but the NeXT libraries are so out of date and sometimes broken that the tests fail for mysterious reasons. For example, with TCL I had to replace the NeXT string libraries (sprintf, fprintf, sscanf) with strio string libraries, and that made most of the test failures disappear.
It would be spectacular if we could get enough interest in the community to finish a more recent GCC and GLIBC port! I do realize that GLIBC would be a huge effort, but it would also provide a huge benefit. I think the above would allow most of the recent software to be compiled cleanly, or mostly so.
I would be interested in what you have included in your libnext as well as any notes you might have on porting. I have been putting together a similar collection of functions and notes, but it's also very limited.
Quote from: "linmag7"
This is really great news! a more recent compiler is really the key to building a bunch of new stuff. How m68k specific is this? is it a big step to get gcc-3.2.3 working for sparc/intel as well?
A while back I worked on some ports of unix software to NextSTEP I put together a "libnext" with a collection of posix functions that were missing or faulty in nextstep, it wasn't much but fixed a lot of unresolved symbols when linking standard unix utilities...
Hi,
Sorry for the delay i my response I've been less active for the past year or so, maybe its time to check in more often.
A more recent glibc along with a recent gcc would really bring nextstep into 2012 in a big way, maybe also a posix-threads library would be required for some compiles.
I agree the porting glibc to NeXT is no small task, not only does it requires some true skill but also a significant amount of time.
my "next library" contains an implementation of a very small subset of the common c-library functions that are either missing or broken in NeXTSTEP. The following functions are implemented:
mkfifo
munmap
mmap
munmap
mprotect
setpgid
rewinddir
uname
putenv
setenv
glob
getcwd
sigpending
sigdelset
sigismember
sysconf
setsid
waitpid
sigemptyset
sigfillset
sigaddset
sigaction
sigprocmask
sigsuspend
strdup
snprintf
strchrnul
strdup
strptime
tempname
strxfrm
tcgetattr
tcsetattr
I'd be happy to make the library available to you if you'd like to use it or add more implementations to it.
Quote
... maybe also a posix-threads library would be required for some compiles.
There is a small pthreads library implemenentation that use cthreads. it is maybe no complete but it may be used for starting a libpthread.
I'm also particulary interrested by your implementation of snprintf, would it be possible to get it ?
I'll put "libnext" on an ftp sometime tomorrow, PM me if you want the login to grab it. Its not well documented and not even fully tested, but it has been useful to may a few software packages build. Its mostly glibc stuff that I've used.
Hi,
That sounds awesome. I was able to collect a small subset of the functions you listed, but a more complete library would be great. If you agree, I can submit this to Nitro to have it posted in the New Ports area of the file archive...
One thing I was struggling with is how to structure the replacement headers and library such that it is almost transparent to the code. Right now I jus redefine the proper functions names to something unique, but it gets a bit hairy with packages like Perl and TCL that load their own modules against the built-in and library functions.
Thanks again!
Quote from: "linmag7"Hi,
Sorry for the delay i my response I've been less active for the past year or so, maybe its time to check in more often.
A more recent glibc along with a recent gcc would really bring nextstep into 2012 in a big way, maybe also a posix-threads library would be required for some compiles.
I agree the porting glibc to NeXT is no small task, not only does it requires some true skill but also a significant amount of time.
my "next library" contains an implementation of a very small subset of the common c-library functions that are either missing or broken in NeXTSTEP. The following functions are implemented: [...]
I'd be happy to make the library available to you if you'd like to use it or add more implementations to it.
I'm not sure what the best approach to header files is. When I was porting stuff to nextstep I collected missing header stuff into new headers that I included into whatever source-files needed them. This was in order to leave the original nextstep header files intact.
Maybe a different approach would be to create a patch-kit that could be applied to original nexstep headerfiles in order to add/fix missing stuff to those files?
I am actually tempted to do at least some cleaning-up on the NeXT header files, if nothing else then just to prevent multiple includes... It is quite a bit of work though, if done manually - the "fixed" headers that I created for gcc-3.2.3 took almost a day of work.
Quote from: "linmag7"I'm not sure what the best approach to header files is. When I was porting stuff to nextstep I collected missing header stuff into new headers that I included into whatever source-files needed them. This was in order to leave the original nextstep header files intact.
Maybe a different approach would be to create a patch-kit that could be applied to original nexstep headerfiles in order to add/fix missing stuff to those files?
Sure thing. I'll give your suggestion a try when I get a chance. Right now I've messed my build chain up something fierce, so It's gonna take some serious work just to get it back where it was. I'll keep you updated.
I was finally able to complete the build process of g++ on m68k NS33, including building and (manually for now) linking abi_check.cc! I was getting undefined symbols during the linking process because libsupc++ is not being automatically linked by g++ the way it should.
As a side note, I now also have gcc-3.2.3 (C only) working on intel OS42, but there are some really "strange" things happening that I do not understand during the build process. Irrespective, the final binaries appear to be working just fine.
Remaining to do, from m68k perpective:
[list=1]
- Fix the libstdc++ configuration such that libsupc++ is automatically linked by g++
- Figure out how to implement section attribute, since NS33 and OS42 do support it - this will allow use of the original crtstuff.c
- Customize the configuration files of libstdc++ for the NeXT
- Finish the port of recent tcl, expect and dejagnu so that we can run the testsuite (I have tcl and expect mostly working)
- Get a volunteer to test the C++ port since I don't know anything about C++[/list:o]
I am also in the process of studying the configuration changes that were made to gcc after 3.2.3 such that an even more recent version can be ported. I am hoping that this is doable for 3.4.6 or even perhaps 4.3.6.
After that it would be really nice to get a small group of people to perhaps try to port a recent libgcc and gdb so that all the good new code can be debugged natively on the NeXT. And to finish the i386 port.
Incredible work.
Thank you, I just wish I knew more about gcc and programming in general, although I am learning quite a bit throughout this exercise.
Are there any volunteers to try porting glibc to NS and OS? :wink:
I spent a bit more time on the compiler and made some good progress.
The libstdc++ issues were related to libtool and a 15 character limit for object file names on NS33 that 'ar' simply truncates. Once truncated, the libtool does not pick them up since they no longer have the ".o" extension. I implemented a workaround (thank you Daniel Richard G.) that fixes this and libstdc++ builds correctly. I am now able to build C++ executables that link and execute.
I was also able to finish my port of recent tcl, expect and dejagnu that is robust enough to be able to run automated test cases. The good news is that it works, and the bad news is that libstdc++ fails some tests, presumably because of its reliance on the outdated NeXT system libraries.
I put the implementation of arbitrary sections aside for now, as it is not critical to the overall function of the compiler with the tweaked crtstuff.c.
I also have some very unexpected good news - I am no longer working on version 3.2.3. I spent a few days tinkering with the GCC configuration files and I now have a bootstrapped & fully working gcc-3.4.6 that includes c, c++, objc and f77! I am currently running a 'make check' on the C portion, and then will see how badly the C++ portion behaves.
My plan is to spend some more time on 3.4.6 to make it as solid as possible, and then to move the configuration over to gcc4 and see how much damage I can do with it :-). After that I would like to try a proper OPENSTEP build on m68k that includes Mach-O PIC support. Then perhaps Intel, unless someone else picks up the flag.
I will also need to revise my fixed system headers, since I naively used non ANSI definitions in the #ifdef statements (such as m68k instead of __m68k__) and compiling with the -ansi flag does not pick up the required headers.
I still want to get a few people interested to port a recent (e)glibc so that we can more easily port all modern GNU tools.
Quote from: "linmag7"I'm not sure what the best approach to header files is. When I was porting stuff to nextstep I collected missing header stuff into new headers that I included into whatever source-files needed them. This was in order to leave the original nextstep header files intact.
Maybe a different approach would be to create a patch-kit that could be applied to original nexstep headerfiles in order to add/fix missing stuff to those files?
I have been doing quite a bit of tinkering (still) and lots of learning. In the process, I came up with an idea for "blending in" some updated functions and introducing new functions into the existing NEXTSTEP libraries. This will also work on OPENSTEP, but perhaps there is a better way with shared libraries that I don't (yet) know much about? In any case, here are my thoughts and I would appreciate some comments on the viability of this approach from real programmers.
[list=1]
- Custom-configure GCC such that INCLUDE_DEFAULTS defines new header directories located at /LocalDeveloper/LibnxHeaders just before /NextDeveloper/Headers. This will allow us to install modified or new headers into this new directory without affecting the NeXT cc tools. The NeXT cc will continue using the factory directories, while our new gcc will satisfy its header dependencies from the new directories before searching the factory directories.
- Create a new libnext.a library with new functions as well as replacement for existing but broken/incompatible functions. This would include all the functions that linmag7 supplied in his library, plus full replacement for printf/scanf functions, strcoll(), mmap() & munmap() and many others. All these functions would have nx_ prepended to them, for example strcoll() would be nx_strcoll(). This new namespace will allow linking against the existing NeXT libraries without duplicate symbol warnings.
- Change the GCC LIB_SPEC to automatically link against libnext.a when -posix argument is not specified to gcc. I think we can safely ignore doing anything with -posix argument since the implementation is so broken on NEXTSTEP and does not exist on OPENSTEP. Our new library will provide some some POSIX support by default.
- Introduce new header files into /LocalDeveloper/LibnxHeaders that define all new and overriding features plus all the existing NeXT library features that we want to use. Here we can use the wonderful GCC #pragma redefined_extname oldname newname, to replace function names at assembly level. For example:
#pragma redefined_extname strcoll nx_strcoll
placed in new string.h will replace all references in our newly compiled code to strcoll() with nx_strcoll(). I noticed that gcc-4.x uses this approach with libmudflap and a light bulb illuminated in my head.
- And that's all folks! It is quite a bit of work, but I can continue chipping away at it in my spare time.[/list:o]
I would appreciate some comments on the above proposal. If it works and is feasible, it would allow us to fix some of the biggest deficiencies in the NEXTSTEP / OPENSTEP libraries without the need for porting a completely new library such as glibc / eglibc.
Looking forward to some comments...
Just wanted to say thanks for the hard work. I wish I could give insightful comments, but you've really way beyond my level of understanding on this one.
Quote from: "gregwtmtno"Just wanted to say thanks for the hard work. I wish I could give insightful comments, but you've really way beyond my level of understanding on this one.
Thanks!
I spent quite a bit more time with gcc and I was able to translate the target configuration files over to gcc-4.6. I am able to build a stage 1 compiler that appears to work, except for the table jumps that are used in switch/case statements. They cause bus errors and I am not certain if it is the code generation or the NeXT assembler itself.
I spent a considerable amount of time on the m68k debian ports mailing list and with the help of the knowledgable people there I was able to find a bug in the newer gcc codebase that was causing internal compiler errors with the NeXT target configuration.
Unfortunately at this point I am a bit stuck due to my lack of knowledge of the MIT m68k assembly and gcc internals... Still trying, but not hopeful that I will be able to create a patch - that part of gcc is just way beyond my knowledge level.
NB - the same problem happens on gcc-4.4 and 4.2. I have not tried 4.1 yet as it needs yet another change to the target configuration files and it will take me some time.
t-r
- UPDATE -
Of course shortly after I posted the above I was able to figure out what was ailing gcc-4.2.4. It is now (finally) happily crunching through the stage 2 build and jump tables are working. Fingers crossed!
Excellent. Keep us updated. I feel like getting a modern compiler going could open the door to a lot more ports.
Quote
- UPDATE -
Of course shortly after I posted the above I was able to figure out what was ailing gcc-4.2.4. It is now (finally) happily crunching through the stage 2 build and jump tables are working. Fingers crossed!
I feel the same way about availability of a more modern compiler. Which is exactly why I am obsessing to get one going...
I now have gcc-4.2.4 completing the bootstrap process on NS33 m68k. There are still some weird issues with the compiler that I do not understand, but just yesterday I was able to make a small step in the right direction in identifying the root cause. For anyone interested, here are some posts that I made to the gcc list looking for some help with the strange warning that my current build generates:
http://gcc.gnu.org/ml/gcc/2012-06/msg00167.htmlOn a related note, many thanks to Thorsten Glaser, Vincent Riviere and Andreas Schwab on debian-68k for their help! I would not be able to identify the jump table issue without them.
On m68k, gcc-4.2.4 is the most recent version that I can get going without diving into the machine description files for the m68k cpu. This is because the NeXT was the only gcc target that used absolute case vectors while all other machines used PC relative case vectors. The m68k.md was substantially rewritten at gcc-4.3 and the compiler generates some PC relative case vector assembly that the NeXT assembler does not understand, causing bus errors. One way forward with this would be to attempt a port of a more modern gas, but it would be a very significant effort. Another possibility would be to patch the m68k.md and m68k.c files, but that's way beyond me.
The "small step forward" yesterday allowed me to determine that the warning messages I have been getting from stage 2 & 3 compilers are related to some of the optimization options. Now I have to determine which one, but it will take a while. This is because the bootstrap process of gcc-4.2.4 takes on the order of 3 days on my Turbo Color :-). And the gdb version supplied with NS33 is too ancient to cope with much of the code within gcc, it would seem. So here is another project - porting a newer version of gdb :-)
t-r
Quote from: "gregwtmtno"I feel like getting a modern compiler going could open the door to a lot more ports.
and save a lot of work and time.
Replacing designated initializers is a pain.
After quite a bit of tinkering I was able to cleanly build quad-fat cctools-435.4 from OS X 10.2.8 on NEXTSTEP 3.3. The only significant item that does not build due to lack of static system libraries is 'dyld'. This is not a big loss however, since NEXTSTEP uses static libraries and shared (shlib) libraries. Perhaps one day we will be able to build dyld as well...
I consider this HIGHLY EXPERIMENTAL and would describe the "port" as OS X cctools running on NEXTSTEP rather than a true port of cctools to NEXTSTEP. However, the fascinating aspect of OS X lineage is that the Mach-O files produced and manipulated by these tools appear to function perfectly on NEXTSTEP. I have done some rudimentary testing on i386 and m68k architectures but I do not have access to SPARC or HPPA hardware. I was able to successfully build numerous packages using these tools, including rebuilding the tools themselves and building the NeXT cc compiler. This exercised 'as', 'libtool' and 'ld'. I also did some ad-hoc testing using 'otool', 'strings', 'nm', and others with good success.
This package has been built on NEXTSTEP 3.3 with NeXT cc compiler modified to support __private_extern__ and -dynamic compiler features. A reasonably recent version of GNU make is also required, such as Apple's gnumake-108. This package will not build with the standard NEXTSTEP 3.3 compiler without additional code modifications.
I tried even more recent cctools but they were producing executable files that failed to run on NEXTSTEP (i386). However, just a few days ago I was able to determine the source of this issue and I also have cctools-622.9 from OS X 10.4.11 creating object files and executables that appear to function correctly on NEXTSTEP 3.3. This newer version, however, will take more time to clean-up and package.
For the adventurous types, I will be uploading the package to the software archive here and it should eventually find its way into the New_Ports directory.
:shock: Congratulations! This sounds really great and quite exciting!
Let us keep posted with your progress.
wow! More incredible work t-rexky. I can't wait to try it out.
Hi T-rexky,
Has your most recent GCC port been made available for download?
Hi,
The most recent files that I submitted are in the following directory:
http://www.nextcomputers.org/NeXTfiles/Software/New_Ports/I have not submitted anything more recent because I am a bit stuck in a chicken-and-egg situation with the ancient NeXT libraries. I was also not able to connect with Nitro for renewed ftp access.
I believe that the library is responsible for a number of failures that I see with the test suite and I would like to fix this. I hope to "blend-in" a significant portion of BSD 4.4 library functions into the NS 3.3 library. However, this is a much larger endeavour.
I am hoping to put together a somewhat more complete build of gcc-3 this winter when I have a little bit more time. This would likely be without any refreshed libraries. I need to collect all the notes and the various test builds that I have on my slab and move it all onto an SVN server that I recently set-up. This will help significantly with revision control.
With respect to making some of the stuff available, I have been trying to put a simple web site together but with no prior experience it has been a bit of a struggle. I resorted to iWeb, but I think it's too awkward for what I wanted, for example hierarchical menus. I am also trying to find an inexpensive hosting service.
In addition to all the above I have been dealing with some ongoing medical issues so it has been very hard for me to concentrate and stay focused on the tinkering. But I have not lost the drive to do it!
Peter.
Hi t-rexky
Thanks for the update. I will give your previous version a try. I have a shared hosting account, so if you are unable to contact nitro maybe we can mirror anything you want there until you get access here. I'll look forward to your progress. I've been considering the possibility of creating something similar to busybox for the next. Basically a single low resource binary with scaled down versions of modern shell tools.
Quote from: barcher174Hi t-rexky
Thanks for the update. I will give your previous version a try. I have a shared hosting account, so if you are unable to contact nitro maybe we can mirror anything you want there until you get access here. I'll look forward to your progress. I've been considering the possibility of creating something similar to busybox for the next. Basically a single low resource binary with scaled down versions of modern shell tools.
Thank you very much for the offer! For now I created a Dropbox account and will use it to "host" some of the files.
I just placed my build of Apple's cctools-435.4 for NEXTSTEP 3.3 at the following link, for those who would like to tinker with it. Please let me know if these work for you, particularly on the hppa and sparc platforms.
https://www.dropbox.com/s/n028mygoss5iay2/cctools-435.4%7Ens1.NIHSs.tar.gz (
https://www.dropbox.com/s/n028mygoss5iay2/cctools-435.4%7Ens1.NIHSs.tar.gz)
cctools-435.4~ns1.NIHSs.tar.gz A note of caution that this port is highly experimental, but the tools seem to work fine for me on m68k and i386. The Installer package will overwrite the corresponding system files but will keep a backup in the Receipts directory. When uninstalled, your system will be restored to the original pre-install configuration. The patch file is included as well, but the source will not compile cleanly with stock NS3.3 compiler due to lack of support for __private_extern__ and -dynamic compiler features, as explained in one of my earlier posts.
Once again, if anyone with a real programming background is interested in helping with any of the porting efforts it would be much appreciated! It is hard to keep the enthusiasm going if it's just me, myself and I :wink:
Glad to see you're still working on this. I, on the other hand, have messed up by system beyond repair. I'll try again from scratch one of these days.
Thanks for sharing it, I hope I will be soon able to contact you from a custom app ...
Quote from: "oneNeXT"Thanks for sharing it, I hope I will be soon able to contact you from a custom app ...
This is really peeking my interest :-). Tell us more!
Just a clue ...
http://www.nextcomputers.org/forums/viewtopic.php?t=3267In few words, pop3 seems to be in a good way, smtp support is started, imap should come later.
Thanks to libcurl (
http://curl.haxx.se/) !
Quote from: "oneNeXT"Just a clue ...
http://www.nextcomputers.org/forums/viewtopic.php?t=3267
In few words, pop3 seems to be in a good way, smtp support is started, imap should come later.
Thanks to libcurl (http://curl.haxx.se/) !
oneNeXT,
It's been ages since I've had time to work on my "NeXT" like I used to, but curl/libcurl don't take too much to get compiled on the NeXT. The last version I compiled clean ie not using -posix w/ the stock NS 3.3 compiler was 7.25 or 7.26 if I remember off the top of my head. If you check out the antic thread of stuff I had ported & got working on the NeXT, you'll find pretty much all the detaills on getting the stock NeXT email app working w/ fetchmail, stunnel, msmtp. I'm in the process of closing on buying a house. If you need more info, I'll post a follow up next week after things get settled down a bit. For those wondering, I do still have all my NeXT gear. ;-) I just tend to do most things in a VM these days. Hope all is well. Take care
t-rexky,
I've read your thread off/on when I've had time over the last few months. I have to say kudos to you! I'd love to see a modern compiler working on the NeXT hardware. In all the time I've been porting things, I've always stayed w/ the stock compiler w/ NS 3.3 for the simple fact that there's some strange bugs when compiling things for the sparc/hppa architectures that you don't see on m68k/i386. Again, one I get through this move this week, I'd be interested in BS'ing one day/night on irc or something. :-) Take care.
Steve, kb7sqi
Cool oneNeXT - great effort!
kb7sqi - my efforts have been a bit stalled lately with too many things on my plate both personally and professionally. However, I am still hoping to chip away at the compiler when I have time so it would be cool to chat at some point if you think you could spare some time. I would like to re-bootstrap the compiler on m68k with the OS X cctools just to see how it affects the test results. I also acquired a Quadra 660av in mint condition (from the original owner), replaced all the power supply and main board electrolytic capacitors and installed NetBSD/m68k on it. I bootstrapped GCC 3.2.3 on it and run the complete suite of tests to see how the failures compare with my NeXT build. There are just so many things to do on many different fronts that I feel scattered all over the place.
Coincidentally, to add to the scatter, in the last few months I also added an HP 9000 Model 715/100XC and a SPARCstation 20 to my collection. The 715 has now been completely cleaned and refurbished with all new capacitors in the power supply (they were about to go) and ready for NS33 installation. The SS20 still needs a PSU refurbishment plus it is giving me a bit of grief with getting the display to work. Ultimately I will be able to test all four platforms at some point.
And now I need a complete remodelling of my small 'cave' room in the basement since the various hardware is scattered all over every horizontal surface.
Let me know Steve when and how you wanted to connect once you settle into your new home.
Peter (t-rexky).
Quote from: "kb7sqi"
oneNeXT,
It's been ages since I've had time to work on my "NeXT" like I used to, but curl/libcurl don't take too much to get compiled on the NeXT. The last version I compiled clean ie not using -posix w/ the stock NS 3.3 compiler was 7.25 or 7.26 if I remember off the top of my head. If you check out the antic thread of stuff I had ported & got working on the NeXT, you'll find pretty much all the detaills on getting the stock NeXT email app working w/ fetchmail, stunnel, msmtp. I'm in the process of closing on buying a house. If you need more info, I'll post a follow up next week after things get settled down a bit. For those wondering, I do still have all my NeXT gear. ;-) I just tend to do most things in a VM these days. Hope all is well. Take care
Hi kb7sqi,
that's very nice to have some news of you, you're still near and that's great!
Is this the thread you're talking about:
http://www.nextcomputers.org/forums/viewtopic.php?t=1154It's good to see that Mail.app can still be used by updating some of the system component.
But I would like to get my own access to the mail for my need.
And maybe share a working new application.
The last release of Curl I've been able to compile is 0.7.32
I should try with 0.7.35, and for NeXTmail support compression must be managed.
I believe that I found a solution...
Another point is the secure access to mail server and for that, openssl is needed
Do you plan to update your openssl package , or show how you did it ?
Take care, you too .
Quote from: "t-rexky"Cool oneNeXT - great effort!
kb7sqi - my efforts have been a bit stalled lately with too many things on my plate both personally and professionally. However, I am still hoping to chip away at the compiler when I have time so it would be cool to chat at some point if you think you could spare some time. I would like to re-bootstrap the compiler on m68k with the OS X cctools just to see how it affects the test results. I also acquired a Quadra 660av in mint condition (from the original owner), replaced all the power supply and main board electrolytic capacitors and installed NetBSD/m68k on it. I bootstrapped GCC 3.2.3 on it and run the complete suite of tests to see how the failures compare with my NeXT build. There are just so many things to do on many different fronts that I feel scattered all over the place.
Coincidentally, to add to the scatter, in the last few months I also added an HP 9000 Model 715/100XC and a SPARCstation 20 to my collection. The 715 has now been completely cleaned and refurbished with all new capacitors in the power supply (they were about to go) and ready for NS33 installation. The SS20 still needs a PSU refurbishment plus it is giving me a bit of grief with getting the display to work. Ultimately I will be able to test all four platforms at some point.
And now I need a complete remodelling of my small 'cave' room in the basement since the various hardware is scattered all over every horizontal surface.
Let me know Steve when and how you wanted to connect once you settle into your new home.
Peter (t-rexky).
Hey t-rexky,
It's taken a while longer than expected, but I'm still around. :-) I got a chance to try your new cctools package yesterday. Nice work! I quickly ran into a problem after using it to compile a few packages though. Everything works fine until you try to link to the libraries w/ the stock linker on say another system. I expected that, but was hoping it would still work. ;-) Testing confirmed it though. I'm working this weekend, but I'd still like to meet up on IRC or something in the near future. Hope all is well! Take care.
Steve, kb7sqi
Quote from: "oneNeXT"Quote from: "kb7sqi"
oneNeXT,
It's been ages since I've had time to work on my "NeXT" like I used to, but curl/libcurl don't take too much to get compiled on the NeXT. The last version I compiled clean ie not using -posix w/ the stock NS 3.3 compiler was 7.25 or 7.26 if I remember off the top of my head. If you check out the antic thread of stuff I had ported & got working on the NeXT, you'll find pretty much all the detaills on getting the stock NeXT email app working w/ fetchmail, stunnel, msmtp. I'm in the process of closing on buying a house. If you need more info, I'll post a follow up next week after things get settled down a bit. For those wondering, I do still have all my NeXT gear. ;-) I just tend to do most things in a VM these days. Hope all is well. Take care
Hi kb7sqi,
that's very nice to have some news of you, you're still near and that's great!
Is this the thread you're talking about:
http://www.nextcomputers.org/forums/viewtopic.php?t=1154
It's good to see that Mail.app can still be used by updating some of the system component.
But I would like to get my own access to the mail for my need.
And maybe share a working new application.
The last release of Curl I've been able to compile is 0.7.32
I should try with 0.7.35, and for NeXTmail support compression must be managed.
I believe that I found a solution...
Another point is the secure access to mail server and for that, openssl is needed
Do you plan to update your openssl package , or show how you did it ?
Take care, you too .
Hey oneNeXT,
I agree, it's pretty easy to still use Mail.app w/ fetchmail, msmtp, stunnel, etc. I've setup things multiple ways testing different email servers. Even using UUCP over SSH/SSL. :-) Sendmail still compiles pretty easily on a NeXT as well. It just uses the -posix flag. The last few versions of Postfix take more work. It's funny, that used to compile pretty much out of the box. OpenSSL posted a new release yesterday & I took the time to compile it quad-fat and linked it against zlib-1.2.8 like I've done in the past. OpenSSL reallly don't take much effort to compile cleanly on a NeXT. It only takes like 3-4 small little mods to the code. The only hassle in compiling it is you have to compile it 4 times (once for each architecture) & then lipo everything together to make a quad-fat package. If you're using a slow system, it can be pretty painful. lol. I compile it in a VM on a fast machine, so it only takes a few minutes. ;-) I've posted the OpenSSL/Z-Lib combo package along w/ other packages here:
https://t.co/CNszTmVHvXI'll post a quick how-to in my old thread here shortly though how to compile OpenSSL so you can reference it & compile it yourself. Take care.
Steve, kb7sqi
Quote from: "kb7sqi"Hey t-rexky,
It's taken a while longer than expected, but I'm still around. :-) I got a chance to try your new cctools package yesterday. Nice work! I quickly ran into a problem after using it to compile a few packages though. Everything works fine until you try to link to the libraries w/ the stock linker on say another system. I expected that, but was hoping it would still work. ;-) Testing confirmed it though. I'm working this weekend, but I'd still like to meet up on IRC or something in the near future. Hope all is well! Take care.
Steve, kb7sqi
Hey, and thanks for your kind words.
I cannot recall if I tried libraries created with the new BSD 4.4 based 'ar' with the native NeXT cctools. I thought that the way they implemented support for long member file names was compatible with the BSD 4.3 format, but I may be wrong. The support for long member file names is required to compile recent GCC and other more complex packages (see this interesting post:
http://lists.gnu.org/archive/html/bug-libtool/2011-11/msg00002.html).
Issues like these contribute to my efforts being stalled, as I mentioned earlier. In order to progress with the porting efforts a unified strategy is required for modernized tools, libraries, etc. All ports compiled with this new setup will require a number of pre-requisites to run. I started thinking about how to structure it all, but I am too inexperienced with software development and too concerned that I make an incorrect assumption or an incorrect implementation. This has to be well planned to work long term and I don't have the knowledge to do it.
It would be great to chat when you are available!
Hi kb7sqi,
Quote from: "kb7sqi"
Hey oneNeXT,
I agree, it's pretty easy to still use Mail.app w/ fetchmail, msmtp, stunnel, etc. I've setup things multiple ways testing different email servers. Even using UUCP over SSH/SSL. :-) Sendmail still compiles pretty easily on a NeXT as well. It just uses the -posix flag. The last few versions of Postfix take more work. It's funny, that used to compile pretty much out of the box. OpenSSL posted a new release yesterday & I took the time to compile it quad-fat and linked it against zlib-1.2.8 like I've done in the past. OpenSSL reallly don't take much effort to compile cleanly on a NeXT. It only takes like 3-4 small little mods to the code. The only hassle in compiling it is you have to compile it 4 times (once for each architecture) & then lipo everything together to make a quad-fat package. If you're using a slow system, it can be pretty painful. lol. I compile it in a VM on a fast machine, so it only takes a few minutes. ;-) I've posted the OpenSSL/Z-Lib combo package along w/ other packages here:
https://t.co/CNszTmVHvX
It's nice to hear from you !
Thanks a lot to update the OpenSSL package and make it available, I hope to try it soon with other lib or in an App.
Quote from: "kb7sqi"
I'll post a quick how-to in my old thread here shortly though how to compile OpenSSL so you can reference it & compile it yourself. Take care.
Steve, kb7sqi
Thanks again, there are some libs that I would make as package and that may help me to do so
Quote from: "t-rexky"Quote from: "kb7sqi"Hey t-rexky,
It's taken a while longer than expected, but I'm still around. :-) I got a chance to try your new cctools package yesterday. Nice work! I quickly ran into a problem after using it to compile a few packages though. Everything works fine until you try to link to the libraries w/ the stock linker on say another system. I expected that, but was hoping it would still work. ;-) Testing confirmed it though. I'm working this weekend, but I'd still like to meet up on IRC or something in the near future. Hope all is well! Take care.
Steve, kb7sqi
Hey, and thanks for your kind words.
I cannot recall if I tried libraries created with the new BSD 4.4 based 'ar' with the native NeXT cctools. I thought that the way they implemented support for long member file names was compatible with the BSD 4.3 format, but I may be wrong. The support for long member file names is required to compile recent GCC and other more complex packages (see this interesting post: http://lists.gnu.org/archive/html/bug-libtool/2011-11/msg00002.html).
Issues like these contribute to my efforts being stalled, as I mentioned earlier. In order to progress with the porting efforts a unified strategy is required for modernized tools, libraries, etc. All ports compiled with this new setup will require a number of pre-requisites to run. I started thinking about how to structure it all, but I am too inexperienced with software development and too concerned that I make an incorrect assumption or an incorrect implementation. This has to be well planned to work long term and I don't have the knowledge to do it.
It would be great to chat when you are available!
I've been trying to give this some thought this week in my free time. Ideally it would great to replace some of the broken libs with routines that work properly. I was thinking about your progress w/ shared libs. That'd also come in handy when it comes to compiling stuff that links against tons of stuff like OpenSSL, Zlib, etc. Think about how much time it takes to re-link all the stuff which is compiled statically. Some people only can/want to run NEXTSTEP 3.3. Others prefer to run OPENSTEP 4.2. With OPENSTEP 4.2, there's the huge advantage of being able to use dylibs/frameworks which helps get rid of the problems when compiling statically. There's the problem w/ POSIX stuff.
I spent sometime trying to find a way to resolve the issues w/ using your newer cctools and still keep things compatible w/ the BSD4.3. There's supposed to be a way to "convert" the libs. That's as far as I got before going to work. Years ago, I had re-compiled the cctools from Rhapsody for x86/m68k. It works, but besides maybe some bug fixes, I think the time/effort you've done w/ the newer cctools would provide the most benefit when it comes to "more modern" code. :-) I need to spend some time testing your cctools on my Gecko/Sparc systems though. I haven't had time to do that yet. I know when it comes to the Gecko, there might be some issues trying to compile a newer gcc. There's some stuff missing there compared to the other arch's. I'm not sure why NeXT did that. I wonder if that's why they never "officially" released OPENSTEP 4.2 for hppa despite it being developed & even passed onto some customers in Europe. Anyway, I'm going to try to look into the converting libs today I think. :-)
If you're on Google Talk, feel free to hit me up sometime. Same nick @ gmail. :-) Take care!
Steve
Quote from: "kb7sqi"I've been trying to give this some thought this week in my free time. Ideally it would great to replace some of the broken libs with routines that work properly. I was thinking about your progress w/ shared libs. That'd also come in handy when it comes to compiling stuff that links against tons of stuff like OpenSSL, Zlib, etc. Think about how much time it takes to re-link all the stuff which is compiled statically. Some people only can/want to run NEXTSTEP 3.3. Others prefer to run OPENSTEP 4.2. With OPENSTEP 4.2, there's the huge advantage of being able to use dylibs/frameworks which helps get rid of the problems when compiling statically. There's the problem w/ POSIX stuff.
I spent sometime trying to find a way to resolve the issues w/ using your newer cctools and still keep things compatible w/ the BSD4.3. There's supposed to be a way to "convert" the libs. That's as far as I got before going to work. Years ago, I had re-compiled the cctools from Rhapsody for x86/m68k. It works, but besides maybe some bug fixes, I think the time/effort you've done w/ the newer cctools would provide the most benefit when it comes to "more modern" code. :-) I need to spend some time testing your cctools on my Gecko/Sparc systems though. I haven't had time to do that yet. I know when it comes to the Gecko, there might be some issues trying to compile a newer gcc. There's some stuff missing there compared to the other arch's. I'm not sure why NeXT did that. I wonder if that's why they never "officially" released OPENSTEP 4.2 for hppa despite it being developed & even passed onto some customers in Europe. Anyway, I'm going to try to look into the converting libs today I think. :-)
If you're on Google Talk, feel free to hit me up sometime. Same nick @ gmail. :-) Take care!
Steve
Hey Steve,
I am a bit late responding - things got busy at work again and I also now have a newborn who keeps my wife and I on our toes many hours of the day and night :-). So finding a bit of time for hobbies has been challenging, but I somehow managed a few hours this past weekend.
On the side of my gcc port I finally got SVN working properly on my Synology box and, more importantly, I learned how to use it in sufficient detail to "be dangerous". I spent a number of hours going through my notes and configuration files of various revisions scattered throughout the multiple partitions on my NeXT (the 2GB partition size is REALLY limiting). I was able to retrace pretty much everything I did and load it into SVN for gcc-3.2, 3.4 and 4.2. I still have to go through my gcc-4.4 and 4.6 files as well as though all my notes and various test scribbles. I also rebuilt gcc-4.2.4 with my cctools and it appears to work as well as my build from two years ago that I did with the NeXT cctools. I was secretly hoping that the endless warnings about static functions that I have seen before would disappear, but unfortunately they remained. However, in-between the feedings and diaper changes I am able to make few tweaks to the config files and continue rebuilding the final 3rd stage with the goal of identifying the root cause. I just wish that the NeXT gdb worked properly with the binaries produced by gcc-4.2.4, but unfortunately it has some issues.
With respect to modernizing the libraries, I think I have an approach that works. We just need some planning and some coding to make it happen. NB, I thought that a recent NetBSD or OpenBSD would be a good starting point for the libraries. Now, to the meat of the matter. The way I understand the shlibs work, the linker still uses a static like archive that contains references to the offsets in the applicable shlib. For example, all the system functions are located in the /usr/shlib/libsys_s.B.shlib and all the currently built software uses that library by means of being linked with the corresponding /lib/libsys_s.a. The linker pulls the references from libsys_s.a to appropriate offsets in libsys_s.B.shlib and inserts those into the executable. We would leave all that alone without touching it at all. We would instead create a new libsysnew_s.A.shlib that includes all the replacement functions with a corresponding libsysnewb_s.a for the linker. We then make a copy of libsys_s.a into libsysnewa_s.a and use 'ar' to delete all objects from libsysnewa_s.a that have been replaced in libsysnewb_s.a, so there are no duplicates. Finally, we build a custom compiler with a modified link spec that, instead of linking against libsys_s.a, links with libsysnewa_s.a and libsysnewb_s.a. This way our new executables will pull the functions still defined in libsysnewa_s.a from libsys_s.B.shlib and all the new replacement functions defined in libsysnewb_s.a will be pulled from libsysnew_s.A.shlib. The bonus is that the original NeXT compiler together will all the pre-existing executables will continue using the unmodified libraries, while all executables created with our new compiler will use the modified libraries...
Pfft - that made some sweat roll down my forehead! I have no idea if this makes sense to you, but I actually played with this approach a little bit and created some test libraries and it all worked exactly as I tried to describe above...
Peter.
Just a very quick update on my GCC port. I focused my efforts on gcc-3.4.6 since 4.2.4 has been extremely difficult to debug. The biggest recent accomplishment was to get the pre-compiled headers feature to work and then to get the c++ exception handling to work. And the exception handling is Dwarf 2 based as opposed to sjlj, since I used the Darwin implementation.
Here are the current results from the test suite runs:
=== gcc Summary ===
# of expected passes 24424
# of unexpected failures 98
# of unexpected successes 3
# of expected failures 67
# of unresolved test cases 25
# of untested test cases 7
# of unsupported tests 402
/LocalDeveloper/Build/gcc-3.4.6-svn92/gcc/xgcc version 3.4.6
=== g++ Summary ===
# of expected passes 9726
# of unexpected failures 23
# of unexpected successes 1
# of expected failures 66
# of unresolved testcases 3
# of unsupported tests 88
/LocalDeveloper/Build/gcc-3.4.6-svn92/gcc/testsuite/../g++ version 3.4.6
=== libstdc++ Summary ===
# of expected passes 2010
# of unexpected failures 25
# of expected failures 4
# of unsupported tests 9
I think both C and C++ compilers are completely usable now and many of the failures are caused by the NeXT libraries. I am looking through the individual test cases in my spare time but it is a very time consuming process...
Note that these results are for NS33 m68k and I have not done any work on other architectures.
I'm very much looking forward to this. I have several questions. Are these test cases provided by GNU or did you develop them yourself? Is this something that we could share the bugs and work one at a time on? Are you hosting these files anywhere? I'm certainly willing to work on some of this if I have direction, but I have no idea where to start. At the very least maybe I can track down the source of some bugs and give a more competent developer information necessary to create a fix.
--
Brian
Quote from: "barcher174"I'm very much looking forward to this. I have several questions. Are these test cases provided by GNU or did you develop them yourself? Is this something that we could share the bugs and work one at a time on? Are you hosting these files anywhere? I'm certainly willing to work on some of this if I have direction, but I have no idea where to start. At the very least maybe I can track down the source of some bugs and give a more competent developer information necessary to create a fix.
--
Brian
Hi Brian,
Thanks very much for your offer to help!
All the test cases are the standard GCC test suite cases that come with the compiler. At the moment all the sources are sitting on my internal SVN server, but I can package them and at some point and throw them into my dropbox account.
My first attempts to get 3.4.6 going (just after I was able to successfully complete the bootstrap process) produced literally hundreds of test case failures. I am delighted that I was able to reduce them to the point we are at today. I also built 3.4.6 on NetBSD mac68k so I can cross-check against those results, since some test cases will always fail on m68k due to the compiler and not due to the target specific issues. I also have test logs from my old 3.2.3 build. So we can prioritize the review by looking first at the stuff that worked on 3.2.3 but fails on 3.4.6. Then we can look at the stuff that fails on the NeXT but works on NetBSD. Finally, we can look at everything else.
I have a bit more work to do in order to package the compiler into a universally usable state - for example it currently uses modified system libraries that include two additional string functions required by the libstdc++ library, it uses two additional system headers that override some NeXT definitions, etc.
I can also try to prepare a list of pre-requisites and additional packages required to bootstrap the compiler from source as it is not completely trivial. But this would take still more time. As a side note, it takes close to 24 hours to bootstrap stages 1 through 3 of the compiler on my Turbo Color. It then takes another 24 hours to run the gcc and g++ test cases, and another 12 or 18 hours to run the test cases on libstdc++. Fun!
I think the easiest way would be to make the compiler binaries available with log excerpts that show how to invoke it in order to run the test cases. You could then compile and run the individual test cases and identify with gdb (or with some source modifications to include diagnostic messages) why they are being reported as failures...
Peter.
Ok, some good news. I finally managed to clean-up and package gcc-3.4.6. I guess I will create a new thread with links to the files in my dropbox account.
Still some work left to do, but the compiler appears to be solid. I have not done much work with the C++ side, but it also passes most of the tests.
Trying to recompile cctools 435.4 using your patch with GCC 2.95.3. I ran into an issue whereby I had to plunk this into the headers:
#define __private_extern__ extern
Not sure how you defined that or where, but after I defined it the compile made a lot more progress.
I am now on the step where it builds libstuff, but when it runs into the libstuff/dynamic_obj subdir it runs into a neverending loop of parsing/token reading/shifting tokens. Example:
Starting parse^M
Entering state 0^M
Reading a token: Next token is 260 (SCSPEC)^M
Reducing via rule 3 (line 247), -> @1^M
state stack now 0^M
Entering state 2^M
Next token is 260 (SCSPEC)^M
Shifting token 260 (SCSPEC), Entering state 6^M
Reducing via rule 133 (line 888), SCSPEC -> declmods^M
state stack now 0 2^M
Entering state 30^M
Reading a token: Next token is 261 (TYPESPEC)^M
Shifting token 261 (TYPESPEC), Entering state 7^M
Reducing via rule 140 (line 923), TYPESPEC -> typespec^M
state stack now 0 2 30^M
Entering state 78^M
Reducing via rule 129 (line 869), -> reserved_declspecs^M
state stack now 0 2 30 78^M
Entering state 197^M
Any suggestions? I left it for hours and hours and it just kept rolling with this process.
Maybe this conversation will help you identify the problem?
https://gcc.gnu.org/ml/gcc-patches/2004-01/msg02724.html
Please note that I compiled the cctools package with a modified (hacked) OS 4.2 gcc compiler. Just like with the subsequent Apple versions of the gcc compilers it has numerous customizations and modifications. Compiling with 'stock' gcc is almost guaranteed to cause issues, such as with the private extern which is a NeXT / Apple special used during linking. It's like a local-global symbol: global to all the linked modules but local to the final linked object so it does not clash with the real globals in the libraries...
I don't remember if I posted the hacked gcc binary packages and source, but I can make it available if required. Just don't use it for objc because it will fail miserably, but plain C works well. I had to do a circus to make it all work since there is a circular reference: the OS 4.2 gcc requires the new cctools to compile and you require the OS 4.2 gcc to compile the cctools. I solved that by compiling the minimal required cctools binaries for NS on OS, moving them over to my NS box and then building the hacked OS 4.2 compiler on NS. I finally rebuilt the cctools on NS with that hacked compiler.
Thanks for the notes on this, I'm attempting to use the binary package that was uploaded to "New_Ports" in the files area here. I'm also installing the header fix packages that are in that same folder.
Specifically I'm looking at a gcc->fltk->dillo buildchain.