Taking the NeXT step (mach kernel upgrades)

(I'll show myself out for that pun)

As a proof-of-concept photo, NeXTstep 3.3 running on OPENSTEP's kernel with the VBE driver present: http://imgur.com/a/R7fzM

---

So in what is possibly a fit of insanity, I was inspired by the posting of the Darwin 0.1 sourcecode to see if it's possible to actually upgrade the underpinnings of the original NeXT system to something slightly more modern. As a first step of this, using a NeXTstep 3.3 install in VirtualBox, I used the OPENSTEP upgrade kit, and the boot floppies to jury-rig 3.3 into using a newer mach and the VBE driver, and got a fairly decent understanding on how most of the stack fits together.

DriverKit basically operates on a back-plane of buses, specifically ISA/EISA, and PCI, and those buses have to be present in mach for the kernel to successfully boot. There's a tight binding between these bus drivers and mach itself, *however*, drivers that attach to the bus appear to be relatively kernel independent; I can successfully load some NeXTstep 3.3 drivers that I've tried. This is a good thing given that the existing mach/driverkit sources don't have the individual drivers themselves. It might be possible to backport IOKit drivers to DriverKit as the API mostly stayed the same from a conceptional level, it just had a change in programming language.

A couple things I've been able to tell from the ASPL darwin 0.1 source is specifically how the kernel loads coming out of boot, and why there are certain limitations such as maxing out at ~500 MiB of RAM. I'm kinda fried at the moment, but I think I could reasonably remove this restriction fairly easily by doing some low level magic within the kernel to properly recalculate the max amount of RAM available up to at least the 1 GiB mark if not more; I have to take apart the MMU driver to know more for sure. x86 gets a bit weird(er) when you start approaching the theoretical max supported by the architecture.

The 8 GiB drive limit also is likely due to using older forms of LBA addressing and may be surpass-able if the ATA driver can be reasonably rewritten and replaced. I need to look more how the storage classes work, but this might simply be a matter of writing a VirtIO driver.

I'm not sure if anyone is actually interested in this, and as a free-time project, I really don't know how much time I can dedicate to it, but I may poke at it here and there. I need to coax Darwin 0.1 to build with NeXTstep's toolchain, or failing that, get Darwin 1.4.1/x86 running, and then go backwards. Not sure if there's ObjC ABI issues with that route though ...

Figured it was worth a post to gauge actual interest though.

Seems incredibly interesting to me.

All of the above, is awesome an upgrade to NeXTStep and Openstep will be amazing.  I will support this endeavor and look forward to testing it all out.

Well, well!  Congratulations!!!  I am now REALLY tempted to try that on m68k!

I have been hunting for old sources for years.  As you may have seen in some of the posts we even wrote several letters to Apple senior management and they were forwarded through some internal connections.  Unfortunately no one ever responded...

The other stuff rumbling in my head was to try back-porting XNU to m68k with a compatibility layer allowing all the BSD4.3 stuff to run on it.  But this is way beyond my scope and simply a daydream.  For me anyhow.  Although from my successful experiments in porting OS X compiler tools onto NEXTSTEP 3.3 it is clear how close the heritage really is.

You will notice that somewhere between Darwin 0.1 and 1.0 the kernel sources started morphing from what was resembling the original CMU mk-*** organization in kernel-1 under Darwin 0.1, to xnu-*** organization under Darwin 1.0.

It would be so awesome to have access to rhapsody kernel sources!  All we need really in addition to driver development is to fix the UNIX time bug, add support for large disks and add support for modern libraries with POSIX...

Why someone may ask?  I don't know.  Maybe because it's cool.

Screen Shot cool stuff!

Apparently my post I made before I went to bed didn't go through. Oops.

So a couple of notes: this is only going to really work for NS/OS i386 as that's what the source code to darwin has (plus PPC, but we don't care about that right now). It would theoretically be possible to port Darwin back to SPARC/HP-RA/m68k but it would require a lot of work to do so, and I think those platforms would not benefit as much as better HDD support as they should be relatively less hamstrung by the inherent issues with the x86. Unfortunately, while we have the sources for drivekit, it doesn't look like the drivers in and of itself survived, but we do have some support for backwards compatibility.

For VBE, the boot sector for NeXTstep has to be updated mostly because mach runs in protected mode; boot0/1 does some memory tests, loads mach, and sets up a kernel boot structure (very similar to ATAGS on Linux/ARM) which passes the memory and such. As such the system starts up in a 1:1 memory map in protected mode and then it does a bunch of house keeping to start up the machine. mach uses /private/Drivers/x/System.config to determine the modules it needs for the DriverKit backplane, and loads all those. The backplane is pretty tied to mach but the individual drivers themselves seem relatively independent of mach version as the 3.3 drivers do load and appear to work; the NIC driver from 3.3 at least initializies but I don't get network. Then again, I also didn't get network under the standard NS 3.3 kernel idea so ... yeah ...

From some the experimentation I did way back in the age, the base ABI of NeXTstep was retained; basically if you have a statically built POSIX app, it should "just work" between NeXtstep and Mac OS X. When I screwed with this ages ago, if you put a non-x86 NeXT bundle, Mac OS X would actually say "This is an unsupported architecture" just like NeXT did so a lot of base stuff survived at least up to 10.4 or so.

This is really interesting. Back when we "hacked" at Rhapsody DR2 we had the same idea to replace the mach_kernel with some of the earlier OS X versions - however we never had any success (nor access to Darwin 0.1 as far as I remember).

But it's fantastic to see this. It opens a lot of fun possibilities.

Since you fear for the amount of time, that you can put into this project - and since you're the first to ever present such a progress with a screenshot; could I encourage you to make a detailed step by step on how you actually succeeded with this mach_upgrade? That'll first of all document this for the future but it might also help getting more people to help with the project.

Also I can would highly recommend developing a script and use Github to store this as a project.


-Daxziz

My NeXT VM doesn't have working network (or won't until I punt OPENSTEP'S AMD driver into it. My OS VM did manage to talk to the world, then a badly time crashed wrecked the filesystem so I just finished re-installing), so I can't copy off the files, but basically, this is what I did.

This was honestly just a few hours of hacking, I never ran NeXTstep before in my life so I'm a bit surprised no one else managed a kernel transplant. The DR2 kernel should have been able to go backwards in time as well by roughly the same process, and once I have Darwin building again, I'll know what it will take to get this actually working. I have however worked on Mach based kernels before and know how they're put together and built Darwin from source years and years ago so that helped.

I didn't take practically good notes since this was just a quick check to see if it was even viable; I was expecting it to take a lot more effort to transplant the kernel. Darwin 0.1's sources were available when Mac OS X Server 1.0 (which is essentially NeXTstep 5.x) was up, and then vanished when 10.0 shipped.

Based on how DriverKit is connected to mach, it MIGHT be possible to swap out new XNUs with DriverKit, or write a IOKit->DriverKit adapter. It mostly depends how much WindowServer talks to it; if it just wants IOGraphics, it's not super hard as that interface is pretty much identical between the two, just needs a new ObjC binding.

Basically this is what I did:
1. Install NeXTstep 3.3

2. Put the OPENSTEP 4.2 Y2K disk in, you need the MachUpdate4.pkg, or the files in it.

3. Take apart the package using installer_bigtar (its in /NextApps/Installer.app/Contents). This will get you a bunch of updated files.

4. Snapshot the VM

5. Replace /mach_kernel with OpenStep's one

6. Replace /usr/standalone/i386/boot with the one from the update. This is necessary because VBE initialization has to do be done in real mode and thus has to be done

7. Update the boot sector with boot /dev/rhd0a /usr/standalone/i386/boot (has the side effect the system says OPENSTEP at startup).

8. Move /private/Drivers/i386 out of the way and recreate it.

9. Copy System.config from the old drivers folder to the new

10. Put in the OpenStep floppies, you need both the boot one, and drivers one. Dump the files into /private/Drivers/i386. Copy the VBE driver from the OS upgrade kit into this folder.

11. Copy any NeXTstep 3.3 drivers you need (DO NOT COPY ANY OF THE PCI OR BUS DRIVERS, YOU'LL HOSE IT) back into /private/Drivers/i386.

12. Run configure to select the VBE display driver.

13. Edit the Instace0.table and System.config files in the System.config directory to correct the DriverKit buses. My NeXT VM is refusing to boot at the moment since I think I hosed the network on it, but I have it snapshotted so I'll fish out the exact config you need later. The System.config on OS boot is how I figured out how the file should look. It's fairly obvious when you get it wrong, you'll get a kernel panic with "missing symbols" due to the bus not being correctly initialized in order.

14.  Cross your fingers and reboot. If you did it right, you'll get a full color startup of NeXTstep 3.3 with the updated mach like I did.

Thanks for the update - really helpful.

It's my personal feeling that a lot of people in this forum are power users of the system itself, but as soon as you dive into the more complicated stuff with compiling and replacing system components it becomes more of a blur for most and in general it's difficult to figure out where to start properly.

I believe that with the step by step instructions you just provided, you might actually be able to spark the interest and get more people to hack away on their setup.

With a properly created project on Github and hereby a master branch starting with the pure Darwin 0.1 code, and instructions for compiling it, the community would be good to go forward with new branches where people could work on different features like increasing the RAM amount.

In general what I'm talking about is a NeXT community project where the aim is to update the MACH kernel where it makes sense.

Thanks for all your efforts and willingness to document it for future usage.


-Daxziz

I'm currently trying to get Darwin to build, I've got the devkit setup, and a NFS mount to my desktop which helps, but I slammed into a brick wall. I need GNUmake to build this, and GNUmake needs a shell slightly less AT&T SVR4 for autoconf to actually work with it.

It seems a lot of the FOSS builds for the platform dropped off the face of the earth, so I'm trying to find a suitably old ksh I can convince NeXT's tools to build, and then bootstrap from there.

So a follow up, after some teeth pulling, I managed to build both bash and gmake and dug somewhat deeper into this. There's some good and bad here; let's start with the good.

As far as I can tell, these sources are complete, and I also have boot which has the graphical start stuff, and the system identifies itself as Apple Rhaspody in the x86; I was also able to build some parts of it but nothing notable to report. That's where the good ends.

The bad is that these sources need newer compilers than what's in the OpenStep compiler which is surprising as its not THAT much older. The compiler falls over trying to parse some of the headers which is irritating to say the least. As far as the code goes, its roughly half way through the filesystem changes so there's a lot of hardcoded paths for /System/Frameworks and such. Furthermore, the ASPL has a license upgrade clause (in 1.1 anyway) that lets any code I use be licensed under the APSL 2.0 which is free software compatible so yay.

In the case of xnu however, it doesn't look like Apple intended for this to be built standalone. Darwin was built as a whole by a master build system which set up some build flags and such and could do what's called a current build, or a release candidate build (where the source is RO). xnu's Makefile looks like it needs to be invoked from that environment and contains references to utilities that are MIA. I can probably fix this, but I need to figure out what sorta environment this actually wants to be built on since in several places such as in boot, it looks like it wants headers that are not normally distributed as part of the installed system (they're in the kernel source directory). Compounding the problem is the C compiler is falling over on them. I may need to see if I can get cctools to actually do anything.

Also, NFS is freaking flaky. Not sure if that's NeXTstep specific, NIC stuff, or whatever but I feel like I'm using a m68k :/. I may run a NFS server from NeXT instead of reverse just so I can use my preferred editor (and git) instead of building on NFS. Tips on setting that up appreicated.

There's been a few updates om the compilers; I don't know if it's enough at all, but if you take a look in this forum, you'll find a few links to these for OpenStep.
They've been uploaded to some of the ftp servers.

Might be good enough to get a bit further?


-Daxziz

Quote from: "Daxziz"There's been a few updates om the compilers; I don't know if it's enough at all, but if you take a look in this forum, you'll find a few links to these for OpenStep.
They've been uploaded to some of the ftp servers.

Might be good enough to get a bit further?


-Daxziz

I saw that there were GCC updates, but I thought the GCC 3 was for m68k but I'm not sure what exactly I need at the moment. I'm fairly fried at the moment so I probably going to leave this for today and pick it up tomorrow.

Sounds like a wise decision. I'll look through the posts and find the ftp link and post it here.
Assuming I'm not wrong about it being for x86 hardware.


-Daxziz

All of my compiler efforts were primarily focused on m68k since this is where my interest was.  I also focused on NEXTSTEP as opposed to OPENSTEP.  I had issues with i386 compiler builds and I abandoned them quite early in the process.

One important thing is that from what I know you will need an Apple compiler to compile XNU and Darwin.  Generic GCC does not have the required customizations.

I think that looking through OpenDarwin archives, whatever is still available, may be helpful in determining how to build Darwin on i386.  I know that it is not trivial.

Some potentially useful links (but you probably know those already):

https://web.archive.org/web/20060427234856/http://www.opendarwin.org/projects/darwinbuild/

http://shantonu.blogspot.ca

https://macosforge.github.io/darwinbuild/

Well, I made some more headway.

Something I knew but didn't remember until this "morning" was that NeXT/Apple's GNUmake had non-standard extensions in it, which is the references XNU wanted. I didn't realize gnumake was installed under that name on OpenStep, but once I found it, I got somewhere.

Course the build falls over pretty quick because some of the pre-dependancies are MIA. With some effort, I got bootstrap_cmds built, and files copied into the right places. This was a headache because it uses ProjectBuilder makefiles + internal makefiles that weren't shipped as part of the base system. Some hacking got those installed, and OS's pb.makefiles now work. The trick is you need the ability to generate the versions objects, which is done in next-sgs.make (in pb-makefiles for Darwin/OSX 10.0). They also want newer headers than what was shipped out of the box which requires some compiler controtions. The new mach headers seem to break some of the other ones in the system (or I installed them wrong), so I couldn't just do a blanket upgrade. I had to setup a specific folder with handpicked headers in them and then lay them out in place so the compiler could find what it wanted.

I also had to build updated yacc and flex from Darwin 0.1, but I've got a decent hand on how these go together, and what the CoreOS Makefiles are supposed to do. At some point its probably possible to fix the sources to point to the correct locations on NeXTstep so a mass build could (theorically) be done. As a note to the wise: anything that uses next-sgs has to have its build-directory renamed to name-ver, or the build will fail in weird ways.

I'm now currently stuck dealing with mig, the Mach Interface Generator. mig is pretty standard fair on mach based systems, but Apple really butchered it. /usr/bin/mig on NeXT is a shell script that calls out to /usr/lib/migcom; which is the actual message compiler. It seems MiG grew multiple variants, and those variants live in bootstrap commands so now I have migcom, migcom_typd, and migcom_utyped. What I don't have is the wrapper driver that runs the hot mess, and it doesn't look like its in any of the darwin 0.1 sources. It's likely in the Darwin 1.x ones but I'll be damned if I can divine which package its in.

EDIT: I'm a turkey. mig.sh was right there and I missed it in bootstrap_cmds. After some bricks to fix the path issues, gnumake got much further. Still need to work out what else is missing though. It falls over generating the MD and MI headers.

While it is very interesting (and useful IMHO) to get rid of some of the hardware limits of NS, however it is important to notice that if you make NS able of achieving current hardware limits, you extend the lifetime of NS, and the 2038 year is not that far from today. So, any attempt at modernizing NS should deal with approaches for overcoming the 2038 year problem, IMHO.

So, a final update before I stop banging my head onto the desk more.

After a LOT of teeth pulling, I managed to get most of mach to build, but there are headers missing, specifically to those relating to the DriverKit IO buses. For mach itself, it isn't super problematic because its mostly related to EISA device initialization, nothing super critical.

Then I ran into a problem it couldn't find "/usr/local/lib/libDriver_kern.so". As far as I can tell, this is a non-distributed build of the driverkit package, which we have the source to (yay!), *BUT* it talks very heavily to the kernbus backplane I talked about before, who's API is defined in the missing headers.

I managed to stub out most of the functions which would get me a broken kernel but see what would actually be necessary beyond this point. I tripped into objc which is the ObjC runtime and has to be statically linked in such a way the PB makefiles can't handle. I'll have to merge the code from the old to the new to try and fix this. That's going to be an epic headache.

This very much reminds me of building OS X cctools for NEXTSTEP that I discussed in another thread.  I had to mix-and-match headers between OS X and NEXTSTEP in order to satisfy all the pre-processor definitions and dependencies.

Although the difference is that you actually know what you are doing, I was just playing  :P

Quote from: "ardi"While it is very interesting (and useful IMHO) to get rid of some of the hardware limits of NS, however it is important to notice that if you make NS able of achieving current hardware limits, you extend the lifetime of NS, and the 2038 year is not that far from today. So, any attempt at modernizing NS should deal with approaches for overcoming the 2038 year problem, IMHO.

Unfortunately, that's not technically possible without the full source code to NeXTstep *and* breaking compatibility with pre-existing applications. The best that can be done is resetting the epoch to something later than 1970 but  would have to have bad side-effects. Notably, 32-bit Linux has the same problem, as does NetBSD (which changed to 64-bit time_t, but has a compat layer for old binaries)

The underlying 2038 value is due to the fact that time_t is a signed integer on many platforms, and unlike Y2K, is a problem for fundamental to the design of the system vs. a formatting/spacing issue. Two very different things.

QuoteThis very much reminds me of building OS X cctools for NEXTSTEP that I discussed in another thread. I had to mix-and-match headers between OS X and NEXTSTEP in order to satisfy all the pre-processor definitions and dependencies.

Although the difference is that you actually know what you are doing, I was just playing  Razz

At least you had a full set of files :). I have no idea if the EISA/ISA/PCI driver source code was ever released (which has the bits I need to fix this properly vs. guessing my way through) which makes my job considerably more difficult. I can at least get an idea of the interfaces requires by examining the existing binaries, but the struct part of it is going to be considerably more difficult. I don't even have a great idea of how much is missing vs. what has to be there and I've done very little ObjC work overall.

Hello All: I know there were differences between NeXTStep and Openstep headers , also I'm wondering if anything in the FreeBSD port to black hardware would help or better yet gnustep.org . So I'm positive many more NeXT developers will join in soon. I'll contact a few and see if they have further insite. Also would having a copy of Rhapsody help this endeavor? I know I have one :). I heard it was possible to copy the NeXT libraries over to Rhapsody and they actually run. I also have the Openstep developer tools for windows nt. Also at one point they even had a Linux port to NeXT hardware going . Andreas and Simon would probably have some helpful ideas as well.

Right now, the biggest hangup is mostly due to the fact that this is an undocumented interface; something internal that connects the mach kernel to driver kit. Annoying/Fortunately, the powerpc port has a full set of files so I have something to reference.

I had to take yesterday off as a mental health day but I'll be poking this more w/ a stick today once I get my bearing straight.

Hello Ncommander: The documentation install packages may have some clues and may be located under / NeXT libraries bookshelves directory , there is also a set of NeXT Developer books including  writing loadable kernel servers , the complete set is 30 lbs which has syntax :) . Manuals Like in This video https://youtu.be/mk7jlh40qLI of a Turbo Cube system I'm currently integrating!

Right now, I doubt the necessary header files are on Rhaspody, but it might be useful. If anyone has a boot log for Rhaspody would be handy, as well as the boot floppies. Now that I've rested and had some time to think about it, I've managed to get the kernel linked and built; I had to implement some missing symbols with stubs (specifically, nodev(), which I can't even figure out where it's defined).

Anyway, here's some interesting screenshots:




It scrolls out of sight, but the kernel does report a build date here in 2017 :). I might have more success swapping out the OpenStep drivers for Rhaspody ones, though I'm probably going to fix the devicekit build to be more in-line w/ reality.

One major problem is the active_u structure doesn't exist between the original NS kernel and this one which appears to be used for device probing; the ATA probe address changes from reboot to reboot. Notably, the PCI detection stuff actually works properly which is really nice which means we're getting fairly far into boot, and displayServer is at least partially working as in non-verbose boot I get a brief flash of it before it panics.

EDIT screenshot of the kernel boot string:

Wow... simply speechless here. Seeing the Rhapsody 5.3 on x86 is huge and I can see the memory amount is fairly high as well.
It does indeed bring some good possibilities if it all succeeds.


-Daxziz

Quote from: "Daxziz"Wow... simply speechless here. Seeing the Rhapsody 5.3 on x86 is huge and I can see the memory amount is fairly high as well.
It does indeed bring some good possibilities if it all succeeds.


-Daxziz

512 MiB is the maximum supported by the OPENSTEP bootloader. We have the source to the Rhapsody bootloader which has the same hard-coded limitation, but I know enough about how it goes together to bump it at least to the 1 GiB mark if not higher. I have not, as of writing, managed to get it to build mostly because of the mess of symbols in the base system. I can fix it, just haven't done it yet.

Given a closer examination of how it works, I can understand why Apple dumped DriverKit for IOKit; there's a massive performance overhead for ObjectiveC in the kernel space since you have to deal with both the very slow mach messaging system, and the objective C runtime overhead which makes device calls painfully slow (NeXTstep feels damn laggy compared to Darwin). It's broadly similar to the performance characteristics of GNU HURD of all things.

Notably, the kernel identifies itself as "Mac OS X Server Operating System" on PowerPC, and as "Rhapsody Operating System" on x86. DR2, based on screenshots, used "Rhapsody Mach Operating System". Right now, my best bet moving forward is to determine the structures of what it wants and see if I can make it happy enough to keep booting then fix it properly. As far as I can tell, the stock mach kernel is built with symbols (or at least nm can read them); if I could managed to attach a kernel debugger, I could read them out and dump out the structures in RAM getting a better idea of what is and is not missing. Unfortunately, I haven't figured out how to this as of yet on OPENSTEP, and its possible the gdb stub is MIA.

The reason for why I said what I did before is because DR2 is crippled on x86, so the RAM is limited to around 192.
Seen from a Rhapsody perspective it would be fantastic to get just 512.


-Daxziz

Well, after a fair bit more of hacking, I was able to reconstruct enough of a guestimate of the missing components to get DriverKit considerably more happy. Aside from one minor code change, driverKit builds out of the box without code modifications now. Now it looks like it just doesn't like OPENSTEP's drivers, though I haven't run down the exact error:



Still, considering its managing to enumerate the harddrive is an impressive bit.

EDIT: Implemented a stub to fix the old syscall and convert it to the new (kinda a dodgy fix but it works for now). I also had to teach the kernel to ignore a duplicate call to biodone() (which was a panic. Now it just silently ignores the second call) Now it gets further.



The failure to mount root filesystem is concerning because its POSSIBLE the UFS format changed between OpenStep/DR2/Mac OS X 1.0 Server; probably to add better support for Mac stuff. I have no good way to test that theory though.

When I popped in the OS4.2 CD, and told it to use the root filesystem from there, it actually got into single user mode, though is very unhappy.



Looks like something broke with mach-o architecture detection; it dislikes the fact the binaries are FAT (i386/SPARC). I can seemingly load anything that's i386-only but other then that, I get bangs.

You might be spot on about the file system. DR2's disks, even the CDs as far as I remember, were not readable on much else than DR2.
However I wouldn't have guessed that it had that much impact on the kernel itself.

Once again I'm amazed with your progress in just 3 days. I've never seen the like here or anywhere else.


-Daxziz

I'm a little shocked Apple changed the UFS ondisk format, especially since HFS support *is* present, and to my knowledge, Server 1.0 used HFS, and not UFS. Theoretically, I could just reformat / to HFS, and copy the files over though who knows if the extended ACLs survived intact. What's really annoying is there are about ten different variants of UFS all across the UNIX family tree, and I'm not aware of anything beside NeXTstep that can mount its own filesystems.

Then again, it did mount the CD. I need to check if the CD is ISO9660 or UFS formatted (my guess its the former, but I'm not sitting in front of the desktop to check). it can also *load* some binaries, sh and ls worked, file didn't; when I boot it back into OS4.2, I need to examine those binaries to see whats different. There look like some odd changes in the mach-o loader, though a dyld failure suggests we're blowing up in userspace and not kernelspace which doesn't make me optimistic on a quick fix.

Since both the HDD and the CD-ROM run through the EIDE driver, at least it can TALK to the ATA interface, and I can browse successfully with "ls" and such. The floppy driver from OpenStep won't load due to duplicated symbols. The DR2 drivers are likely more compatible overall though.

The saving grace here is that the format might be well enough documented in the NextHeaders folder to fix it and get the kernel to agree to NeXTstep's variant of UFS. Might just be as simple as a filesystem magic check, vs. an actual on disk change. From as far as I can tell, UFS in NeXT/Rhaspody is directly descended from the BSD variants, which means it has a very simple on-disk format. If I can find what broke and line the bits back up, it should get things back up and working. I might also try a netbooting solution if I can figure out how to do it though it's not loading the PCnet driver; that might just require a boot to the head in Instance0 to load it in preboot though. I don't get a symbol load error or anything on it though; it might be trying to come fully up before loading it since as far as I can tell, when you netboot NeXT, it downloads mach_kernel, and the necessary configuration files over TFTP; it never (officially) supported disk-assisted netboot.

NeXT built filesystem support directly into mach and boot. This is in contrast to  most other operating systems, but similar to Mac OS X. I always thought it was a holdover from NeXTstep basically unchanged from there. The only time UFS ever had a use was if you needed case sensitivity in the early Mac OS X's before 10.4(?) added HFSX for case sensitivity.

So, quick update before I call it for the night.

Traced through mach's filesystem code, what it calls "UFS" is actually the Berkeley Fast Filesystem which was basically what BSD 4.4 used with some NeXTstep modifications. Comparing this to what's in Mac OS X 10.0 shows that the only real change that happened here was that HFS replaced UFS as the first filesytem tried for mounting /.

I found a few Apple specific changes that are enabled in the UFS code and some reference docs talking about NeXTstep/OS filesystem changes. I'm hoping this will be straightforward to fix ...

Hello Ncommander : Way to burn rubber man, Jedi coding going down, if this is redundant stuff apologies but it may help...
so NeXTSTEP 3.3 had 2  variants of the user CD NeXTSTEP 3.3  68K/Intel  and NeXTSTEP 3.3  HPPA Risc/Sparc  .  So you would be able to compile quad fat NeXT apps with Developer tools.  
Then with Openstep the idea was to write the app using Openstep for Mach, Sparc or NT then with a recompile to Run on  Mach , NT and Sparc for crossplatform compatability.
Also I have a weird Openstep 4.0 alpha , I can drop box it to you if you like?  It appear as NeXTSTEP 4.0 in verbose boot with different filesystem folders on login , its buggy , they dropped it but it may have transition stuff from the NeXTSTEP 3.3 release to gold master Openstep 4.0 , may be there is something there you can use.
There were 2 versions of Rhapsody  for Intel which the kept under wraps  until MAC OS X on Apple Hardware switched from Power PC to Intel processors so the processor error may be related to the version of Rhapsody you are running if its PPC instead Intel.  
Apple had Rhapsody set to autoeject on Black Hardware to prevent us from tinkering LOL> so it would be fun to see if that could be bypassed to start a Rhapsody install on NeXT 68K hardware.
How much 68K stuff was still intact from NeXTSTEP / Openstep  for a port of Rhapsody back to NeXT hardware is an unknown.  
The NeXT app Daydream now  Darkmatter boots up a 68K  NeXT as a Macintosh , I often wondered if that was possible as NeXT computer architecture was very similar to Mac Quadras  to modify NeXTSTEP 68K to boot a Mac ... Ncommander you would be the one to ask.
Also check out our software archives in case you haven't seen this http://www.nextcomputers.org/NeXTfiles/Software/    the Peanuts disks haves loads of NeXT software also tons of developer apps !
Another cool upgrade for NeXT 68K Hardware would be either a 68060 Processor or powerpc chip 603E upgrade for Black Hardware   he he he like this one.
http://lowendmac.com/2016/powerpc-601-upgrades-for-68040-pds-macs/
and/or support for 30pin 16Mb simms or 72pin 64Mb simms not sure if this would be possible possibly even in software .
, I've heard rumors it existed not sure if it did or if even possible quadras had a daughter card ..... the NRW was going going to be NeXT's new risc based hardware but they shut down hardware production before completion
http://www.nextcomputers.org/NeXTfiles/Images/Rare_NeXT_Hardware/NRW-NeXT_RISC_Workstation/
, I also heard the had NeXTSTEP running on IBM 6000 and DEC Alpha but never released.

Seems to be hard to find info about NeXT's/Apple's UFS implementation.

There seems to be a problem with block sizes between NeXT's and Apple's UFS: https://lists.apple.com/archives/darwinos-users/2001/Apr/msg00423.html
Some general info: http://www.softpanorama.org/Internals/Unix_filesystems/ufs.shtml
https://web.cs.wpi.edu/~rek/DCS/D04/UnixFileSystems.html
Linux could or still can mount NeXT's UFS volumes read-only: ftp://linuxmafia.com/kb/VALinux-kb/nextstep-ufs-support.html and https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/linux-2.6.19.y/Documentation/filesystems/ufs.txt
Current Ubuntu doc say they still can: http://manpages.ubuntu.com/manpages/zesty/man8/mount.8.html
Linux source for UFS support: https://github.com/torvalds/linux/tree/master/fs/ufs
MacFUSE UFS support: http://osxbook.com/blog/2008/12/16/more-user-space-file-system-goodies/

Maybe you can learn something from this info.

HTH, bobo68

Just a brief update. I did find some of those UFS sources but unfortunately, the names in the UFS driver don't match each other so I can't tell by quick examination what changed and what didn't. Compounding the matter is the keyboad on the desktop decided that today is a lovely day to kill itself so I'm stuck until I can replace it.

Here's an update folks. Decided to put the UFS issue aside since I was getting a migrane from it, and dealt with the mach-o loading issue. The primary issue came down to the fact that *something* broke with subarchitecture detection. the MachO binary filesystem can actually contain variants of the same architecture (i386/i486/etc).

What was happening is that code built for i386 (that is an actual 386 processor) would work, but i486 (which are most of the binaries) won't. After a LOT of fiddling, I tracked down the root issue to the architecture detection code; it looks like Apple made some breaking changes on how mach handles both architecture detection and pentium detection, and some left over test code. I haven't decoded the rats nest yet, so I cheated, and forced the kernel to always believe it's running on a Pentium/586 filesystem. I can now run all the apps on the CD successfully w/o issue.

Needless to say, that's a victory. I need to track down the mounting root filesystem issue which probably means adding a ton of debug code to this, but we've now crossed the "it's running console apps" milestone of progress.

Quote from: "Rob Blessin Black Hole"also I'm wondering if anything in the FreeBSD port to black hardware would help or better yet gnustep.org

NetBSD/next68x
http://wiki.netbsd.org/ports/next68k/

Quote from: "NCommander"Needless to say, that's a victory. I need to track down the mounting root filesystem issue which probably means adding a ton of debug code to this, but we've now crossed the "it's running console apps" milestone of progress.

That's amazing! Good work. I do hope you track changes in Git ;)

So, after more head-disk banging, I think the fundamental problem is an issue with disk label vs. filesystem. I can (seemingly) read and write the HDD so we're talking to the disk controller properly but it's not getting any partitions or anything. It also looks like raw disk access is present on the "h" slice of the partition map. Might be a bit late, but that's probably help for dumping NeXTstep MO disks since that seems to have been an issue for Previous.

I need to find where mach loads disklabels and such and work from there.

I cannot offer much of value, unfortunately.

I do however have two shots in the dark for you, based on my previous tinkering:  (1) is the system page size correct? I ran across some changes Apple did in cctools that modified the i386 architecture page size.  (2) perhaps this has something to do with big-endian / little-endian conversion of the file system?  I know there was something funky going on on NS file system between m68k and i386...

Just some random ramblings.

Quote from: "NCommander"the MachO binary filesystem can actually contain variants of the same architecture (i386/i486/etc).

Yes, although I've never seen this documented enough. For example, in theory it should be possible to have an OS X fat binary that has both G4 and G5 code, but I've never seen it, and I really don't know if it can be done.

Quote from: "NCommander"it looks like Apple made some breaking changes on how mach handles both architecture detection and pentium detection, and some left over test code.

Do you mean changes between NeXTSTEP/OpenSTEP and Darwin 0.1, or between Darwin 0.1 and later OS X releases? I suppose you refer to the former. It's a pity that Apple dropped 68K support in Darwin 0.1, it could have helped us to maintain NeXTSTEP in the future.

Quote from: "t-rexky"I cannot offer much of value, unfortunately.

I do however have two shots in the dark for you, based on my previous tinkering:  (1) is the system page size correct? I ran across some changes Apple did in cctools that modified the i386 architecture page size.  (2) perhaps this has something to do with big-endian / little-endian conversion of the file system?  I know there was something funky going on on NS file system between m68k and i386...

Just some random ramblings.

I'm going to assume it's correct right now, since if the system page size was wrong, I would expect considerable amounts of confetti trying to load and access dynamic libraries. I'm tied up at work at the moment but as soon as I have a brether, I'm going to check the OPENSTEP ISOs and see what file system formatting its using; if its UFS based (which I suspect), then I'm almost certain I'm dealing with a disk label problem

Quote from: "ardi"

Quote from: "NCommander"the MachO binary filesystem can actually contain variants of the same architecture (i386/i486/etc).

Yes, although I've never seen this documented enough. For example, in theory it should be possible to have an OS X fat binary that has both G4 and G5 code, but I've never seen it, and I really don't know if it can be done.

Quote from: "NCommander"it looks like Apple made some breaking changes on how mach handles both architecture detection and pentium detection, and some left over test code.

Do you mean changes between NeXTSTEP/OpenSTEP and Darwin 0.1, or between Darwin 0.1 and later OS X releases? I suppose you refer to the former. It's a pity that Apple dropped 68K support in Darwin 0.1, it could have helped us to maintain NeXTSTEP in the future.

Its not documented at all; I didn't even realize this support existed until I dug into mach and I've played in MachO land before. Basically, when dealing with fat binaries, you have multiple mach-o headers sandwiched together; with the m68k header being first (likely due to backwards compatiblility so old NeXTsteps can load FAT binaries).

Past the architecture bit is the subtype field which defines what type of processors can be used or not. This is a bitfield, and the values in the bitfield changed between Darwin and NeXTstep; specifically dealing with Pentium support. When the kernel tries to launch a mach-o binary file, it tries to assign a grade to each mach-o image it can run; highest grade wins. So if you have a hypothetical mach-o binary with 386/486/586 code, it will run the best possible code on the platform you're currently on. What broke here is that Apple was in the process of redoing this to be more "generic" as far as I can tell.

Mach runs two sets of CPU detection tests; the first one is at very early boot and makes sure that you're running on a 486SX or greater. If it fails, kernel panic. This one works as is out of the box.

The second one (which was broken) was it tries to do cpuid and gets confused, and believes its running on a i386. This breaks any binary that is tagged for running on the 486/586, which is why I kept getting all those CPU type incorrect load errors. There were some hardcoded strings being tested and comments saying engineering test that *shouldn't* be there. I couldn't figure out how to make the test happy so I just forced it to 586 compatibility. Anyone who's running this on a legit 486 isn't going to get much from a new mach kernel :). This allows the subtype ID check to successfully pass.

Reporting Darwin to the 680x0 is likely possible; I've proven that the core components for at least the kernel are in place. It's just a massive headache but a lot of the support for for the 680x0 is still in place; i.e., mach IDs and such. It's more they deleted anything with "m68k/sparc/hppa" out of the file folder. Assuming the kernel bus architecture is similar between x86->other, getting it all working is in the realm of possible. If you were persistent enough, you could probably even add support for some IOKit device drivers and bridge them to DriverKit ones. Pity that attempts to get the older NeXT kernels out of Apple failed though :/

You may check etc/fstab  for disk tab entries , you can also write custom disk tab entries or mount additional partitions. Also when NeXTSTEP runs check disk  by default, I've had a complete new clean install  on a larger hard drive even after partitioning them and mounting the partions in etc / fstab only to have the install completely wiped out on a hard shutdown and reboot , where it goes to the default check disk app , everything goes haywire , it may be because it doesn't understand the disk drive geometry beyond 2gb partitions for NeXTSTEP , Openstep handles it better,
the man pages should be located under /nextlibrary/bookshelves  should help
http://polarhome.com/service/man/?qf=disktab&tf=2&of=NeXTSTEP&sf=5
Hope this helps! Great work :)

I haven't had that much time to work on this as of late. I'll try and hack on it this weekend.

I had another, likely very stupid thought.  Is it possible that the partition layout or structure has changed somehow?

I don't really see that happening since in i386 world everything pretty much had to be MBR until UEFI was adopted.  But maybe they did not follow or even support MBR at that point because this was so early in the life of Darwin?

This is likely not the case because, of course, they had everything running on i386 with Rhapsody and with Darwin DP2...

Quote from: "t-rexky"I had another, likely very stupid thought.  Is it possible that the partition layout or structure has changed somehow?

I don't really see that happening since in i386 world everything pretty much had to be MBR until UEFI was adopted.  But maybe they did not follow or even support MBR at that point because this was so early in the life of Darwin?

This is likely not the case because, of course, they had everything running on i386 with Rhapsody and with Darwin DP2...

That's actually what I think happened. Without having the old kernel source available as reference, I've had to infer basically how it works. On old school MBR partitioning, you'd have a set of physical partitions, and then NeXtstep would install into one of thoses, and write the disk label in it.

On MBR disks, you basically have a small 1k set of data at the start to load boot code (which is boot0 and boot1), and then each partition has an additional header for a second stage bootloader (which is just plain boot). OPENSTEP/NEXTSTEP's second stage bootloader switches to protected mode opens the filesystem up, grabs mach_kernel, the System.config file, and then loads the "Boot" driver set from Instance0.table. Configuration is passed into mach via the kernel parameters structure and control passes off to that. Mach loads its brain and the in-RAM drivers, and then re-initializes all devices off the each of the three buses, including the physical partition of where stuff lives.

Within that physical partition, you have a logical partition, which NeXTstep calls the disklabel and is how that space is subdivided into smaller parts. This scheme was extremely common on BSD derived systems. As far as I can tell, those DOS partitions are invisible in the running system, only the bootloader actually knows where they are. There were some extensive changes made to support PowerPC partition maps including a Mac OS boot partition, OpenFirmware partitions, etc (if you've ever used old Mac OS X, this is why / is something like disk0p9). Unless there's some difference on the DVD image, the image itself is formatted UFS. NeXTstep identifies the filesystem in mount as "type 4.3".

The kernel source has this to say on the subject
#if COMPAT_43
       /*
        * Historically filesystem types were identified by number. If we
        * get an integer for the filesystem type instead of a string, we
        * check to see if it matches one of the historic filesystem types.
        */
       fstypenum = (u_long)uap->type;
       if (fstypenum < maxvfsconf) {
               for (vfsp = vfsconf; vfsp; vfsp = vfsp->vfc_next)
                       if (vfsp->vfc_typenum == fstypenum)
                               break;
               if (vfsp == NULL) {
                       vput(vp);
                       return (ENODEV);
               }
               strncpy(fstypename, vfsp->vfc_name, MFSNAMELEN);
       } else
#endif /* COMPAT_43 */


COMPAT_43 is noted as "BSD 4.3" compatibity. The new kernel can also mount that partition. I notably also can't get FDISK to show partions even when started with the old kernel on the CD so I need to look at the installer and figure out how it does it's voodoo.

(compat 43 is defined)

EDIT: Just had a brainway. I *think* the device major/minor numbers might have changed between Rhaspody and NeXTstep. I need a DR2 CD to check though :/

I think you can get all of those old releases here:

https://winworldpc.com/product/mac-os-x

I am curious if you were able to dedicate any more time to this and if you had any progress?

I spent a little bit of time over the weekend sailing the murky waters of Apple OS strategy changes in the early days of Darwin, since I was very confused about the kernel source code in Darwin 0.1.  This is probably not news to you, but to my surprise I was able to piece together the fact that Darwin 0.1 is effectively Mac OS X Server 1.0, which is Rhapsody 5.3.  This explains why the kernel source looks very much like the original mach source rather than the xnu kernel style sources.

Moral of the story is that, if our little community ever embarks on hacking the NEXTSTEP kernel, this would be the best place to start.

And of course you are already doing that  :D

I've been tied up on work and hence haven't had any new progress on i.t

whoa I just happend to see this thread, and I'm amazed that so much progress has been made!

Awesome job for getting it built!!!!!

Apple was shifting filesystems ever so slightly post 3.3 so it really doesn't surprise me.  I recall that Rhapsody couldn't read OS X 1.0 stuff, and neither could OS X 10.0 read OS X 1.0 stuff either when it came to UFS.

You may have to built a newfs, and build a new filesystem on a target disk ....?

I got reproducable results!!!!


Darwin 0.1 kernel booting


Panic on my Rhapsody DR2 filesystem.... I know the APM probably isn't correct as machdep/i386/kernBootStruct.h is missing, and I copied the PowerPC version, and took some liberties googline stuff.....




And Booting off an OPENSTEP 4.2 user CD, and I get the CPU misidentification too!!!

I know it's errors, but reproducability is GREAT!!!!!

Oh yeah fixed up the CPU detection and I can run commands off the OPENSTEP 4.2 CD-ROM

get_cpuid  // machine_configure in ../../machdep/i386/i386_init.c



I can boot to the Rhapsody disk OK, but the fun drivers won't load.



But with enough time it will drop to a console login prompt, and boom I'm IN.......

VERY exciting!

Quote from: "neozeed"Oh yeah fixed up the CPU detection and I can run commands off the OPENSTEP 4.2 CD-ROM
...
VERY exciting!

Definitely great to see! Just wondering, are the fixed-up sources for the Darwin 0.1 kernel already available somewhere?

-- Michael

Quote from: "cuby"

Quote from: "neozeed"Oh yeah fixed up the CPU detection and I can run commands off the OPENSTEP 4.2 CD-ROM
...
VERY exciting!

Definitely great to see! Just wondering, are the fixed-up sources for the Darwin 0.1 kernel already available somewhere?

-- Michael

This is by not means comprehensive....!!!!!

pass1.diff (http://vpsland.superglobalmegacorp.com/install/Darwin/Darwin-0.1/x/pass1.diff)

Although on reflection this patch may make more sense....

pass2.diff (http://vpsland.superglobalmegacorp.com/install/Darwin/Darwin-0.1/x/pass2.diff)

I need to setup the includes more correctly, but this is essentially what I changed...  I think.

You will get the 404 page, the username/password are on the page, click the link, and you can use those credendials and you shoudl be set.

Tomorrow I'll see about making a 100% reproducable build from Rhapsody DR-2.

My source tree is here:

kern_build.tar.gz (http://vpsland.superglobalmegacorp.com/install/Darwin/Darwin-0.1/x/kern_build.tar.gz)

I'll upload a kernel binary, although it is kind of useless.....

mach_new.gz (http://vpsland.superglobalmegacorp.com/install/Darwin/Darwin-0.1/x/mach_new.gz)

Very, very cool!

As a somewhat related side project, a few weeks ago I compiled a summary of NS / OS / Rhap / OS X kernel version evolution.  I will post it in this thread in the next few days.  Just for reference...

Edit - here it is: https://www.flickr.com/photos/t-rexky/shares/3a52Z4

Here is my rough notes, I've just re-built the kernel on a fresh Rhapsody DR2 disk image, and the best part is that I had no missing symbols!

Also the build process needs a clean destination to install stuff to.  In this example I'm using /mnt/hdb which is a mount to a 2nd disk.  An empty directory will suffice as well.

Quote
=== get to the source code
I'm using NFS, maybe you have it on CD-ROM?
mkdir -p /mnt/data
/sbin/mount_nfs -a 1 -D 1 -R 1 -r 512 -t 1 -w 512 192.168.1.3:/volume1/Data /mnt/data

=== prepare links:
ln -s /usr/src/kernel/kern /usr/include/kern
ln -s /usr/src/kernel/machdep /usr/include/machdep
mkdir -p /usr/local/bin
mkdir -p /usr/local/lib


=== bootstrap_cmds
tar -zxvf /mnt/data/darwin/bootstrap_cmds-1.tar.gz
cd bootstrap_cmds/
patch -p1 < /mnt/data/darwin/bootstrap_cmds.diff
make
make install
make DSTROOT=/mnt/hdb install

=== kernel
cd /usr/src
tar -zxvf /mnt/data/darwin/kernel-1.tar.gz
ln -s /usr/src/kernel/BUILD/RELEASE_I386/features.h /usr/include/machdep/i386/features.h
ln -s /usr/src/kernel/machdep/i386/mach_param.h /usr/include/machine/mach_param.h
ln -s /usr/src/kernel/bsd /usr/include/bsd
cd kernel
patch -p1 < /mnt/data/darwin/kernel.diff
setenv DSTROOT=/mnt/hdb
make  (it will fail)
setenv DSTROOT=

=== objc
cd /usr/src
tar -zxvf /mnt/data/darwin/objc-1.tar.gz
cd /usr/src/objc
make
make install
make DSTROOT=/mnt/hdb install

=== add in missing headder from NeXTSTEP
mkdir /usr/include/machkit
cp /mnt/data/darwin/NXLock.h /usr/include/machkit


=== driverkit
cd /usr/src
tar -zxvf /mnt/data/darwin/driverkit-139.1-1.tar.gz
cp /mnt/data/darwin/kernBootStruct.h /usr/include/machdep/i386/
cd driverkit-1
patch -p1 < /mnt/data/darwin/driverkit-1.diff
ln -s /usr/src/kernel/driverkit /usr/include/driverkit
ln -s /usr/src/driverkit-1/libDriver/Kernel /usr/include/audio
touch /usr/include/driverkit/i386/PCIKernBus.h
make
make DSTROOT=/mnt/hdb install


=== kernel
cd /usr/src/kernel/BUILD/RELEASE_I386/
rm  /usr/include/driverkit
ln -s /usr/src/driverkit-1/driverkit /usr/include
mkdir -p /System/Library/Frameworks/System.framework/PrivateHeaders/driverkit
cp /usr/src/driverkit-1/libDriver/i386Release/driverServer.defs /System/Library/Frameworks/System.framework/PrivateHeaders/driverkit
mkdir -p /usr/local/lib/
cp /usr/src/driverkit-1/libDriver/syms/libDriver_kern.o /usr/local/lib/
make
cp mach_kernel /mach_new
sync
reboot

=== testing
hd(0,0)mach_new

# cat bootstrap_cmds.diff
diff -ruN original/bootstrap_cmds/migcom_untypd.tproj/Makefile.preamble bootstrap_cmds/migcom_untypd.tproj/Makefile.preamble
--- original/bootstrap_cmds/migcom_untypd.tproj/Makefile.preamble       Wed Mar 17 00:10:34 1999
+++ bootstrap_cmds/migcom_untypd.tproj/Makefile.preamble        Fri Apr 21 11:47:24 2017
@@ -67,7 +67,7 @@
BUILD_OFILES_LIST_ONLY =

# Additional relocatables to be linked into this project
-VERS_OFILE=migcom_untyped_vers.o
+VERS_OFILE=#migcom_untyped_vers.o
OTHER_OFILES = $(VERS_OFILE)

# Additional libraries to link against
diff -ruN original/bootstrap_cmds/migcom_untypd.tproj/utils.c bootstrap_cmds/migcom_untypd.tproj/utils.c
--- original/bootstrap_cmds/migcom_untypd.tproj/utils.c Wed Mar 17 00:10:39 1999
+++ bootstrap_cmds/migcom_untypd.tproj/utils.c  Fri Apr 21 11:47:24 2017
@@ -134,6 +134,8 @@

extern char *MessFreeRoutine;

+char migcom_untypd_VERS_STRING[11] ="Darwin 0.1";
+
void
WriteIdentificationString(file)
    FILE *file;




cat driverkit-1.diff
diff -ruN original/driverkit-1/driverkit/Makefile driverkit-1/driverkit/Makefile
--- original/driverkit-1/driverkit/Makefile     Fri May  7 08:15:54 1999
+++ driverkit-1/driverkit/Makefile      Fri Apr 21 12:04:13 2017
@@ -14,7 +14,8 @@
PUBLIC_DSTDIR=$(SYSTEM_LIBRARY_DIR)/Frameworks/System.framework/Versions/B/Headers/driverkit
LOCAL_DSTDIR=$(SYSTEM_LIBRARY_DIR)/Frameworks/System.framework/Versions/B/PrivateHeaders/driverkit
INSTFLAGS=-c -m 644
-MACHINE_SUBDIRS=ppc i386 machine
+#MACHINE_SUBDIRS=ppc i386 machine
+MACHINE_SUBDIRS=i386 machine

PUBLIC_HFILES=return.h interruptMsg.h driverTypes.h \
             IOConfigTable.h IODisk.h IOLogicalDisk.h IODiskPartition.h \
diff -ruN original/driverkit-1/libDriver/Makefile driverkit-1/libDriver/Makefile
--- original/driverkit-1/libDriver/Makefile     Fri May  7 08:15:59 1999
+++ driverkit-1/libDriver/Makefile      Fri Apr 21 12:24:32 2017
@@ -43,7 +43,8 @@
#
# Directories containing 'mode-dependent' source.
#
-SOURCE_DIRS= User Kernel ppc i386 eisa pci pcmcia
+#SOURCE_DIRS= User Kernel ppc i386 eisa pci pcmcia
+SOURCE_DIRS= User Kernel i386 eisa pci

AR= ar
LD= ld
@@ -51,8 +52,10 @@
MIG= /usr/bin/mig

MACHINE=i386
-MACHINE_LIST=ppc i386
-BUS_LIST=eisa pci pcmcia
+#MACHINE_LIST=ppc i386
+MACHINE_LIST=i386
+#BUS_LIST=eisa pci pcmcia
+BUS_LIST=eisa pci

ARFLAGS= rucv
ARCHFULL_RC_CFLAGS = $(foreach X, $(RC_ARCHS),$(addprefix -arch , $(X)))
@@ -130,8 +133,8 @@
pci_BUS_MFILES=                IOPCIDirectDevice.m IOPCIDeviceDescription.m
pci_BUS_CFILES=

-pcmcia_BUS_MFILES=     IOPCMCIADirectDevice.m IOPCMCIADeviceDescription.m \
-                       IOPCMCIATuple.m
+pcmcia_BUS_MFILES=     #IOPCMCIADirectDevice.m IOPCMCIADeviceDescription.m \
+                       #IOPCMCIATuple.m
pcmcia_BUS_CFILES=


diff -ruN original/driverkit-1/libDriver/pci/IOPCIDirectDevice.m driverkit-1/libDriver/pci/IOPCIDirectDevice.m
--- original/driverkit-1/libDriver/pci/IOPCIDirectDevice.m      Fri May  7 08:16:08 1999
+++ driverkit-1/libDriver/pci/IOPCIDirectDevice.m       Fri Apr 21 12:23:22 2017
@@ -45,7 +45,10 @@
static inline id
getThePCIBus(void)
{
-    return [KernBus lookupBusInstanceWithName:"PCI" busId:0];
+// JASON
+// [KernBus is missing...................
+//    return [KernBus lookupBusInstanceWithName:"PCI" busId:0];
+return 0;
}

@implementation IODirectDevice(IOPCIDirectDevice)




#cat kernel.diff
diff -ruN original/kernel/driverkit/i386/EISAKernBus.h kernel/driverkit/i386/EISAKernBus.h
--- original/kernel/driverkit/i386/EISAKernBus.h        Thu Jan  1 07:30:00 1970
+++ kernel/driverkit/i386/EISAKernBus.h Fri Apr 21 12:15:10 2017
@@ -0,0 +1,17 @@
+
+
+// from here
+// http://unix.superglobalmegacorp.com/darwin01/newsrc/driverkit/KernDeviceDescription.m.html
+
+#define IRQ_LEVELS_KEY         "IRQ Levels"    // XXX
+#define BUS_TYPE_KEY           "Bus Type"      // XXX
+#define BUS_ID_KEY             "Bus ID"        // XXX
+
+
+//from here
+// http://unix.superglobalmegacorp.com/darwin01/newsrc/driverkit/ppc/PPCKernBus.h.html
+
+#define IO_PORTS_KEY           "I/O Ports"
+#define MEM_MAPS_KEY           "Memory Maps"
+#define IRQ_LEVELS_KEY         "IRQ Levels"
+#define DMA_CHANNELS_KEY       "DMA Channels"
diff -ruN original/kernel/driverkit/i386/autoconf_i386.m kernel/driverkit/i386/autoconf_i386.m
--- original/kernel/driverkit/i386/autoconf_i386.m      Wed Mar 17 00:54:19 1999
+++ kernel/driverkit/i386/autoconf_i386.m       Fri Apr 21 12:36:05 2017
@@ -57,8 +57,8 @@
#import <driverkit/driverTypesPrivate.h>
#import <driverkit/autoconfCommon.h>
#import <driverkit/i386/EISAKernBus.h>
-#import <driverkit/i386/PCMCIA.h>
-#import <driverkit/i386/PCMCIAKernBus.h>
+//#import <driverkit/i386/PCMCIA.h>
+//#import <driverkit/i386/PCMCIAKernBus.h>
#import <driverkit/i386/PCIKernBus.h>

#import <machdep/i386/kernBootStruct.h>
diff -ruN original/kernel/machdep/i386/APM_i386.c kernel/machdep/i386/APM_i386.c
--- original/kernel/machdep/i386/APM_i386.c     Wed Mar 17 00:55:20 1999
+++ kernel/machdep/i386/APM_i386.c      Fri Apr 21 12:46:27 2017
@@ -98,6 +98,7 @@
 */
PMReturn PMConnect(void)
{
+#if 0
    KERNBOOTSTRUCT *kbp = KERNSTRUCT_ADDR;

    APM_BIOS_version.major = kbp->apm_config.major_vers;
@@ -110,6 +111,12 @@
       return PM_R_SUCCESS;
    }
    return PM_R_NO_PM;
+#else
+       //For me APM crashes immediatly on boot.
+       //so It's not hard coded out
+    APM_BIOS_connected = FALSE;
+    return PM_R_NO_PM;
+#endif
}

/*
diff -ruN original/kernel/machdep/i386/i386_init.c kernel/machdep/i386/i386_init.c
--- original/kernel/machdep/i386/i386_init.c    Wed Mar 17 00:55:25 1999
+++ kernel/machdep/i386/i386_init.c     Fri Apr 21 12:50:34 2017
@@ -305,6 +305,8 @@
       return pid;

    pid = cpuid();
+//hardcoded to become a Pentium
+       pid.family=5;

    efl &= ~EFL_ID;
    set_eflags(efl);
@@ -335,6 +337,9 @@
    (void) strcpy(machine, "i386");

    pid = get_cpuid();
+       //hardcode this to become a Pentium
+       //touch look for 486sx users....
+       pid.family=5;
    switch (pid.family) {

    case 0:
diff -ruN original/kernel/machdep/i386/kernBootStruct.h kernel/machdep/i386/kernBootStruct.h
--- original/kernel/machdep/i386/kernBootStruct.h       Thu Jan  1 07:30:00 1970
+++ kernel/machdep/i386/kernBootStruct.h        Fri Apr 21 12:08:44 2017
@@ -0,0 +1,225 @@
+/*
+ * Copyright (c) 1999 Apple Computer, Inc. All rights reserved.
+ *
+ * @APPLE_LICENSE_HEADER_START@
+ *
+ * "Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights
+ * Reserved.  This file contains Original Code and/or Modifications of
+ * Original Code as defined in and that are subject to the Apple Public
+ * Source License Version 1.0 (the 'License').  You may not use this file
+ * except in compliance with the License.  Please obtain a copy of the
+ * License at http://www.apple.com/publicsource and read it before using
+ * this file.
+ *
+ * The Original Code and all software distributed under the License are
+ * distributed on an 'AS IS' basis, WITHOUT WARRANTY OF ANY KIND, EITHER
+ * EXPRESS OR IMPLIED, AND APPLE HEREBY DISCLAIMS ALL SUCH WARRANTIES,
+ * INCLUDING WITHOUT LIMITATION, ANY WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT.  Please see the
+ * License for the specific language governing rights and limitations
+ * under the License."
+ *
+ * @APPLE_LICENSE_HEADER_END@
+ */
+
+/*
+ * kernBootStruct.h
+ * What the booter leaves behind for the kernel.
+ */
+
+#define GRAPHICS_MODE  1
+#define TEXT_MODE 0
+
+#define KERNBOOTMAGIC 0xa7a7a7a7
+
+/* The config table has room for 12 drivers if their config files
+ * are the maximum size allowed.
+ */
+#define CONFIG_SIZE (12 * 4096)
+
+/* Maximum number of boot drivers supported, assuming their
+ * config files fit in the bootstruct.
+ */
+#define NDRIVERS               64
+
+#ifndef __ASSEMBLER__
+
+typedef struct {
+    char    *address;                  // address where driver was loaded
+    int            size;                       // entry point for driver
+} driver_config_t;
+
+typedef struct {
+    unsigned short     major_vers;     // == 0 if not present
+    unsigned short     minor_vers;
+    unsigned long      cs32_base;
+    unsigned long      cs16_base;
+    unsigned long      ds_base;
+    unsigned long      cs_length;
+    unsigned long      ds_length;
+    unsigned long      entry_offset;
+    union {
+       struct {
+           unsigned long       mode_16         :1;
+           unsigned long       mode_32         :1;
+           unsigned long       idle_slows_cpu  :1;
+           unsigned long       reserved        :29;
+       } f;
+       unsigned long data;
+    } flags;
+    unsigned long      connected;
+} APM_config_t;
+
+#if 0
+typedef struct {
+    short   version;
+    char    bootString[160];           // string we booted with
+    int            magicCookie;                // KERNBOOTMAGIC if struct valid
+    int            rootdev;                    // booters guess as to rootdev
+    int            convmem;                    // conventional memory
+    char    boot_file[128];            // name of the kernel we booted
+    int            first_addr0;                // first address for kern convmem
+    int            graphicsMode;               // did we boot in graphics mode?
+    int       kernDev;                 // device kernel was fetched from
+    int     numBootDrivers;            // number of drivers loaded by booter
+    char    *configEnd;                        // pointer to end of config files
+    int            kaddr;                      // kernel load address
+    int     ksize;                     // size of kernel
+    void    *rld_entry;                        // entry point for standalone rld
+
+    driver_config_t driverConfig[NDRIVERS];    // ??
+
+    char   _reserved[4052 - sizeof(driver_config_t) * NDRIVERS];
+       int apm_config;
+
+    char   config[CONFIG_SIZE];                // the config file contents
+} KERNBOOTSTRUCT;
+
+//from http://unix.superglobalmegacorp.com/xnu/newsrc/pexpert/pexpert/i386/boot.h.html
+#else
+
+#define KERNSTRUCT_ADDR   ((KERNBOOTSTRUCT *) 0x11000)
+
+
+typedef struct _EISA_slot_info_t {
+    union {
+       struct {
+           unsigned char       duplicateID     :4;
+           unsigned char       slotType        :1;
+           unsigned char       prodIDPresent   :1;
+           unsigned char       dupIDPresent    :1;
+       } s;
+       unsigned char d;
+    } u_ID;
+    unsigned char      configMajor;
+    unsigned char      configMinor;
+    unsigned short     checksum;
+    unsigned char      numFunctions;
+    union {
+       struct {
+           unsigned char       fnTypesPresent  :1;
+           unsigned char       memoryPresent   :1;
+           unsigned char       irqPresent      :1;
+           unsigned char       dmaPresent      :1;
+           unsigned char       portRangePresent:1;
+           unsigned char       portInitPresent :1;
+           unsigned char       freeFormPresent :1;
+           unsigned char       reserved:1;
+       } s;
+       unsigned char d;
+    } u_resources;
+    unsigned char      id[8];
+} EISA_slot_info_t;
+
+typedef struct _EISA_func_info_t {
+    unsigned char      slot;
+    unsigned char      function;
+    unsigned char      reserved[2];
+    unsigned char      data[320];
+} EISA_func_info_t;
+
+#define NUM_EISA_SLOTS 64
+
+typedef struct _PCI_bus_info_t {
+    union {
+       struct {
+           unsigned char configMethod1 :1;
+           unsigned char configMethod2 :1;
+           unsigned char               :2;
+           unsigned char specialCycle1 :1;
+           unsigned char specialCycle2 :1;
+       } s;
+       unsigned char d;
+    } u_bus;
+    unsigned char maxBusNum;
+    unsigned char majorVersion;
+    unsigned char minorVersion;
+    unsigned char BIOSPresent;
+} PCI_bus_info_t;
+
+/*
+ * Video information..
+ */
+
+struct boot_video {
+        unsigned long   v_baseAddr;     /* Base address of video memory */
+        unsigned long   v_display;      /* Display Code (if Applicable */
+        unsigned long   v_rowBytes;     /* Number of bytes per pixel row */
+        unsigned long   v_width;        /* Width */
+        unsigned long   v_height;       /* Height */
+        unsigned long   v_depth;        /* Pixel Depth */
+};
+
+typedef struct boot_video       boot_video;
+
+#define BOOT_STRING_LEN                160
+
+
+
+typedef struct {
+    short   version;
+    char    bootString[BOOT_STRING_LEN];// string we booted with
+    int            magicCookie;                // KERNBOOTMAGIC if struct valid
+    int            numIDEs;                    // how many IDE drives
+    int            rootdev;                    // booters guess as to rootdev
+    int            convmem;                    // conventional memory
+    int            extmem;                     // extended memory
+    char    boot_file[128];            // name of the kernel we booted
+    int            first_addr0;                // first address for kern convmem
+    int            diskInfo[4];                // bios info for bios dev 80-83
+    int            graphicsMode;               // did we boot in graphics mode?
+    int            kernDev;                    // device kernel was fetched from
+    int     numBootDrivers;            // number of drivers loaded by booter
+    char    *configEnd;                        // pointer to end of config files
+    int            kaddr;                      // kernel load address
+    int     ksize;                     // size of kernel
+    void    *rld_entry;                        // entry point for standalone rld
+
+    driver_config_t driverConfig[NDRIVERS];
+    APM_config_t apm_config;
+
+    char   _reserved[7500];
+
+    boot_video video;
+
+    PCI_bus_info_t pciInfo;
+
+    int            eisaConfigFunctions;
+    EISA_slot_info_t eisaSlotInfo[NUM_EISA_SLOTS];// EISA slot information
+
+    char   config[CONFIG_SIZE];                // the config file contents
+} KERNBOOTSTRUCT;
+#define KERNSTRUCT_ADDR   ((KERNBOOTSTRUCT *) 0x11000)
+#endif
+
+#ifdef KERNEL
+#define static_KERNBOOTSTRUCT (*KERNSTRUCT_ADDR)
+//extern KERNBOOTSTRUCT *KERNSTRUCT_ADDR;
+#else
+/* The assumption is that the booter is the only other consumer of this */
+extern KERNBOOTSTRUCT *kernBootStruct;
+//#undef KERNSTRUCT_ADDR
+// #define KERNSTRUCT_ADDR ((KERNBOOTSTRUCT *) BOOTSTRUCT_ADDR)
+#endif
+
+#endif /* __ASSEMBLER__ */


While building other stuff DR2 has a crazy affinity for the PowerPC ...

Just comment out ppc from

Quote/System/Developer/Makefiles/CoreOS/ReleaseControl/Common.make

Probably a good idea to kill my crappy static links too.....

rm -f /usr/include/kern
rm -f /usr/include/machdep
rm -f /usr/include/machdep/i386/features.h
rm -f /usr/include/machine/mach_param.h
rm -f /usr/include/bsd
rm -f /usr/include/driverkit
rm -f /usr/include/audio

There is of course issues when flipping the switch adn trying to make the PCI kernbus actually work.  Lots of segfaults.  Compiling with -O0 makes it crazy unstable.  Since the driverkit is newer than the kernel it's interfaces have obviously drifted.  It's going to require some tearing into to see how they have drifted and to meet them in the middle.

One thing of interest is that there is a BUNCH of the m88k documents inside of the driver kit.  Does anyone have FrameMaker 2.0J?  It may be exciting to go through this stuff, for the machine that never was....  Just look for m88k

Quote from: "neozeed"One thing of interest is that there is a BUNCH of the m88k documents inside of the driver kit.  Does anyone have FrameMaker 2.0J?  It may be exciting to go through this stuff, for the machine that never was....  Just look for m88k

There's the demo version of FrameMaker 3.2 for NeXTstep at http://www.nextcomputers.org/NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design/FrameMaker.3.2.NIH.b.tar.gz - no idea if this is able to open older documents...

-- Michael

Quote from: "cuby"

Quote from: "neozeed"One thing of interest is that there is a BUNCH of the m88k documents inside of the driver kit.  Does anyone have FrameMaker 2.0J?  It may be exciting to go through this stuff, for the machine that never was....  Just look for m88k

There's the demo version of FrameMaker 3.2 for NeXTstep at http://www.nextcomputers.org/NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design/FrameMaker.3.2.NIH.b.tar.gz - no idea if this is able to open older documents...

-- Michael

Cool, hoping there is some cool info  burried in there!

booting up


at the desktop


shutting down.


So I copied my HD image, and then mounted it under /mnt/hdb did a rm -rf /mnt/hdb/* to clean it out.  I went ahead and built a bunch of the userland stuff (needed libs, and all the cmd stuff)


put headers back for driverkit
Libcompat
Libtelnet
kernel/BUILD/RELEASE_I386
objc
driverkit-1
Libc
(cd  /mnt/hdb/System/Library/Frameworks/System.framework/Versions/B/PrivateHeaders/;tar -cf - .)|(cd /System/Library/Frameworks/System.framework/PrivateHeaders;tar -xvf -)
make install DSTROOT=/mnt/hdb RC_ARCHS=i386
cp /usr/src//kernel/bsd/dev/ppc/drvATADisk/ata_extern.h /usr/include/bsd/dev/
make DSTROOT=/mnt/hdb RC_ARCHS=i386

/System/Library/Frameworks/System.framework/Headers/rpc/rpc.h has a rpcent/getrpcbynumber conflict
commands.....

/usr/src/network_cmds/netstat.tproj/inet.c
//#include <netinet/in_pcb.h>
#include "/usr/include/bsd/netinet/in_pcb.h"

/usr/src/network_cmds/ypinit.tproj/Makefile
       -chown root.wheel $(USRSBIN)/ypinit.sh


So after an hour I had a blank disk with all the new stuff, new kernel new libs new new new!!

I copied the /etc directory from my live DR2 system onto that partition, created the /dev tree and tried to boot but I got some mach_init error 88  thing.  I don't even know where to start on that.

In a fit of rage I saved the disk images of my HD in phases of the install, so I have a HD image that has the first boot/install as a clean system.  I booted that up single user mode with my 'new' slave disk and did a dump/restore overlaying my new userland & kernel onto a bare DR2 system.

dump 0f - /dev/hd1a | restore -rf -

natrually make sure the / is mounted read-write, and of course you are in the root directory.

So with the overlay, I rebooted, and BLAMO ... it booted up.

PCI drivers still have issues loading.

Is there any ISA based NE2000 drivers for OpenSTEP/Rhapsody??

I just realized that the headers that are in the archive for the driver kit don't work.

And as part of it's compilation it'll actually MIG working headers. sigh. so basically build it once and comment out the part that doesn't work, get the haders, then uncomment out the one line, and point IOPCIDirectDevice.m to the MIG'd headder.

driverkit-1/libDriver/pci/IOPCIDirectDevice.m
/mnt/hdb/System/Library/Frameworks/System.framework/Versions/B/PrivateHeaders/driverkit/KernBus.h

I now have PCI, Video and Ethernet.




4.4 BSD (rhapsody) (ttyp0)

login: root
Welcome to Rhapsody!

rhapsody:1# hostinfo
Mach kernel version:
        Kernel Release 5.3:
Sat Apr 22 14:04:38 SGT 2017; root(rcbuilder):kernel/BUILD/RELEASE_I386
Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.


Kernel configured for a single processor only.
1 processor is physically available.
Processor type: i586 (Intel 80586)
Processor active: 0
Primary memory available: 64.00 megabytes.
Default processor set: 34 tasks, 74 threads, 1 processors
Load average: 1.00, Mach factor: 0.25
rhapsody:2# uname -a
Rhapsody rhapsody 5.3 Kernel Release 5.3: Sat Apr 22 14:04:38 SGT 2017; root(rcb                                                                                                                                                             uilder):kernel/BUILD/RELEASE_I386 Copyright (c) 1988-1995,1997-1999 Apple Comput                                                                                                                                                             er, Inc. All Rights Reserved.  i386
rhapsody:3# dmesg
Kernel Release 5.3:
Sat Apr 22 14:04:38 SGT 2017; root(rcbuilder):kernel/BUILD/RELEASE_I386
Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.

physical memory = 64.00 megabytes.
using 227 buffers containing 1.27 megabytes of memory
available memory = 59.20 megabytes. vm_page_free_count = 1d9a
minimum quantum is 10 ms
Copyright (c) 1982, 1986, 1989, 1991, 1993
       The Regents of the University of California.  All rights reserved.

PCI Ver=2.10 BusCount=1 Features=[ BIOS16 CM1 ]
Found PCI 2.0 device: ID=0x12378086 at Dev=0 Func=0 Bus=0
Found PCI 2.0 device: ID=0x70008086 at Dev=1 Func=0 Bus=0
Found PCI 2.0 device: ID=0x70108086 at Dev=1 Func=1 Bus=0
Found PCI 2.0 device: ID=0x00a81013 at Dev=2 Func=0 Bus=0
Found PCI 2.0 device: ID=0x802910ec at Dev=3 Func=0 Bus=0
PCI bus support enabled
PnP: Plug and Play support enabled
ISA/EISA bus support enabled
ISA bus
DriverKit version 500
hc0: device detected at port 0x1f0 irq 14
hc0: Checking for ATA drive 0...  Detected
hc0: Checking for ATA drive 1...
hc0: Checking for ATAPI drive 1...
hc0: Resetting drives...
Registering: hc0
hd0: QEMU HARDDISK 0.9.0
hd0: 4063 cylinders, 16 heads, 63 spt (disk geometry)
hd0: using single sector transfers.
Registering: hd0
hd0: Device Block Size: 512 bytes
hd0: Device Capacity:   1999 MB
hd0: Disk Label:        Disk
hc1: device detected at port 0x170 irq 15
hc1: Checking for ATA drive 0...
hc1: Checking for ATAPI drive 0... Detected
hc1: Checking for ATA drive 1...
hc1: Checking for ATAPI drive 1...
hc1: Resetting drives...
hc1: Drive 0: ATAPI CD-ROM (FAST DRQ, REMOVABLE, CMD PKT LEN=12)
hc1: LBA supported.
hc1: buffer type 3, 512 sectors.
Registering: hc1
IDEDisk: disk 0 is ATAPI
Registering: sc0
SCSIDisk: sc0 does not support ClientThreads selector, using default
sd0: QEMU QEMU CD-ROM 0.9.
Registering: sd0 at Target 0 LUN 0 at sc0
sd0: No Valid Disk Label
sd0: Device Block Size: 2048 bytes
sd0: Device Capacity:   226 MB
SCSIDisk: sc0 does not support ClientThreads selector, using default
Registering: fc0
Registering: PS2Controller
Registering: PCKeyboard0
PCI bus support enabled
Registering: PCI0
Registering: EISA0
Registering: event0
Registering: kmDevice0
rootdev 300, howto 40000
BusMouse mouseInit: no resolution in config table.  Default is 400
Bus mouse running
Registering: BusMouse
NE2000 Generic Driver v0.9.1b
by Gabor Sebestyen

Vendor: RealTek RTL-8029
BASE: 0xc100; IRQ: 11
Registering: en0
en0: Ethernet address 52:54:00:12:34:56
Device inited...
Display0: Cirrus Logic GD5434 detected (2097152 Bytes)
Display0: PCI Dev: 2 Func: 0 Bus: 0
Display: Mode selected: 800 x 600 @ 60 Hz (RGB:256/8)
Registering: Display0





Oh yeah!

For anyone who wants to just run the binaries, first install Rhapsody DR2.

I don't know enough about NS/OS/Rh to make a standalone system.  So we are going to do a cheap upgrade.

Once you have your DR2 installed, boot it into single user mode. (-s).

Then mount the root rw (it'll tell you how)

Then mount THIS CD-ROM IMAGE (https://sourceforge.net/projects/aapl-darwin/files/latest/download?source=navbar)!



fsck -y /dev/hd0a
mount -w /
mkdir /tmp/CD
mount_cd9660 /dev/sd0a /tmp/CD
gzip -dc /tmp/CD/darwin01.gz | restore -rf -
reboot


It'll print a bunch of stuff, then once you are back to the prompt just type in reboot.

Quote from: "t-rexky"Very, very cool!

As a somewhat related side project, a few weeks ago I compiled a summary of NS / OS / Rhap / OS X kernel version evolution.  I will post it in this thread in the next few days.  Just for reference...

Edit - here it is: https://www.flickr.com/photos/t-rexky/shares/3a52Z4

Kernel version 'kernel-154.5.1-7.tar.gz' is Darwin 0.3 ... !

Quote from: "neozeed"Kernel version 'kernel-154.5.1-7.tar.gz' is Darwin 0.3 ... !

I did not think the kernel was modified from Darwin 0.1 to Darwin 0.3.  I cannot find any evidence to the contrary.  Even the sources found online and on your server for Darwin 0.3 kernel diff'd against the Darwin 0.1 kernel source are identical...

Edit: UHOH, I found your other post!  You can send me the image if you'd like and I can export it.  But a toast image should be just an ISO I think.  Did you try mounting it as ISO?

Quote from: "t-rexky"

Quote from: "neozeed"Kernel version 'kernel-154.5.1-7.tar.gz' is Darwin 0.3 ... !

I did not think the kernel was modified from Darwin 0.1 to Darwin 0.3.  I cannot find any evidence to the contrary.  Even the sources found online and on your server for Darwin 0.3 kernel diff'd against the Darwin 0.1 kernel source are identical...

Edit: UHOH, I found your other post!  You can send me the image if you'd like and I can export it.  But a toast image should be just an ISO I think.  Did you try mounting it as ISO?

I'm still downloading it! ... the connection is quite slow and seems to be prone to connection issues.  My home has a 1GB connection and I get timeouts.  I rent a server in Chicago, and it's also frought with issues..

I currently have 110MB of 650MB of this toast image.  I've just run strings against it to see what was in the header... I'm not sure how to best mount this toast image, I guess its HFS so I'll need a toast capable mac.

They also have a Darwin 1.0 toast image, I'll get that next for the sake of comparissons...
Maybe we already have Darwin 0.3 ....?

The file Im excited about is called "drvEIDE-33-1.tar.gz" ....


Some glimpse of information from the era:

http://macosx-admin.omnigroup.narkive.com/xmCB6wzs/version-of-osx

OK I got the files off the toast images... that was another adventure in it's self.


Darwin 0.3 is 99% the same as Darwin 0.1

HOWEVER there is source to the EIDE driver! yay so it's not a total loss.

99% of the kernel changes are:

--- kernel-1/vm/vnode_pager.h   1999-03-16 10:56:09.000000000 -0600
+++ kernel-7/vm/vnode_pager.h   1999-07-22 17:58:33.000000000 -0500
@@ -3,21 +3,21 @@
 *
 * @APPLE_LICENSE_HEADER_START@
 *
- * "Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights
+ * Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights
 * Reserved.  This file contains Original Code and/or Modifications of
 * Original Code as defined in and that are subject to the Apple Public
- * Source License Version 1.0 (the 'License').  You may not use this file
+ * Source License Version 1.1 (the "License").  You may not use this file
 * except in compliance with the License.  Please obtain a copy of the
 * License at http://www.apple.com/publicsource and read it before using
 * this file.
 *
 * The Original Code and all software distributed under the License are
- * distributed on an 'AS IS' basis, WITHOUT WARRANTY OF ANY KIND, EITHER
+ * distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER
 * EXPRESS OR IMPLIED, AND APPLE HEREBY DISCLAIMS ALL SUCH WARRANTIES,
 * INCLUDING WITHOUT LIMITATION, ANY WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT.  Please see the
+ * FITNESS FOR A PARTICULAR PURPOSE OR NON- INFRINGEMENT.  Please see the
 * License for the specific language governing rights and limitations
- * under the License."
+ * under the License.
 *
 * @APPLE_LICENSE_HEADER_END@
 */


the biggest change is a new boot logo, and it looks like a bunch of USB PowerPC stuff was deleted in the kernel.  I have a 3MB diff of the 0.1 & 0.3 kernel, I just quickly looked at it, so maybe there is more.

I'll have to check some of the other files to see if they are also just largely a license change from source license 1.0 to 1.1

So yeah same logo... and new version # to further complicate things.





4.4 BSD (rhapsody) (ttyp0)

login: root
Welcome to Rhapsody!

You have mail.
rhapsody:1# dmesg
Kernel Release 0.3:
Sun Apr 23 02:04:16 PDT 2017; root
Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.

physical memory = 64.00 megabytes.
using 227 buffers containing 1.27 megabytes of memory
available memory = 59.19 megabytes. vm_page_free_count = 1d99
minimum quantum is 10 ms
Copyright (c) 1982, 1986, 1989, 1991, 1993
       The Regents of the University of California.  All rights reserved.

PCI Ver=2.10 BusCount=1 Features=[ BIOS16 CM1 ]
Found PCI 2.0 device: ID=0x12378086 at Dev=0 Func=0 Bus=0
Found PCI 2.0 device: ID=0x70008086 at Dev=1 Func=0 Bus=0
Found PCI 2.0 device: ID=0x70108086 at Dev=1 Func=1 Bus=0
Found PCI 2.0 device: ID=0x00a81013 at Dev=2 Func=0 Bus=0
Found PCI 2.0 device: ID=0x802910ec at Dev=3 Func=0 Bus=0
PCI bus support enabled
PnP: Plug and Play support enabled
ISA/EISA bus support enabled
ISA bus
DriverKit version 500
hc0: device detected at port 0x1f0 irq 14
hc0: Checking for ATA drive 0...  Detected
hc0: Checking for ATA drive 1...
hc0: Checking for ATAPI drive 1...
hc0: Resetting drives...
Registering: hc0
hd0: QEMU HARDDISK 0.9.0
hd0: 4063 cylinders, 16 heads, 63 spt (disk geometry)
hd0: using single sector transfers.
Registering: hd0
hd0: Device Block Size: 512 bytes
hd0: Device Capacity:   1999 MB
hd0: Disk Label:        Disk
hc1: device detected at port 0x170 irq 15
hc1: Checking for ATA drive 0...
hc1: Checking for ATAPI drive 0... Detected
hc1: Checking for ATA drive 1...
hc1: Checking for ATAPI drive 1...
hc1: Resetting drives...
hc1: Drive 0: ATAPI CD-ROM (FAST DRQ, REMOVABLE, CMD PKT LEN=12)
hc1: LBA supported.
hc1: buffer type 3, 512 sectors.
Registering: hc1
IDEDisk: disk 0 is ATAPI
Registering: sc0
SCSIDisk: sc0 does not support ClientThreads selector, using default
sd0: QEMU QEMU CD-ROM 0.9.
Registering: sd0 at Target 0 LUN 0 at sc0
sd0: Disk Not Ready
SCSIDisk: sc0 does not support ClientThreads selector, using default
Registering: fc0
Registering: PS2Controller
Registering: PCKeyboard0
PCI bus support enabled
Registering: PCI0
Registering: EISA0
Registering: event0
Registering: kmDevice0
rootdev 300, howto 40000
BusMouse mouseInit: no resolution in config table.  Default is 400
Bus mouse running
Registering: BusMouse
Display0: Cirrus Logic GD5434 detected (2097152 Bytes)
Display0: PCI Dev: 2 Func: 0 Bus: 0
Display: Mode selected: 800 x 600 @ 60 Hz (RGB:256/8)
Registering: Display0
NE2000 Generic Driver v0.9.1b
by Gabor Sebestyen

Vendor: RealTek RTL-8029
BASE: 0xc100; IRQ: 11
Registering: en0
en0: Ethernet address 52:54:00:12:34:56
Device inited...


So yeah, it's not a 5x kernel anymore but 0.3

rhapsody:2# uname -a
Darwin rhapsody 0.3 Kernel Release 0.3: Sun Apr 23 02:04:16 PDT 2017; root Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.  i386


Still very cool that you were able to find it!

In-between mourning one of my PowerMac G5 starting to exhibit some kind of hardware issue causing kernel panics, I was thinking a little bit of how to integrate this kernel into OS or NS.  I think we would require some form of BSD 4.3 to BSD 4.4 call translation layer in order to allow the existing pre-compiled binaries to function. This is way beyond the capabilities of my grey matter  :lol:

if it matters.........

Rhapsody only reads NeXTSTEP disks as read only.

so I instaleld NS 3.3, then went to make a dump on Rhapsody.  Rhapsody can't dump NS 3.3 either.  So added another disk, booted NS in single user mode, and dumped the install to another disk so I have a 'clean snapshot'.  Created a backup Rhapsody disk, and restored nextstep into that.

copied in the new boot, the new kernel & overlaid the driver tree...

reboot and the nextstep scripts dont like the disk to be Read only on boot. So I altered the kernel to mount root as read write.

MNT_RDONLY
       mp->mnt_flag = MNT_RDONLY;


bsd/vfs/vfs_subr.c
bsd/sys/mount.h:#define MNT_RDONLY      0x00000001      /* read only filesystem */


It's a one liner (dont do it, no point right now).  Anyways the scripts freak out as things like fsck won't work, so I comment them out.  Darwin-7 then gives me some illegal instruction error when trying to load drivers.  Darwin-1 (0.3/0.1) just hangs....

So it seems that the newer tools need to be compiled against older libraries if there is any hope at anything fusion with this stuff, as it's just too new.

As Im going to bed as apparently I have to work outdoors tomorrow... I found the missing bits from my building a bunch of stuff yesterday, my /System/Library/Frameworks/System.framework is missing stuff.. not sure what though, but it's too light.  Also I need to see about building Message.framework and /usr/lib/dylib ... naturally there is apparently a lot of other things missing, but filling in those bits plus the Rhapsody boot stuff and pre-compiled drives, and I can drop to single user mode OK'ish.

Does anyone know how they used to make those 2048 byte read only CD-ROM UFS filesystems?

At some point I'd like to make one.

First I made a copy of the 2 kernels, as I'm going to alter and destroy stuff.

By default running diff gives me 3.9MB worth of changes, the vast majority of them are the license change from 1.0 to 1.1...

So I came up with this to fix up the licensing to match, and to fix a stupid hyphen typo..  Also to drop the ppc directories since I don't care about the PowerPC port.


find . -type f -exec sed -i "s/\"Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights/Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights/g" {} +
find . -type f -exec sed -i "s/Source License Version 1.0 (the 'License').  You may not use this file/Source License Version 1.1 (the \"License\").  You may not use this file/g" {} +
find . -type f -exec sed -i "s/distributed on an 'AS IS' basis, WITHOUT WARRANTY OF ANY KIND, EITHER/distributed on an \"AS IS\" basis, WITHOUT WARRANTY OF ANY KIND, EITHER/g" {} +
find . -type f -exec sed -i 's/FITNESS FOR A PARTICULAR PURPOSE OR NON- INFRINGEMENT.  Please see the/FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT.  Please see the/g' {} +
find . -type f -exec sed -i 's/under the License."/under the License./g' {} +
find ./ -name 'ppc' -exec rm -rf {} \;
find ./ -name '*ppc*' -exec rm -f {} \;


Now I have a 69Kb diff, much more managable.

There is some TCP/IP fixes, and more debugging added into NFS.

And of course this change:


diff -ruN kernel-1/conf/tools/newvers/newvers.csh kernel-7/conf/tools/newvers/newvers.csh
--- kernel-1/conf/tools/newvers/newvers.csh 2017-04-23 21:34:14.451506589 -0500
+++ kernel-7/conf/tools/newvers/newvers.csh 2017-04-23 21:34:12.875506633 -0500
@@ -47,9 +47,9 @@
  /bin/echo "int  version_major      = ${major};" ;
  /bin/echo "int  version_minor      = ${minor};" ;
  /bin/echo "char version_variant[]  = \"${variant}\";" ;
-  /bin/echo "char version[] = \"Kernel Release ${v}:\\n${t}; $w($h):$d\\nCopyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.\\n\\n\";" ;
+  /bin/echo "char version[] = \"Kernel Release ${v}:\\n${t}; $w\\nCopyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.\\n\\n\";" ;
  /bin/echo "char osrelease[] = \"${major}.${minor}\";" ;
-  /bin/echo "char ostype[] = \"Rhapsody\";" ;
+  /bin/echo "char ostype[] = \"Darwin\";" ;
) > vers.c
if [ -s vers.suffix -o ! -f vers.suffix ]; then
    rm -f vers.suffix
diff -ruN kernel-1/conf/version.major kernel-7/conf/version.major
--- kernel-1/conf/version.major 2017-04-23 21:34:14.447506589 -0500
+++ kernel-7/conf/version.major 2017-04-23 21:34:12.867506633 -0500
@@ -1 +1 @@
-5
+0

Hello Neo: Cool stuff !

Quote from: "Rob Blessin Black Hole"Hello Neo: Cool stuff !

Yeah, sadly it's only exciting for Rhapsody...   Although Darwin 0.1 and 0.3 seem to build best with themselves, it's quite the dance to update and go..

Apparently enough has shifted from the time of nextstep that I don't think this is all useful there.. maybe I can build the userland with the old /system I'm not sure, but not being able to mount or load drivers isn't very usefull... Unless you want to load nextstep in single user mode, or off the CD then it's awesome...

Well the good news is that the EIDE driver source from 0.3 builds, and runs.  The bad news is that it's just as defective as all the others.

There is something fundamentally flawed in the driver the way it triggers and expects interrupts, and something gets lost in the shuffle.

I had been thinking it was originally related to the 0x58 return codes, which in the world of LINUX was DMA problems ... So with source in hand I turn off the DMA, same error. I turn off the multi-sector read/writes since the Qemu090 detects in single mode, and it makes no difference.  I limit it to a single controller, single drive, and it still happens, after issuing a read command.

Also when it resets the bus it does it in such a way that once more again the interrupts are lost and the devices just stop responding.  Considering its broken on real hardware as well, it's safe to say that there is a fundamental problem with the EIDE driver, and I think it was never fixed, as even the old Darwin 1.x+++ on x86 releases also feature a flakey EIDE driver.

Since I'm already installed, I guess the way to proceed is to gut the driver to a super minimum, something that isn't the interdependent rats nest, and see if I can get something simple to work.  Or try to trace raised and lowered IRQ's to see what on earth goes wrong.  I donno unlike typing in 'make' this will take time. ....

Just thinking out loud here...

Since Rhapsody kernel uses BSD 4.4 and NS/OS uses 4.3, we would most likely need some form of "translation" in the kernel in order for the old NS and OS libraries and binaries to work.  If this indeed is the case, then the total scope of gluing-in the old kernel may not be significantly larger with OS X xnu kernel than it would be with the Rhapsody kernel.

The bonus is that on i386 xnu kernels have sufficient drivers to be usable through the old OpenDarwin initiative, etc...

On m68k, on the other hand, Rhapsody kernel would possibly integrate better since it's much closer to the NS/OS kernels.

Quote from: "t-rexky"Just thinking out loud here...

Since Rhapsody kernel uses BSD 4.4 and NS/OS uses 4.3, we would most likely need some form of "translation" in the kernel in order for the old NS and OS libraries and binaries to work.  If this indeed is the case, then the total scope of gluing-in the old kernel may not be significantly larger with OS X xnu kernel than it would be with the Rhapsody kernel.

The bonus is that on i386 xnu kernels have sufficient drivers to be usable through the old OpenDarwin initiative, etc...

On m68k, on the other hand, Rhapsody kernel would possibly integrate better since it's much closer to the NS/OS kernels.

The Rhapsody kernel can mount up the NS/OS CD's just fine, and execute binaries.  The problem starts when executable want to talk to kernel level stuff, and all hell breaks loose.  But I've seen this before as old BSD stuff, when you built a new kernel commands like dmesg and ps would stop working and you'd have to re-build those as well.  Not to mention since the Rhapsody kernel will only mount nextstep read only, my transported copy onto a rhapsody filesystem means that command like fsck obviously won't work.

Indeed.  This is why I assumed that there would be a need for a kernel level "translation" such that the kernel would know the calling function is expecting BSD 4.3 behaviour.

I wish I had some time to tinker but I will not have that luxury for a while.  Not that I know what I'm doing, but it would be fun anyway.

If anyone is actually following this, in the build process I do this

touch /usr/include/driverkit/i386/PCIKernBus.h

As in the prior patches I've commented out :


libDriver/pci/IOPCIDirectDevice.m

//    return [KernBus lookupBusInstanceWithName:"PCI" busId:0];


However it'll build OK if PCIKernBus.h simply is:



// missing file

#import <driverkit/KernBus.h>



Some stupid script nuked my /mnt/hdb while I was buiding stuff. sigh... backups backups...

Anyways all the Lib* stuff needs to be re-built but this image is far far closer to being complete........

still single user, the netinfo services are busted.


Darwin03_29_4_2017.7z (https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin03_29_4_2017.7z/download)

As always it's a Qemu 0.90 image + win32 exe.. it'll run on Wine.

The EIDE driver is a perfect example of OOP gone wrong, which is why it loses interrutps.  :cry:

There is a reason it's broken on real hardware as well......

I found in NeXTSTEP 3.3 dev CD's there is an Adaptec driver example that doesn't look too crazy.  Maybe it can be gutted.

Oddly I don't see any driverkit examples on the 3.1 or 3.2 CD's...

Oh yeah, instead of Darwin 0.3, I had the kernel re-name itself to Rhapsody 5.5 since that is basically what it is.  I don't think there ever was a Darwin 0.4 or 0.5 to roll in the Rhapsody 5.6 G4 changes, was there?

Darwin 0.3 boots multiuser!

Nice work!

Impressive :)

Work has been killing me, but I did a superficial port of the QuakeWorld server to Darwin, and left it running.

quakeworld.superglobalmegacorp.com (http://www.gametracker.com/server_info/quakeworld.superglobalmegacorp.com:27500/)

It's a pretty vanilla coop.

I've also made modifications to Qemu 0.90 to give me a 3rd IDE controller, so hopefully it works, and I can use that to try to either fix or write an IDE driver.

While building libc I noticed there is a static version, but I haven't tested it.  It may be worth looking at building the disk ,network & driver commands as static images, they may be able to be overlayed on a NeXTSTEP or OPENSTEP system, keeping everything else in place (namely the dynamic linker!)

I also added a 10GB disk to Qemu 0.90 ,which saw it as 8GB... It may be a platform issue from the day, I'm not sure.  But Darwin see's it as 8GB, and happily formatted it!

awesome work -- i love it!

Had some time to compare 01 & 03 ...

missing projects are:

drvEISABus
objcunique

burried in this post (http://macosx-admin.omnigroup.narkive.com/xmCB6wzs/version-of-osx), is a laundry list of the project files that apparently made up Darwin 0.3 on the CVS server from Wilfredo Sanchez.

Some things that are in the 0.1 dump are *NOT* on that 0.3 toast image I was lucky enough to get.  I have a feeling that some things were either just on CVS, or just were never made public at all.

I haven't started to map the tree yet, but the 0.3 versions include a debian make setup that at least spells out dependencies.  I've never done something like a 'make world' on Debian so I don't know if those are machine readable, or just informational... Although it's something to go on, vs my endless trial and error.

There is a lot of NetBSD userland, and from the looks of it, the BSD kernel & Libc is 4.4BSD  I'm not sure if they even got the CSRG tail end.  Also now that I have a Darwin 1.0 toast image, I haven't checked to see just how big of a change there was between "kernel-7' & "XNU-68".  As the legend goes, NeXT's kernel includes *some* of Mach 3, however Mach 2.5 & Mach 3 are radically different.  And XNU should use Mach 3.  After sitting on it for ages, I do know that the 4 disc CD set of CSRG's BSD includes some Mach 2.5 ... if I had enough time I'd do a comparison, but I don't think it matters... Then again if I knew how I'd build Mach 2.5 ....

I'm doing another listpurge & reload of my CVS (http://unix.superglobalmegacorp.com/cgi-bin/cvsweb.cgi/?cvsroot=Darwin0#dir), but once it's done I'll up it all to source forge, and try to make out packages that can be checked out over the internet along with my uh 'patches' that at least get it building for me.

Obvious projects are........

FIX EIDE ...!!!!
Integrate newer XNU's BSD into kernel-7
Start walking updates of the cmd's and Libs from XNU into Darwin0 .... if at all possible, the calling convention may have radically changed.  While it's dated 1999 there is a lot of 1994... or older in there...

On the topic of filesystem drift...

From ufs/blob/master/super.c (https://github.com/theojulienne/ufs/blob/master/super.c)

case UFS_MOUNT_UFSTYPE_NEXTSTEP:
UFSD("ufstype=nextstep\n");
uspi->s_fsize = block_size = 1024;
uspi->s_fmask = ~(1024 - 1);
uspi->s_fshift = 10;
uspi->s_sbsize = super_block_size = 2048;
uspi->s_sbbase = 0;
uspi->s_dirblksize = 1024;
flags |= UFS_DE_OLD | UFS_UID_OLD | UFS_ST_OLD | UFS_CG_OLD;
if (!(sb->s_flags & MS_RDONLY)) {
if (!silent)
printk(KERN_INFO "ufstype=nextstep is supported read-only\n");
sb->s_flags |= MS_RDONLY;
}
break;

case UFS_MOUNT_UFSTYPE_NEXTSTEP_CD:
UFSD("ufstype=nextstep-cd\n");
uspi->s_fsize = block_size = 2048;
uspi->s_fmask = ~(2048 - 1);
uspi->s_fshift = 11;
uspi->s_sbsize = super_block_size = 2048;
uspi->s_sbbase = 0;
uspi->s_dirblksize = 1024;
flags |= UFS_DE_OLD | UFS_UID_OLD | UFS_ST_OLD | UFS_CG_OLD;
if (!(sb->s_flags & MS_RDONLY)) {
if (!silent)
printk(KERN_INFO "ufstype=nextstep-cd is supported read-only\n");
sb->s_flags |= MS_RDONLY;
}
break;

case UFS_MOUNT_UFSTYPE_OPENSTEP:
UFSD("ufstype=openstep\n");
uspi->s_fsize = block_size = 1024;
uspi->s_fmask = ~(1024 - 1);
uspi->s_fshift = 10;
uspi->s_sbsize = super_block_size = 2048;
uspi->s_sbbase = 0;
uspi->s_dirblksize = 1024;
flags |= UFS_DE_44BSD | UFS_UID_44BSD | UFS_ST_44BSD | UFS_CG_44BSD;
if (!(sb->s_flags & MS_RDONLY)) {
if (!silent)
printk(KERN_INFO "ufstype=openstep is supported read-only\n");
sb->s_flags |= MS_RDONLY;
}
break;

They are all different..

I still have no idea how they made nextstep-cd filesystems...  :oops:

This is the closest I came across to answering your question:

http://peterwong.net/blog/creating-an-openstep-boot-cd/

Quote from: "t-rexky"This is the closest I came across to answering your question:

http://peterwong.net/blog/creating-an-openstep-boot-cd/

Yeah, I've ended up there so many times.. he does everything but create a filesystem.

I just found a Usenet archive of NeXTSTEP stuff from 1988 to 1995, so hopefully someone else asked the question....

I'd love to make my shovelware distro "live" on a CD, much like how NeXT did it..  I'm guessing at this point I need to label a 600MB disk, and manually format it with 2k blocks.  I wonder how NeXT did it...  Do I need to modify a disk controller to read and write in 2k sectors as well, like a CD does?

Someone other than next had to ship stuff on CD, right?

Back when I had a cube (personal mainframe at that!) It came with a few apps already loaded.  And I hate to say it, but I ended up using SoftPC the most.  It's got really great 286 emulation and CGA.  Shame it always cost so damned much.

I thought I was going to have time today, but it's not looking good.  However my favorite junk Mart had some circa 2005 Intel motherboards...  At $10 US, I picked one up.  It's probably far too new for Rhapsody thought.  I think it's closer in pedigree to the Intel transition kit.

I run my NS and OS i386 setup on an IBM ThinkCentre with AGP bus.  It's a P4 but I don't remember the exact flavor.  Storage sits on Adaptec SCSI though.

Quote from: "t-rexky"I run my NS and OS i386 setup on an IBM ThinkCentre with AGP bus.  It's a P4 but I don't remember the exact flavor.  Storage sits on Adaptec SCSI though.

For some reason, SCSI is hard to find here...and I wanted stuff I have source too, which isn't a lot

I changed the boot block to go to 640MB as a test.



And yes, the good news is that this actually booted!

The 'catch' is the bootblocks in Darwin 0.3 can't read a Rhapsody DR-2 disk, so you have to dump/restore onto a newer filesystem.. LOTS of UFS breakage.

Actually it'll go up until 825

physical memory = 825.00 megabytes.
using 2176 buffers containing 16.50 megabytes of memory
available memory = 796.52 megabytes. vm_page_free_count = 18e43


anything more then we get a pmap_expand panic in the kernel....

I know from the booter's libsa/memory.h the layout is specified for the bottom

#define KERNEL_ADDR             0x100000
#define KERNEL_LEN              0x400000

for example...

I need to find where the top is .

That's pretty awesome neozeed.  I still cannot believe that there is this much breakage with the UFS.  I understand going from the ancient BSD4.2 UFS with 2GB limit to something newer, but it seems that each iteration of the OS was breaking file system compatibility with the preceding versions.  Maybe they were tweaking it to get a more "Mac like" UFS, before they threw in the towel and settled on HFS+?

Quote from: "t-rexky"That's pretty awesome neozeed.  I still cannot believe that there is this much breakage with the UFS.  I understand going from the ancient BSD4.2 UFS with 2GB limit to something newer, but it seems that each iteration of the OS was breaking file system compatibility with the preceding versions.  Maybe they were tweaking it to get a more "Mac like" UFS, before they threw in the towel and settled on HFS+?

It could be.  I thought my toolchain was broken as the new boot programs didn't work...  I tried my test disk, and I'm not getting file not found errors.

I guess this is also why the DR2 install is full of "this is not an upgrade" style messages in the setup.  I haven't tried DR1 but I'm guessing it's filesystem is different in slight ways too.

It's been a long time since I ran OS X 1.0 and I don't think Qemu can run it at the moment... But I think the UFS was needed and capped at 8GB.  I need to install 10.1 on pearpc, and see how big UFS can go.  I just remember after 10.2 I used HFS+ onward from there.

It occurred to me that it's possible to run NeXTSTEP's userland chroot'd under Rhapsody.  So I took one step further, and added a shell in the /etc/rc so the system would mostly boot up, but not start anything graphical yet.

I then restored a backup of a virgin NeXTSTEP install chroot'd to it, and ran something like:

/usr/sbin/chroot /nextstep bin/sh
cd /usr/lib/NextStep
./WindowServer & loginwindow

And I get the login window!



HOWEVER... it won't let me login.   :x

rhapsody:28# /usr/sbin/chroot /nextstep bin/sh
# cc -v
Reading specs from /lib/i386/specs
NeXT Computer, Inc. version cc-437.2.6, gcc version 2.5.8
# hostinfo
Mach kernel version:
        Kernel Release 5.3:
Sat Apr 22 14:20:09 SGT 2017; root(rcbuilder):kernel/BUILD/RELEASE_I386
Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.


Kernel configured for a single processor only.
1 processor is physically available.
Processor type: I386 (Intel 586)
Processor active: 0
Primary memory available: 825.00 megabytes.
Default processor set: 36 tasks, 79 threads, 1 processors
Load average: 1.29, Mach factor: 0.22
#


It's not much, but at least I can run userland stuff....

It also occurred to me, that we really don't need the NeXTSTEP login, if we just run commands via -NXHost...



Just enable remote connections, IE insecure mode in preferences



chroot /nextstep /NextApps/Terminal.app/Terminal -NXHost rhapsody

So I'm actually still alive. We're getting close to a release date at work and free time has evaporated into non-existence.

I'm happy to see neozeed took up the torch I left though I really wish I put the code up on github for him so he wouldn't have had to reproduce the wheel to get as far as I did (you could have PMed me :)). It also looks like the missing drivers showned up to get around my hacks in DriverKit to allow the OS4.2 ones to load (at least HDD/CD).

Based on my earlier tests, it *is* possible to build almost all the packages we do have if you can sort the CoreOsMakefiles situation. I was able to get most of the base utilities to build.

Login failure is likely due to NetInfo taking a **** in a chroot. There's some serious deep magic on how NetInfo ties into the boot system which I remember back in Mac OS X 1.0. Its at least theorically possible to disable NI and get fallback behavior to /etc/* (this was doable in Mac OS X 10.0), and what's used in single user boot. You might want to try dumping out the NI database with nidump and putting the stuff in (/private)/etc/{passwd,group} and see if that let's things go.

EDIT: Reading the backscroll, you MIGHT want to try going with VirtualBox over QEMU. I had better luck with EIDE behaving under it. VBox has fairly decent logging so I might be able to pin down the crash if I can get a crash log or at least know how the converation goes.

As for mkfs and friends, the "right" way to do this is we need to rebuild the base system with the NeXTstep/OpenSTEP paths, and them slipstream them into the original installation media. That should create an installable OPENSTEP 4.2 + new voodoo. NeXT's install system is essentially a giant shell script (in /etc/rc.installer I think on the CD), and it shells out fdisk/mkfs/friends. It *might* be easier to get the hot mess to run on HFS+ if Darwin 0.3 has the missing "hfs_mountroot" function. That way we can get away from the UFS madness.

Quote from: "NCommander"So I'm actually still alive. We're getting close to a release date at work and free time has evaporated into non-existence.

I'm happy to see neozeed took up the torch I left though I really wish I put the code up on github for him so he wouldn't have had to reproduce the wheel to get as far as I did (you could have PMed me :)). It also looks like the missing drivers showned up to get around my hacks in DriverKit to allow the OS4.2 ones to load (at least HDD/CD).

Based on my earlier tests, it *is* possible to build almost all the packages we do have if you can sort the CoreOsMakefiles situation. I was able to get most of the base utilities to build.

Login failure is likely due to NetInfo taking a **** in a chroot. There's some serious deep magic on how NetInfo ties into the boot system which I remember back in Mac OS X 1.0. Its at least theorically possible to disable NI and get fallback behavior to /etc/* (this was doable in Mac OS X 10.0), and what's used in single user boot. You might want to try dumping out the NI database with nidump and putting the stuff in (/private)/etc/{passwd,group} and see if that let's things go.

EDIT: Reading the backscroll, you MIGHT want to try going with VirtualBox over QEMU. I had better luck with EIDE behaving under it. VBox has fairly decent logging so I might be able to pin down the crash if I can get a crash log or at least know how the converation goes.

As for mkfs and friends, the "right" way to do this is we need to rebuild the base system with the NeXTstep/OpenSTEP paths, and them slipstream them into the original installation media. That should create an installable OPENSTEP 4.2 + new voodoo. NeXT's install system is essentially a giant shell script (in /etc/rc.installer I think on the CD), and it shells out fdisk/mkfs/friends. It *might* be easier to get the hot mess to run on HFS+ if Darwin 0.3 has the missing "hfs_mountroot" function. That way we can get away from the UFS madness.

Hey glad u are alive... I didn't think a PM would reach you so I didn't try...  I've been dragged down writing c# rest code and all around hating life.

4.4 BSD (darwin) (ttyp0)

login: root
[darwin:~] root# uptime
6:59PM  up 22 days, 20:11, 2 users, load averages: 1.92, 1.54, 1.28
[darwin:~] root# uname -a
Rhapsody darwin 5.5 Kernel Release 5.5: Sun Apr 30 10:53:53 SGT 2017; root(rcbuilder):kernel-7/BUILD/RELEASE_I386 Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.  i386


This is the Darwin 0.3 I changed the string back to Rhapsody as it was wrecking havoc with any 3rd party auto-detect as they all want a Darwin 5 or something...

I've been trying to figure out how they made the OS & devkit distribution CD's but I came up with nothing.  Maybe it was a tool that did a dump of a filesystem but output to a special 2k sector UFS?  I really don't know.

I've gotten the new boot programs to work on a new filesystem just fine, so slipstreaming on an install CD would certainly be best.  I was going to try to use my chroot'd NeXTSTEP/OPENSTEP to try to build disk_commands, but didn't get around to it.

The EIDE driver is a freaking nested mess from hell.  Like I mentioned the only thing I could figure out is that, yes it does lose interrupts, and with a bit more messing around it looks like sometimes it tries to do reads that are just too big.

It's super painful for me to do floppies in real life, as I only have one working diskette, but I have a physical machine I got to run Rhapsody.  When I tried to run NeXTSTEP on it, the EIDE controller went crazy.  However I bought a new (lol) disk that let's me jumper it down to 2GB, and Rhapsody installed fine.  I would have to re-install but I have a suspicion that it may not be the controller / driver code exactly failing but some capability on the newer disks that is freaking it out.

Also running some NS/OS programs give me a syscall missing error.... so it may not just cleanly run, I have to re-build a kernel to at least tell me what syscall # it tried to run.  Running GDB ch-rooted is kinda useless, it runs but errors on some weird ioctl on the tty.

I'm just an amateur in this stuff, however I think enough time and with (LOTS) of help we can certainly get this thing going again!  I've also started to look at updating the bsd directory from Darwin 1.0 vs Darwin 0.3 and it doesn't look like a tremendous amount of drift, some fixes here & there in the filesystem but they did add about 10 new memory syscalls.

I also don't want to get anyone all crazy excited about 68k, but I did find the source to Mach 2.5 on CD #4 of the CSRG set.

mach25 (http://unix.superglobalmegacorp.com/cgi-bin/cvsweb.cgi/mach25/?cvsroot=Mach;only_with_tag=MAIN) & mach25-i386 (http://unix.superglobalmegacorp.com/cgi-bin/cvsweb.cgi/mach25i386/?cvsroot=Mach;only_with_tag=MAIN).  Yes there is both SUN-3 & i386 stuff in there. It may be also possible to de-evolve Darwin back to an earlier BSD compatibility level? Kornbluth (https://en.wikipedia.org/wiki/Cyril_M._Kornbluth)?

It may not be too much at the moment, but it's nice having an 8GB root disk.  And seamlessly going up to 825MB of RAM.

Wait, greater than 8 GiB?

Is the physical drive larger? It might be having a heart attack due to LBA. That's a problem on Xenix due to the fact it only supports CHS. The ATA controller coughs up a response and interrupts it didn't know how to handle and it shat a rather large brick because of it.

For the time era that we're talking about, larger drives would have been always universally SCSI which has a more sane history attached to it. If the EIDE driver was completely shot, then we should be similar issues when reading from the CD interface (which is ATAPI granted, but thats ATA with extensions).

I need to look at my DriverKit code to see how I solved this issue, but I think I was using the OS4.2 driver with patches to DriverKit to implement changed ABI calls.

EDIT: I'm not seeing the EIDE driver source. I can't easily pull out the VM images here to check what I did but you ultimately got further than I did. The base memory check in the kernel is controlled by boot1 in libsa. You could get DR2 to boot to 512 by using OpenStep's boot sector.

Quote from: "NCommander"Wait, greater than 8 GiB?

Is the physical drive larger? It might be having a heart attack due to LBA. That's a problem on Xenix due to the fact it only supports CHS. The ATA controller coughs up a response and interrupts it didn't know how to handle and it shat a rather large brick because of it.

For the time era that we're talking about, larger drives would have been always universally SCSI which has a more sane history attached to it. If the EIDE driver was completely shot, then we should be similar issues when reading from the CD interface (which is ATAPI granted, but thats ATA with extensions).

I need to look at my DriverKit code to see how I solved this issue, but I think I was using the OS4.2 driver with patches to DriverKit to implement changed ABI calls.

EDIT: I'm not seeing the EIDE driver source. I can't easily pull out the VM images here to check what I did but you ultimately got further than I did. The base memory check in the kernel is controlled by boot1 in libsa. You could get DR2 to boot to 512 by using OpenStep's boot sector.

I'm just at 8GB, although I know that diskinit command tries to live in bios bootable space (8GB according to the source) I should try a larger one....

It may be QEMU I don't know.

disk -u -i /dev/rhd1h

8GB although I made a 10GB qcow.  Obviously you need to run this from the source version of diskdev-cmds-1 ...

Well, the boot structure and early IDE initialization lives in boot0. The processor is basically kicked into Protected Mode, and then it does a lot of the magic to make xnu happy. That's why there's supririsngly little initialization code in  machdep places since it all happens in early boot. Also a distinct lack of commenting.

How exactly are you seeing EIDE errors? Divining your debugging from the earlier posts isn't going well for me. We should probably get a canonical repo on github. I can't promise a lot of time on my part to work on this but I always intended to get back to it. My thought here is the "right" thing to do is try and build all the Darwin 0.1/0.3 sources into a bootable system, then glue on the userland there. If we can get that to boot more or less standalone, we're golden.

Annoyingly, I can't get boot0 to compile. As far as I can tell, I *have* all the bits to it, its just got a stupid number of hard dependencies on header files and compiler magic that weren't on OS4.2, and I'm not convinced are there on DR2. NeXT (and Apple) didn't ship the private header files required for most of this (this made my first few attempts at building driverkit a sad panada).

There's a jump table in the kernel of bootable root filesystems. THe kernel knows how to boot from UFS, and FFS2-style filesystems. There's support for HFS, but the mountroot function is ENOTIMPLEMENTED. Looking at the UFS one though, it looks really simple to implement so part of me is wondering if the correct way to move forward is to dump teh UFS madness

I do vaguely remember however that early Mac OS X Server's required a UFS boot partition; it was listed in Disk Setup of the eras (Mac OS X Server (Boot)/Mac OS X Server (Root)).

EDIT: Brainwave here. It MIGHT be possible to convince the entire system to build from Mac OS X Server. On Darwin, it was possible to build a FAT build of the entire operating system and there's shims in place to set the arch flags. Biggest headache is we're stuck with the Rhaspody filesystem layout, but that might be fixable.

SON OF EDIT: Saw your posts with the 0x20 command error; that's ATA READ SECTORS and it looks like we're stuck in PIO mode. PIO is basically data in/out through the the command registers of the ATA interface. It "works" but is slow as dog feces in winter. Looking at how the driver initialization works, it looks like it always tries to put the drive in fast timing mode for PIO unless doing DMA ...

I'm going to have to get a debugger out and trace the ATA blowup to figure out WTF this is actually trying to do.

boot0 compiles fine..... You need to jump up to Rhapsody to start the adventure though, OPENSTEP is.. too old.  And I think some of the key headers & whatnot is missing..

make OBJROOT=/tmp/boot-2/obj SYMROOT=/tmp/boot-2/sym DSTROOT=/tmp/boot-2/roo RC_ARCHS=i386

which gets me:

rhapsody:14# ls /tmp/boot-2/sym/i386/
boot       boot1      libsa.a    nasm*      ndisasm.1  sarld.sys*
boot.sys*  boot1f     libsaio.a  nasm.1     nullboot1  vers.h
boot0      librcz.a   machOconv* ndisasm*   sarld*


I use qemu as I can futz with devices on the otherside, although other than making an IDE controller that sits in the same place as a parallel port, I haven't done anything... .sigh.  Plus it was crazy easy to convert my qcow into a vmdk so I could put my Darwin install on my VMWare ESXi server.

Anyways most of the project directories have a dpkg/control file that has dependencies listed.  Of course what is 'build-base' ... some debian alias that I haven't spent time trying to find what is what, but ..... yeah as always once the weird dance of doing a make world is figured out it wont be so bad for building a system.

rhapsody:20# cc -v
Reading specs from /usr/libexec/i386/2.7.2.1/specs
Apple Computer, Inc. version cc-771.4, based on gcc version 2.7.2.1


On my qemu image I put on sourceforge it's missing some bits to actually compile some stuff, I was pissed as the dpkg thing actually did a rm -rf DSTROOT for me.... so I took a backup and uploaded it. I was so mad that day.

If memory serves, I think build-base is a meta-package. Poking the Darwin 0.3 sources. The super makefile is there (buildtools). You should theorically be able to setup everything in the right places, push the button and get stuff out.

I may need to go see if I can host a VM somewhere so I can try to kick off a build.

neozeed got me setup on a build image so we can try and collobrate on this. Right now, it looks like the fundamental problem is there's some incompatibilities between Darwin 0.3 and DR2 with shared library handling which is why so many things are pooping the bed.

I also found that the image works somewhat smoother on VirtualBox (sans networking). I'm currently seeing if I can get the toolchain to build successfully so I can build Perl and kick off a master build

Do you think the perl from OS X Server 1.0 may build better, as it's Rhapsody and should be fine with -DNEXT and friends vs the shift to Darwin?

perl-5.6.0-13.tar.gz

I'm copying this one over now... If I was smart I'd have put it on the source CD.  :oops:

SOoo... I got the OS X Server perl to build.

As you know they were in some weird process of purging NEXT from OS X.  And so many of the NX* headers are missing or just in really weird places.  As a quick stop-gap I did this:

   55  13:08   mkdir /usr/local/include
   56  13:08   mkdir /usr/local/include/objc
   57  13:08   cp /usr/src/build/Libc-1/internat.subproj/NX*h /usr/local/include/objc
   60  13:09   make OBJROOT=/tmp/perl-1/obj SYMROOT=/tmp/perl-1/sym DSTROOT=/tmp/perl-1/roo RC_ARCHS=i386


And miniperl built


qemuvm:67# /tmp/perl-1/obj/miniperl -version

This is perl, version 5.004_01

Copyright 1987-1997, Larry Wall


I had to enable DNS to let me wget the OS X Server 1.0 files from staticky.com (http://staticky.com/dl/ftp.apple.com/developer/macosxserver/Sources/), and had to reboot.  I thought I could mount my NFS through my VPN but... I'm sure the latency is insane, or I'm too impatient or both.

I get the impression that lots of stuff gets hidden in the private parts of the /System framework, or is brushed aside in some weird manner, like the internal build stuff is OK being all NEXT, but what is shared out is not.  Then again this is such a rough spot of transition, politically and all that.

Sorry about the reboot...



*--** edit

One last update, in the circular dependencies from hell.  I tried the perl from Darwin 0.3 and there is a missing function, _environ from libc....


cc -dynamiclib \
     -compatibility_version 1 \
     -current_version ${dylib_version} \
     -image_base ${base_address} \
     -install_name ${framework_path}/Perl \
     -o Perl \
     perl.o  gv.o toke.o perly.o op.o regcomp.o dump.o util.o mg.o byterun.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o taint.o deb.o universal.o globals.o perlio.o
DYLD_LIBRARY_PATH=/private/tmp/perl-2/obj/Perl:/tmp/perl-2/obj cc  -arch i386  -o miniperl miniperlmain.o Perl
DYLD_LIBRARY_PATH=/private/tmp/perl-2/obj/Perl:/tmp/perl-2/obj ./miniperl -w -Ilib -MExporter -e 0 || make minitest
dyld: ./miniperl Undefined symbols:
__environ
rm -f lib/re.pm
cat ext/re/re.pm > lib/re.pm
You may see some irrelevant test failures if you have been unable
to build lib/Config.pm.
cd t && (rm -f perl; /bin/ln -s ../miniperl perl) \
       && DYLD_LIBRARY_PATH=/private/tmp/perl-2/obj/Perl:/tmp/perl-2/obj ./perl TEST base/*.t comp/*.t cmd/*.t io/*.t op/*.t pragma/*.t </dev/tty
dyld: ./perl Undefined symbols:
__environ
make[2]: [minitest] Error 67 (ignored)
DYLD_LIBRARY_PATH=/private/tmp/perl-2/obj/Perl:/tmp/perl-2/obj ./miniperl configpm tmp
dyld: ./miniperl Undefined symbols:
__environ
make[1]: *** [lib/Config.pm] Error 67
make: *** [build] Error 2



Which is in perl.c

#if defined(__NeXT__) && defined(__DYNAMIC__)
   _dyld_lookup_and_bind
       ("__environ", (unsigned long *) &environ_pointer, NULL);
#endif /* environ */


So being me, I changed __environ to _environ and it's compiling!!.. further.


       Making Errno (nonxs)
Writing Makefile for Errno
mkdir ../../lib/auto/Errno
mkdir blib
mkdir blib/man3
../../miniperl -I../../lib -I../../lib -I../../lib -I../../lib Errno_pm.PL
errno.c:2: illegal external declaration, found `ENOSYS'
cp Errno.pm ../../lib/Errno.pm
Manifying blib/man3/Errno.3
dyld: perl Undefined symbols:
_Perl_Sv
_Perl_markstack_ptr
_Perl_na
_Perl_stack_base
_Perl_stack_sp
_Perl_sv_yes
_do_undump
_perl_destruct_level

       Everything is up to date. 'make test' to run test suite.


That's what happened when I tried the Darwin 0.3 Perl.

THe header paths are very strange. A lot of stuff stuff install Frameworks, but only install in the B directory but don't make the Current link.

I think the fundamental problem is even with Rhapsody-DR2 there has simply been too much drift in the versions.

I downloaded my 0.3 build thing, and this time I got perl to build.  I'll build some more stuff manually as this is looking somewhat better, then build an 8GB disk, and put that online...  I don't think you need or care for the Rhapsody stuff on top? I could do an overlay but I don't think its needed.


Also what is the best way to install manually built stuff?

I know that setting DSTROOT=/ is... dangerous as these scripts tend to do stupid shit like rm -rf DSTROOT..

:roll:

cp -R DSTROOT /

?

ugh anyone have PDO?

/usr/bin/cc -arch i386 -O  -Wmost  -g  -fno-common -I/build/objc4-1/sym/objc4.build/ProjectHeaders -I -I -I/build/objc4-1/sym/objc4.build/derived_src/runtime -I. -pipe       -I/build/objc4-1/sym -I/build/objc4-1/sym/objc4.build/Headers -I/build/objc4-1/sym/objc4.build/PrivateHeaders -F/build/objc4-1/sym    -Wno-unused -DNSBUILDINGOBJC -I/build/objc4-1/sym  -ObjC       -c -o /build/objc4-1/obj/objects-optimized/runtime/objc-file.i386.o objc-file.m
In file included from objc-file.m:29:
objc-file-generic.m:29: pdo.h: No such file or directory
In file included from objc-file.m:29:


I don't have any pdo.h anywhere.  :oops:

I know it was sold as an addon, I have EOF somewhere, but I don't think PDO was sold with EOF, I think it was it's own thing.

The less things we have on top, the saner our world is going to be.

For DSTROOT build it, and then copy it in place. Prevents / from getting nuked anyway.

The problem is the %$#@ build system has circular dependencies and depends on itself to get going. This is one of the most convulated systems I've ever worked with and that's saying a lot. As best I can tell, Apple engineers basically built the new code into /usr/local/* as that's where most of the scripts look for components like kobjc and friends.

I also can't figure out for the life of me what creates things like the Current links in frameworks.

A lot of problem is toolchain drift. The flags set by Rhaspody DR2 are different than what Darwin wants and that's why we're hitting bootstrapping problems because a lot of the NeXTstep behavior was stripped out and then replaced. What's annoying is not THAT much changed, cc-tools in DR2 is only a few versions behind what's in Darwin 0.3 (I think it corresponds very close to Darwin 0.1).

The basic build system requires architecture, machkit, driverkit headers, gnumach headers, and a few other things but I haven't divined the correct order to get things rolling.

Oh for sure it's a crazy show of things going all over the place.  The drift doesn't help things in the slightest.  I don't know how but when I got what I would guess is about 70% of a Darwin build to go I was beyond shocked.

I was also feeling certain I had objc4 build, but now with it wanting pdo.h I have to wonder.

While grepping around on pdo.h I found this in machkit/internal.h


/*
* Abosolutely brilliant.  Assume that sizeof(long) == 4.  The author of this
* code should give classes on how to write non-portable code.  Other bitches:
*
*    (1) assuming that all architectures are big endian
*    (2) storing/accessing shorts on unaligned boundaries
*    (3) assuming all machines align things to 4 byte boundaries
*
* Define a 2-byte word type and a 4-byte word type.
*/


I've never really looked that close into the source, but it's nice to see that the frustration was real.

Quote from: "NCommander"A lot of problem is toolchain drift. The flags set by Rhaspody DR2 are different than what Darwin wants and that's why we're hitting bootstrapping problems because a lot of the NeXTstep behavior was stripped out and then replaced. What's annoying is not THAT much changed, cc-tools in DR2 is only a few versions behind what's in Darwin 0.3 (I think it corresponds very close to Darwin 0.1).

I just realized the compiler isn't defining NeXT by default anymore.  manually adding it to the file that tripped me up on the ObjectiveC runtime compiled by manually adding the -DNeXT ....

So, time to find the -DDARWIN whatever the ******** it is, and either removing the darwin crap and replacing it with next, or just adding it along for the ride.

One thing is for sure this can't be 'darwin' anymore it'll just trip shit up.

I guess that is the philosophical question, is this DARWIN or NeXT?

There is stuff like this buried in makefiles:

# teflon is really the same as nextstep - don't let anybody fool ya
ifeq ($(PLATFORM_OS), teflon)
 PLATFORM_OS = nextstep
endif


Hunting for CFLAGS and adding -DNeXT seems to get some of the job done, although I think java missing may have some issues too..

make[2]: javaconfig: Command not found


when I try to install the Objc4 runtime, although it looks like it built in the SYMROOT directory...  sigh, I hate java.  :roll:

That's likely the ObjC-Java bridge that was part of Mac OS X up until the 10.4 days. I haven't had that much time to work on this today. The objc runtime from Darwin 0.1 will build both userspace and kernelspace without it.

Nothing in the core system should depend on Java and its likely possible to disable it via a build time flag.

Quote from: "NCommander"That's likely the ObjC-Java bridge that was part of Mac OS X up until the 10.4 days. I haven't had that much time to work on this today. The objc runtime from Darwin 0.1 will build both userspace and kernelspace without it.

Nothing in the core system should depend on Java and its likely possible to disable it via a build time flag.

sure no worries.  I have to get busy for the rest of the day, and get back to .. REST. sigh.

But I'll up this disk image and swap it on the "server" ... since perl builds at least this will be far more useful.  Also I didn't have to change a line on Darwin 0.3 perl so this system has 'drifted' into the right way.... so that's nice.

ugh I have to really do my day job but.... I built the driverkit on my darwin 0.3 thing.  for some werid reason there is a missing headder ..  :evil:

From Apple here:
https://opensource.apple.com/source/boot/boot-80.1/i386/libsaio/legacy/PCI.h.auto.html
(https://opensource.apple.com/source/boot/boot-80.1/i386/libsaio/legacy/PCI.h.auto.html)
driverkit/i386/PCI.h
/* Copyright (c) 1994-1996 NeXT Software, Inc.  All rights reserved.
*
* PCI Configuration space structure and associated defines.
*
* HISTORY
*
* 13 May 1994  Dean Reece at NeXT
*      Created.
*
*/

/* The IOPCIConfigSpace structure can be used to decode the 256 byte
* configuration space presented by each PCI device.  This structure
* is based on the PCI LOCAL BUS SPECIFICATION, rev 2.1, section 6.1
*/

typedef struct _IOPCIConfigSpace {
       unsigned short  VendorID;
       unsigned short  DeviceID;
       unsigned short  Command;
       unsigned short  Status;
       unsigned long   RevisionID:8;
       unsigned long   ClassCode:24;
       unsigned char   CacheLineSize;
       unsigned char   LatencyTimer;
       unsigned char   HeaderType;
       unsigned char   BuiltInSelfTest;
       unsigned long   BaseAddress[6];
       unsigned long   CardbusCISpointer;
       unsigned short  SubVendorID;
       unsigned short  SubDeviceID;
       unsigned long   ROMBaseAddress;
       unsigned long   reserved3;
       unsigned long   reserved4;
       unsigned char   InterruptLine;
       unsigned char   InterruptPin;
       unsigned char   MinGrant;
       unsigned char   MaxLatency;
       unsigned long   VendorUnique[48];
} IOPCIConfigSpace;


/* PCI_DEFAULT_DATA is the value resulting from a read to a non-existent
* PCI device's configuration space.
*/

#define PCI_DEFAULT_DATA        0xffffffff


/* PCI_INVALID_VENDOR_ID is a Vendor ID reserved by the PCI/SIG and is
* guaranteed not to be assigned to any vendor.
*/

#define PCI_INVALID_VENDOR_ID   0xffff


How did you get Perl to build without the missing symbol errors?

Quote from: "NCommander"How did you get Perl to build without the missing symbol errors?

Somehow in some fit of crazyness in that Darwin 0.3 qemu image, I had cross built enough crap where I could make that image more or less standalone.  So instead of using Rhapsody as a build base, I used that.  Basically from what I recall, I built the build tools, driver kit, objc-1 & kernel, then using those headers and overlays built what I could of the libraries, then build more commands, more libraries, more commands etc and my DSTROOT was just an empty disk that I had rm -rf  'd to have a bootable / empty image, so enough cycles of building and installing I had a complete enough system.  I never could get perl to build but it must have been legacy crap from Rhapsody in the way, so I took my darwin and I had to add in some bits, but I pretty much just ran:

make OBJROOT=/build/perl-1/obj SYMROOT=/build/perl-1/sym DSTROOT=/build/perl-1/roo RC_ARCHS=i386 RC_OS=teflon

then a 'install' and I just did a

cp -R /build/perl-1/roo/ /

and.. there it is.

For anyone crazy enough to play along, here is the Qemu image:


Darwin03_qemu090_26_5_2017.7z (https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin03_qemu090_26_5_2017.7z/download)

Looks like you already managed to do what I was going. WIth Perl in place, it should be possible to build dpkg-* which is required to run the master build system. Then it should be possible to run the entire build and get the result to pop out. That should crap out a ton of deb files, and the root filesystem in DSTROOT. You need to extract all the source in SRCROOT with the directory names unchanged based on my memories as the version string is extracted on the fly (there's code in CoreOsMakefiles to do that).

There's a couple of places in the makefiles where -arch ppc is hardcoded. At the very least, the C headers, gnumach kernel, and a few other places try to do an independent build assuming its on a PPC build host. It looks like Apple was building both in tandium and then stripping out the x86 components. I may try that this weekend if you don't beat me to it.

I think x86 stuff was code named marklar http://www.eweek.com/apple/apple-keeps-x86-torch-lit-with-marklar
I like it perhaps your code name version of this hybrid os may be call it Kenny keeping in theme with Apples code name marklar in reference to South Park! Awesome work ! Sense of humor in all this but it is amazing, when did they strip out the 68k code , wondering if any remnants in Rhapsody? !

Quote from: "Rob Blessin Black Hole"I think x86 stuff was code named marklar http://www.eweek.com/apple/apple-keeps-x86-torch-lit-with-marklar
I like it perhaps your code name version of this hybrid os may be call it Kenny keeping in theme with Apples code name marklar in reference to South Park! Awesome work ! Sense of humor in all this but it is amazing, when did they strip out the 68k code , wondering if any remnants in Rhapsody? !

I would imagine it was out for quite some time by the time Rhapsody happened.  Although it still hangs around in the strangest of locations.  The binutils package cctools still builds a m68k assembler... among others.

/build/cctools-1/roo/usr/local/libexec/m88k/as
/build/cctools-1/roo/usr/local/libexec/hppa/as
/build/cctools-1/roo/usr/local/libexec/i860/as
/build/cctools-1/roo/usr/local/libexec/m68k/as
/build/cctools-1/roo/usr/local/libexec/sparc/as
/build/cctools-1/roo/usr/libexec/ppc/as
/build/cctools-1/roo/usr/libexec/i386/as


the driverkit has a bunch of m68k stuff in there, HOWEVER it doesn't have anything related to the actual hardware.  All the needed bit twiddling to keep it portable is there, but not an ounce of black hardware.  As I'd mentioned before there is some presentations about the NRW, when it was 88k based.

I'm 100% talking out my but, buttt... I think a SUN-3 port may be possible, using the platform code that surfaced in both mach2.5 & Research UNIX v10.  I know there is TME to attempt to run stuff, although I haven't even looked at building Mach 2.5, but it'd be cool if it was buildable, although I suspect since it is just the kernel directory without the tools (like mig) and NeXT changed things just enough that I doubt we could easily build a vanilla Mach 2.5...  I'm pretty sure I know what you want and if I had a budget/team I'd get Mach 2.5 running on the SUN-3, then try to mash that into the Rhapsody kernel to get it up on the SUN-3, then go for the 68040 as it has a far simpler MMU than the 68030.

As cool in one hand it'd be to 'return' to the original boxes, I fear it'd be insanely slow, and require 64MB of ram at a minimum.

I don't want to be the killjoy but if I had to put up boards & timelines and projectsions I'd venture a x86_64 port would bring you far far far more bang per buck, or even an ARM64 port.

What I really should do is get a Rhapsody version of Previous running.....  Or even Qemu.  My offtopic thing of the moment would be wondering if Qemu can run usermode 68000 programs... could it be possible for something like rosetta using Qemu?  Or even the 68000 emulation out of WinUAE that powers Previous?  Even if it is 100x slower per tick, machines are so fast these days it'd still be faster than the real thing.  As always the overall 'problem' is that not only did x86 win out, but it is so much cheaper, and there is so many of them out there.  Getting this 'fixed' to not be the gigantic PITA to install would go a LONG LONG way.  And knowing that the WindowServer from NeXTSTEP can drive the display driver for Rhapsody is simply amazing.  There is some breakage reagrding Netinfo, and some syscalls being strangely absent.  I know NeXT tried to do a clean break going into OPENSTEP, and a further break once it was obvious that Rhapsody was doomed to the same failure as nobody wanted Cocoa/Objective C, the $ motivation wasn't there (It wasn't until Apple released a NeXT computer you can carry in your pocket, people got excited).

I wonder if Apple put *any* arm code back into XNU?  I know they would be terrified of the iPhone being sideloaded with Darwin.  And I don't even know how big of a difference there is from the "Mach 2.5 + some 3.0 enhancements" vs the OSFMK 3.0 that XNU uses.

I imagine due to your business, you'd know best than anymore.  Who uses NS/OS anymore? What do they use it for?  Who phones you in the middle of the night begging to get their black box up and running because it's the secret machine in the corner that runs something that they don't want to even dream about replacing?

When I was running parts of NeXTSTEP in a chroot, I couldn't think of anything 'must have', and it really hit me, that when I had my NeXT, I mostly used it as a mod player & IRC machine, with the occasional game of BattleTech 3025 in SoftPC.  All of which I can do on other things far more easier.  Heck I don't even have SoftPC for intel.  :oops:

Rob, some of the m68k code remained in OS X for a very long time.  For example cctools (assembler, linker, archiver and all the other mach-o goodies) persisted at least through 10.5 if not until today - I have not bothered checking.  So when you build cctools on OS X 10.5 it builds the m68k, hppa, spare, i860, etc. cross-assemblers.  The mach-o tools are also aware of FAT binaries and can transparently deal with all those architectures.  This is how I was able to build cctools from OS X for NS3.3 and they work.

But all the critical kernel stuff for m68k has been removed from Rhapsody before the Darwin 0.1 version.  The "private" release notes might shed some light on when that happened, if someone locates them and has the time to read...

(edit: and neozeed beat me to it :) )

dpkg-3 really thinks there is ONLY PowerPC devices.

diff -ruN dpkg-3_orig/dpkg.src/lib/arch.c dpkg-3/dpkg.src/lib/arch.c
--- dpkg-3_orig/dpkg.src/lib/arch.c Sun May  9 19:22:33 1999
+++ dpkg-3/dpkg.src/lib/arch.c Fri May 26 03:49:30 2017
@@ -159,11 +159,15 @@
  }

  /* for the moment, we support only native compilation on Rhapsody */
+/* but we aren't PowerPC.. wtf*/
+/****
  if (strcmp (uts.sysname, "Rhapsody") == 0) {
    return snprintf (dst, n, "%s", "powerpc-apple-rhapsody");
  } else if (strcmp (uts.sysname, "Darwin") == 0) {
    return snprintf (dst, n, "%s", "powerpc-apple-darwin");
  }
+******/
+    return snprintf (dst, n, "%s", "i386-apple-rhapsody");

  /* on linux, use gcc --print-libgcc-file-name to find the compilation architecture */
  if (strcmp (uts.sysname, "Linux") == 0) {
diff -ruN dpkg-3_orig/dpkg.src/scripts/controllib.pl dpkg-3/dpkg.src/scripts/controllib.pl
--- dpkg-3_orig/dpkg.src/scripts/controllib.pl Sun May  9 19:22:45 1999
+++ dpkg-3/dpkg.src/scripts/controllib.pl Fri May 26 03:51:01 2017
@@ -40,9 +40,9 @@
    $unames = <UNAMEOUT>; chop ($unames); $unames =~ tr/[A-Z]/[a-z]/;
    $unamev = <UNAMEOUT>; chop ($unamev); $unamev =~ tr/[A-Z]/[a-z]/;
    if ($unames eq "rhapsody") {
- return "powerpc-apple-rhapsody";
+ return "i386-apple-rhapsody";
    } elif ($unames eq "darwin") {
- return "powerpc-apple-darwin";
+ return "i386-apple-darwin";
    } else {
 return "unknown-unknown-unknown";
    }


this is... going to be challenging.

Anyways Build.PM naturally hard codes to powerpc ... sigh so many places!

And it looks like Apple took the Darwin mailing list archive offline?!!?!???  I sure hope this isnt all lost!   :shock:

https://web.archive.org/web/20140115082719/http://lists.apple.com/archives/darwin-development/2001/Sep/msg00571.html

otherwise, yeah, I have no idea what I'm doing.  I guess extract all the things.....


bash-2.02# darwin-buildpackage --dir --target headers /usr/jason_src/architecture-1 /usr/jason_src /tmp/xx
building architecture_226-1_i386-apple-darwin from /usr/jason_src/architecture-1:

building package:

Package: architecture
Maintainer: Darwin Developers <darwin-development@public.lists.apple.com>
Version: 226-1
Source: architecture
Build-Depends: build-base
Architecture: i386-apple-darwin
Description: Architecture-specific headers.

using parameters:

PACKAGEROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.pkg
LOGFILE: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.log
HDRROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.hdr
BUILDROOT: /buildit/architecture-226-1.roots/architecture-226-1.root
SRCDIR: /usr/jason_src/architecture-1
SUBLIBROOTS: /buildit/architecture-226-1.roots/architecture-226-1.root/usr/local/lib/objs
LIBCOBJROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.cobj
PACKAGEDIR: /tmp/xx
SRCROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1
SYMROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.sym
OBJROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.obj
DSTROOT: /buildit/architecture-226-1.roots/architecture-226-1.root/buildit/architecture-226-1.roots/architecture-226-1.dst

using build parameters:

PACKAGEROOT: /buildit/architecture-226-1.roots/architecture-226-1.pkg
LOGFILE: /buildit/architecture-226-1.roots/architecture-226-1.log
HDRROOT: /buildit/architecture-226-1.roots/architecture-226-1.hdr
BUILDROOT: /buildit/architecture-226-1.roots/architecture-226-1.root
SUBLIBROOTS: /usr/local/lib/objs
LIBCOBJROOT: /buildit/architecture-226-1.roots/architecture-226-1.cobj
SRCROOT: /buildit/architecture-226-1.roots/architecture-226-1
SYMROOT: /buildit/architecture-226-1.roots/architecture-226-1.sym
OBJROOT: /buildit/architecture-226-1.roots/architecture-226-1.obj
DSTROOT: /buildit/architecture-226-1.roots/architecture-226-1.dst

Building build root:
Uncaught exception from user code:
unable to find dependency for "architecture-hdrs" at /usr/lib/perl5/Dpkg/Package/Builder.pm line 684.
Dpkg::Package::Builder::makeroot('Dpkg::Package::Package=HASH(0x150e78)', '/buildit/architecture-226-1.roots/architecture-226-1.root', 'ARRAY(0xe3ec)') called at /usr/lib/perl5/Dpkg/Package/Builder.pm line 761
Dpkg::Package::Builder::setupdirs('Dpkg::Package::Package=HASH(0x150e78)', 'HASH(0x12a010)', '/usr/jason_src/architecture-1', 'dir', 'ARRAY(0xe3ec)') called at /usr/lib/perl5/Dpkg/Package/Builder.pm line 850
Dpkg::Package::Builder::build('dir', '/usr/jason_src/architecture-1', 'ARRAY(0xe3ec)', 'headers', '/tmp/xx', 0) called at /usr/bin/darwin-buildpackage line 41


"I imagine due to your business, you'd know best than anymore.  Who uses NS/OS anymore? What do they use it for?  Who phones you in the middle of the night begging to get their black box up and running because it's the secret machine in the corner that runs something that they don't want to even dream about replacing?"

There are still some small niches where Intel based machines still run NeXTSTEP 3.3 / Openstep 4.2 the kicker is the proprietary legacy 3rd party software like the Hubble Optics Controls , yes really run on NeXTSTEP and they will use it for the James Webb Telescope as well. Fannie Mae for Mortgages,  Lam Research in Silicon Chip manufacturing and I think some 3 letter agencies  as well as trading firms mission critical 3rd party apps and perhaps the NOAA to predict Hurricanes although I won't go there with the budget cut thing currently under review .    

Then in the forum a lot of folks still running NeXTSTEP and Openstep on black hardware with all the cool 3rd party stuff , they are great machines and I'll stay with them it is my destiny.

Now this hybrid OS you are all are inventing in real time is intriguing, call it a NeXTSTEP/ Openstep / Rhapsody shells? with an upgraded Mach Kernal BSD 4.4 at the core built on Darwin source?   with Unix FOO magic, will this potentially run or be compatible with any legacy NeXT software, Apple Software or Unix software  as it looks like the dev tools are compatible... but I'm not really sure...

As I say I see a lot of insanely great work going on here and it really is fun to see it happening but what will it do in layman's terms.  I would like to help but I'm sure I would probably throw a newby coding wrench  into the works LOL unless it is something universal that I and many others are familiar with like ducktape or WD40 (just in fun), I'll  just watch you guys work your coding magic  and happily beta test whatever manifests or emerges out of this project , gazing on sometimes like a deer in the headlights but whatever this is I know in the end run it is cool.  
I'm no stranger to coding and I know I had a 200 page  program that wouldn't run until I inserted a space between to parenthesis in archaic cobol and compiled it literally () to ( ) , it was a nested bug, so I feel your pain here working on "Kenny" . :)  The program for inventory control of a database of VCR tapes would have easily been creatable in NeXTSTEP in a day lol and it took month's... it was  something like Itunes .... which still runs on webobjects

So have high hopes that we can get both the disk drivers working for modern disks along, with some way to create those UFS CD-ROM's so we can do "live testing" with as much hardware as possible....

I've been running my QuakeWorld server since I got the 0.3 CD although I'm not sure about stuff like web servers & whatnot..

I'd love to get the missing syscalls worked out, and if we can run a bunch more of NS and OS, even chrooted, that is still damned impressive to me!

While it would have been nice if Apple would release or sell NS source, this is the NeXT best thing!

I got a bit further. The system creates a chroot on the fly to build and create each package installing debs that it wants and needs from a central repo. I'm seeing if I can cheese it right now to get around that. I just need to make a root filesystem deb.

So with some serious sight of hand, some patching, I managed to get the buildtools to kick.

I modified the base-build stuff to look for a package called "rootfs", then I tarred up the entire root filesystem, and manually made a deb file out of that. After a bit of patching to remove/replace i386 with ppc, I've got stuff building. I got architecture and architecture-hdrs to work.

Perl also cooked successfully but I didn't test it yet:
warning, `/private/tmp/roots/perl-5.004_04-1.roots/perl-5.004_04-1.root/private/tmp/roots/perl-5.004_04-1.roots/perl-5.004_04-1.dst/DEBIAN/control' contains user-defined field `Build-Depends'
dpkg-deb: building package `perl' in `/usr/src/dstroot//perl_5.004_04-1_i386-apple-rhapsody.deb'.
dpkg-deb: ignoring 1 warnings about the control file(s)

I need to cook the entire core dependencies and then we're in business.[/code]

Wow awesome!!!

Further update, I'm getting through the core dependencies as best I can but for the most part it seems happy. Decent chance I can get everything to build once I resolve all the build loops.

Disk space is becomign a problem however. I need to regularly delete tmp due to my hack which is a real headache.

EDIT: Driverkit failed to build (predictably) due to missing PCMICA headers, but I got cc to build and libc to build. I'm attacking cctools now. We likely need to get a git repo or something with all the patched source in it.

I just got up I'll add some disks and I guess a redirect in for port 80 so we can use Apache to get stuff out....

SON OF EDIT:
Libc failed :/
evs_api.c:41: drivers/event_status_driver.h: No such file or directory
evs_api.c:101: parse error before `NXOpenEventStatus'

I think that's as far as I'm going to get now, but there are a bunch of debs in /usr/src/sysroot.

This is the list we need to get building:

  # Tools
  'cc', 'cctools', #'objcunique',
  'file-cmds',
  'text-cmds', 'awk', 'grep',
  'shell-cmds',
  'developer-cmds',

  # Makefiles
  'pb-makefiles', 'coreosmakefiles',

  # Libraries
  'libsystem',

  # System
  'files',

  # Junk we should get rid of
  'csu',
  'libc-hdrs', 'gnumake',
  'basic-cmds',
  'architecture-hdrs', 'kernel-hdrs', 'gnutar',
  'project-makefiles', 'bootstrap-cmds', 'objc4-hdrs',
  'system-cmds',

It would be nice to have a cross-reference like the you had for Darwin 0.1

Csu is the c start up.. crt0 and friends...

I have to run to the office.  HK is like NYC,   I'm already at the train station...

What a morning, I still have the sleep in my eyes

Having spent consider time in Hong Kong, there was nothing worse than being shoehorned into the MRT and feeling like a sardine at 7am in the morning. I did love walking long the waterfront though, near the Muesum of Science.

EDIT: libc had a circular dependency on it's self. Resolved. Fixing other things now

Quote from: "NCommander"Having spent consider time in Hong Kong, there was nothing worse than being shoehorned into the MRT and feeling like a sardine at 7am in the morning. I did love walking long the waterfront though, near the Muesum of Science.

EDIT: libc had a circular dependency on it's self. Resolved. Fixing other things now

Oh cool!  I'm out in the territories as I'm not here on the expat, rent paid for thing...  I used to have my office in Wan Chai, but it was too much for too little and just far too busy.  I moved it to Do Tan for the same price, but 6x the size!

My wife wants to get a place in Japan which would be nice if we can get some land for a swimming pool or something nice like that.  Housing here makes London and New York look sane and sensible investments...  

The nice thing about being the boss means I can put the office close to my home (haha) one more station.

I cleared the libc build loop as well as some more fudging and got both libc and libcm to compile. cctools is building. If it links, we've got the toolchain in shape which should drastically simplify everything going forward.

Quote from: "NCommander"I cleared the libc build loop as well as some more fudging and got both libc and libcm to compile. cctools is building. If it links, we've got the toolchain in shape which should drastically simplify everything going forward.

cool.  I'm in the office see u on IRC.

it just hit me, I forgot to expand that disk to 8GB..  :oops:

With some effort, I got the essential system to all compile. I think we're ready to attempt a full build now

[darwin:src/build/buildtools-2] root# ls /usr/src/dstroot/
adv-cmds_24.1-2_i386-apple-rhapsody.deb                                     libedit_13-1_i386-apple-rhapsody.deb
architecture-hdrs_226-1_i386-apple-rhapsody.deb                             libinfo-hdrs_44.2-1_i386-apple-rhapsody.deb
architecture_226-1_i386-apple-rhapsody.deb                                  libinfo-obj_44.2-1_i386-apple-rhapsody.deb
awk_1998.10.20-1_i386-apple-rhapsody.deb                                    libinfo_44.2-1_i386-apple-rhapsody.deb
basic-cmds_25-1_i386-apple-rhapsody.deb                                     libkvm-obj_9.1-1_i386-apple-rhapsody.deb
bootstrap-cmds_13.2-1_i386-apple-rhapsody.deb                               libkvm_9.1-1_i386-apple-rhapsody.deb
cc-hdrs_783.1-1_i386-apple-rhapsody.deb                                     libm-obj_15-1_i386-apple-rhapsody.deb
cc_783.1-1_i386-apple-rhapsody.deb                                          libm_15-1_i386-apple-rhapsody.deb
cctools-obj_295-2_i386-apple-rhapsody.deb                                   librpcsvc-hdrs_8-1_i386-apple-rhapsody.deb
cctools_295-2_i386-apple-rhapsody.deb                                       librpcsvc-obj_8-1_i386-apple-rhapsody.deb
coreosmakefiles-hdrs_9.1-1_i386-apple-rhapsody.deb                          librpcsvc_8-1_i386-apple-rhapsody.deb
coreosmakefiles_9.1-1_i386-apple-rhapsody.deb                               libstreams-hdrs_15.2-1_i386-apple-rhapsody.deb
csu_23.1-1_i386-apple-rhapsody.deb                                          libstreams-obj_15.2-1_i386-apple-rhapsody.deb
developer-cmds_1998.10.05-1_i386-apple-rhapsody.deb                         libstreams_15.2-1_i386-apple-rhapsody.deb
driverkit-hdrs_139.1-3_i386-apple-rhapsody.deb                              libsystem-hdrs_25.1-2_i386-apple-rhapsody.deb
file-cmds_1998.10.06-1_i386-apple-rhapsody.deb                              libsystem-obj_25.1-2_i386-apple-rhapsody.deb
files_282.0-5_i386-apple-rhapsody.deb                                       libsystem_25.1-2_i386-apple-rhapsody.deb
flex_2.5.4a-1_i386-apple-rhapsody.deb                                       libtelnet-hdrs_11-1_i386-apple-rhapsody.deb
gnudiff_2.7-1_i386-apple-rhapsody.deb                                       libtelnet_11-1_i386-apple-rhapsody.deb
gnumake_67-1_i386-apple-rhapsody.deb                                        machkit-hdrs_318.2-1_i386-apple-rhapsody.deb
gnutar_1.12-1_i386-apple-rhapsody.deb                                       machkit_318.2-1_i386-apple-rhapsody.deb
grep_2.1-1_i386-apple-rhapsody.deb                                          network-cmds_57-1_i386-apple-rhapsody.deb
kernel-hdrs_154.5.1-7_i386-apple-rhapsody.deb                               objc4-hdrs_174-1_i386-apple-rhapsody.deb
libc-hdrs_78.8-1_i386-apple-rhapsody.deb                                    objc4-obj_174-1_i386-apple-rhapsody.deb
libc-obj_78.8-1_i386-apple-rhapsody.deb                                     objc4_174-1_i386-apple-rhapsody.deb
libc_78.8-1_i386-apple-rhapsody.deb                                         pb-makefiles-hdrs_89.5.1-1_i386-apple-rhapsody.deb
libcat_27.1-1_i386-apple-rhapsody.deb                                       pb-makefiles_89.5.1-1_i386-apple-rhapsody.deb
libcompat-hdrs_12-1_i386-apple-rhapsody.deb                                 perl_5.004_04-1_i386-apple-rhapsody.deb
libcompat_12-1_i386-apple-rhapsody.deb                                      project-makefiles-hdrs_118.2.1-1_i386-apple-rhapsody.deb
libcurses-hdrs_19-1_i386-apple-rhapsody.deb                                 project-makefiles_118.2.1-1_i386-apple-rhapsody.deb
libcurses-obj_19-1_i386-apple-rhapsody.deb                                  shell-cmds_1998.10.09-2_i386-apple-rhapsody.deb
libcurses_19-1_i386-apple-rhapsody.deb                                      system-cmds_51.1-2_i386-apple-rhapsody.deb
libedit-hdrs_13-1_i386-apple-rhapsody.deb                                   text-cmds_1997.11.10-1_i386-apple-rhapsody.deb
libedit-obj_13-1_i386-apple-rhapsody.deb                                    yacc_1998.10.12-1_i386-apple-rhapsody.deb


I added some more bits, and I'm @ single user!  :lol:

Drivers I'm using (and the bulk are the dreaded binary blob)

(source available)
EIDE
NE2K
(binary)
EISABus
Floppy
ISASerialPort
Intel824X0
PCIBus
PS2Keyboard
VGA

To say tremendous progress has been made today is an understatement.  I think we are at the 80/20 point in the source where we probably have 80% built.  Naturally the last 20 takes the most time.

One thing that would help me a lot is disabling netinfo.

is there a way to tell NS/OS/Rhapsody to ignore netinfo, and just use password files instead?

On the path,
nibindd needs the Foundation framework

and lookupd has the following unresolved symbols
NXFreeObjectBuffer
NXReadArray
NXReadObjectFromBuffer
NXReadTypes
NXWriteArray
NXWriteRootOBjectToBuffer
NXWriteTypes


Anyways, here we go.


So other than the drivers mentioned above, this is self hosted single user.  The only thing preventing it being multiuser is netinfo....  :D

Actually as I had suspected, the netinfo servers were re-written for Darwin 1.0 to not rely on the Foundation framework, as seen here in MachRPC.h (http://unix.superglobalmegacorp.com/cgi-bin/cvsweb.cgi/netinfo/servers/lookupd/MachRPC.h.diff?r1=1.1.1.1;r2=1.1.1.2;cvsroot=Darwin0;f=h).  Ill have a stab at building the 1.0 netinfo and see if that'll run allowing us to come up multiuser.

I also don't know why the boot linker (sarld?) links drivers just fine, but the runtime linker driverLoader silently fails.

Here's the failed to build list:

must build driverkit_139.1-3_i386-apple-rhapsody.deb using dir /usr/src/srcroot//driverkit-3
must build netinfo_33.0-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//netinfo-1
must build at-cmds_15.1-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//at_cmds-1
must build autoconf_2.12-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//autoconf-1
must build buildtools_0.1-2_i386-apple-rhapsody.deb using dir /usr/src/srcroot//buildtools-2
must build cdis_156.1-3_i386-apple-rhapsody.deb using dir /usr/src/srcroot//cdis-3
must build dpkg-scriptlib_0.2-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//dpkg_scriptlib-1
must build drveide_33-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//drvEIDE-1
must build gdb_142-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//gdb-1
must build groff_1.11a-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//groff-1
must build hfs_67.1-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//hfs-1
must build libgpp_100.1-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//libgpp-1
must build rcs_5.7-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//rcs-1
must build sendmail_8.9.1-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//sendmail-1
must build volfs_0-2_i386-apple-rhapsody.deb using dir /usr/src/srcroot//volfs-2
must build zsh_3.1.5-1_i386-apple-rhapsody.deb using dir /usr/src/srcroot//zsh-1

If newer NetInfo can build w/o Foundation, then we should probabaly drop that into the build and see what we can get to go. We can probably also create debs to package up components from DR2 or OS4.2 to clear the remaining failures.

I took a stab at the darwin 1.0 netinfo and found out that the objectivec runtime doesn't make a libarary so it can't link the first server with -lobjc

building by the manual way:

make OBJROOT=/build/objc4-1/obj SYMROOT=/build/objc4-1/sym DSTROOT=/build/objc4-1/roo RC_ARCHS=i386 RC_OS=teflon

and same thing it'll build all the objects but it doesn't actually create the library.  

wife+kids are screaming.  :cry:

Part of me wonders if we can get away with using the newer objc ...

dylds weren't very well supported until the Mac OS X era unfortunately.

I haven't figured out an easy way to get files in and out of the image.

Quote from: "NCommander"Part of me wonders if we can get away with using the newer objc ...

dylds weren't very well supported until the Mac OS X era unfortunately.

I haven't figured out an easy way to get files in and out of the image.

http://nextstep33.info:42380/debs/


And I've been using wget...

Didn't work for HTTPS websites though in hindsight that's predictable.

Quote from: "NCommander"Didn't work for HTTPS websites though in hindsight that's predictable.

Yeah.  Even if it did "work" it wouldn't.  

I need to see if that Apache does webdav.  Then we can just mount it

I just had a few mintues to compile expat + webdav.  I'll pm the incredibly weak credentials.

:lol:

random notes..

make OBJROOT=/build/libm-1/obj SYMROOT=/build/libm-1/sym DSTROOT=/build/libm-1/roo RC_ARCHS=i386 RC_OS=teflon

/usr/bin/libtool -static -arch_only i386 -o /build/libm-1/sym/libm.i386.a -filelist /build/libm-1/obj/static_obj/i386/m.ofileList,/build/libm-1/obj/static_obj/i386

/usr/bin/libtool -arch_only i386 -o /build/libm-1/sym/libm.dynlib -filelist /build/libm-1/obj/static_obj/i386/m.ofileList,/build/libm-1/obj/static_obj/i386

Deb files are just ar archives.  


ar x $1
tar -zxvf data.tar.gz
rm -f data.tar.gz
rm -f control.tar.gz
rm -f debian-binary


And chroot to make some tests.  Lots of things built as objects...  Not sure what is going on, or if that is bad.

obj is used for building the super libraries like libSystem. Install only the base debs for a chroot to get a system that would be close to reliability.

You can use dpkg -x to extract things somewhat saner.

Quote from: "NCommander"obj is used for building the super libraries like libSystem.

That is what I had thought, although for 1.0' netinfo I needed an libobjc.a which I spliced out of objc4 as a static lib.  Netinfod. Needs to know a bit about the structures from bootp, and other scattered renamed types in XNU...  Sigh I made good progress, only one daemon remaining to build, but the kids are crying. :oops:

I have to go to sleep I had  some time to play.

/usr/src/jason_build/netinfo.tar.gz

its the binary build!

the old style thing
make OBJROOT=/build/netinfo-167/obj SYMROOT=/build/netinfo-167/sym DSTROOT=/build/netinfo-167/roo RC_ARCHS=i386 RC_OS=teflon


slightly tweeked source for the Darwin 1.0 netinfo is in /tmp/test/netinfo-167

I used the /tmp/test fake chroot to build it.  I haven't had much luck reproducing it doing it with the tool thing though.  I forced copy my /System ontop of the fake one but it's always missing headers (wtf!?)

In the meantime I took a netinfo non networked rhapsody database shoved it on an image and.. now it'll come up multiuser

Latest binary is here: Darwin03_qemu090_30_5_2017.7z (https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin03_qemu090_30_5_2017.7z/download), if anyone wants to play along...

the immediate downside right now is that post sarld drivers are not loading.  i copied over my Darwin 0.1 kernel that I know works, but same thing, silent failure.  The only binary blobs on this is the drivers... its so close!

It hit me that the missing headers PCI/Eisa are part of those drivers.  Much like diskutils relying on drveide header exports these packages are not slightly clean and have subtle and intertwined dependencies, like netinfo needing bootp.

I found a few nice sites on gnustep which may be helpful http://gnustep.made-it.com/GSPT/xml/Tutorial_en.html  
and this one https://ubuntuforums.org/archive/index.php/t-1338278.html
https://ftp.nice.ch/pub/next/developer/languages/c/_gcc-i386.2.7.2.3.3.README.html
This following one is really good!
http://www.gnustep.org/resources/documentation/Developer/Base/Reference/Base.html I know this is probably in some cases redundant or completely irrelevant but I'm not sure how much gnustep is in  the project :) but I know the GNUstep developers have done a lot of amazing work as well!
Hoping it helps you all out as well at this point thinking that maybe there are tidbits or hints to help clean up that remaining 20% that will make it easier for you all.
I also know some of the longtime  gnustep developers will have a sincere interest in this  project perhaps a further hybrid of this beyondstep and gnustep fork ?

I was actually thinking SecondStep would be a more reasonable name.

I'm unfortunately stupidly tied up at work so I can't really put more time at this at the moment but I think what we need to do is get Darwin 0.3's core fully building. I think neozeed is probably on the right track that we're missing driverkit headers that's causing the remaining components to go bang (though zsh's build error is just frankly bizarre).

For NetInfo, given we can get the 10.0 NI to build, we should package that up and add it to the build system; there's a script that can load it right the right default data for a new build. That should be everything required to get us to multiuser console. (and all as free and open software to boot)

neozeed: what exactly is the driver failure you're hitting?

looks like more weirdness in the packages between 0.3 and 1.0

there is no netinfo headers target or I'm missing something.  this time I was more aggressive, I did a :

  220  15:33   rm -rf netinfo-1.67-167.root
  221  15:33   mkdir netinfo-1.67-167.root
  222  15:33   cd netinfo-1.67-167.root
  225  15:34   ( cd /tmp/test/ ; tar -cf - * ) | tar -xf -
  230  15:36   darwin-buildpackage --dir --target headers /usr/src/srcroot/netinfo-167/ /usr/src/repo/ /usr/src/dstroot/
  231  15:37   darwin-buildpackage --dir --target all /usr/src/srcroot/netinfo-167/ /usr/src/repo/ /usr/src/dstroot/


and now we have a deb at least.

# ls -l /usr/src/dstroot/netinfo_1.67-167_i386-apple-rhapsody.deb
-rw-r--r--  1 root  wheel  340478 May 30 15:40 /usr/src/dstroot/netinfo_1.67-167_i386-apple-rhapsody.deb


and we get the binaries...

usr
usr/sbin
usr/sbin/lookupd
usr/sbin/netinfod
usr/sbin/nibindd
usr/sbin/nidomain
usr/bin
usr/bin/nicl
usr/bin/nidump
usr/bin/nifind
usr/bin/nigrep
usr/bin/niload
usr/bin/nireport
usr/bin/niutil
System
System/Library
System/Library/CoreServices
System/Library/CoreServices/Resources
System/Library/CoreServices/Resources/English.lproj
System/Library/CoreServices/Resources/English.lproj/NetInfo.strings


Ill have a look how to manually build a headers package, and just do that for now.  But yes now we have a netinfo.  On that qemu image my other build of netinfo works enough to go multi-user.  So that is nice.  I should try to see if it can build itself.

I do know that it's own headers missing is a problem but there is other weirdness that I had corrected in /tmp/test although now Im not sure what it was.... Ill have to try it again.

[darwin:/tmp/x] root# df -k
Filesystem  1K-blocks     Used    Avail Capacity  Mounted on
/dev/hd0a     1980038   246784  1634253    13%    /
fdesc               1        1        0   100%    /private/dev
/dev/hd1a     7998800  1486129  6112731    19%    /usr/src
/dev/hd2a     7998800  1015697  6583163    13%    /private/tmp


at least we are OK for space.

I was almost thinking of PriOS   Although some buzz word bingo vendor (https://www.prios1.com/about.aspx) already has it.

Do you guys think that any of the ancient build instructions from OpenDarwin might be of any help at all, or are they too new?

For example, this seems quite interesting:  https://web.archive.org/web/20060715194412if_/http://www.opendarwin.org:80/projects/darwinbuild/doc/build/index.html

Quote from: "t-rexky"Do you guys think that any of the ancient build instructions from OpenDarwin might be of any help at all, or are they too new?

I'm just a monkey banging on the keyboard.

Any and all information is great!!

Also, how do you bring up a netinfo db?

There must be some script..  or a way to back it up and walk the tree?  For now, the one I copied from Rhapsody is working well enough.  But, netinfo really is the key service in all of this.

NEXT_ROOT=
The NEXT_ROOT variable allows for the finished products to be rooted somewhere other than /. Its presence here is for legacy support only. In Mac OS X, all projects are installed into /, so this is always an empty string.

Huh.  Sounds more like my many versions in chroots was a thing...

Almost all Darwin projects support an alternate target called "installhdrs". Building with this target instead of "install" causes only the header files produced by the project to be installed into $DSTROOT. When Apple builds Mac OS X, and "installhdrs" build is performed on every project that supports it. The resulting header files are all installed into the build root before the full build begins.
Since dependencies are calculated at a project-level granularity, but two projects often need access to each others' header files, the installhdrs phase helps avoid most circular dependency issues.

This right here is the most deceptive part about these source projects.  Although they are packaged as small units, they are heavily interdependent on each other.  As we found out that the EIDE driver is required to make the filesystem utilities, and no doubt, the only reason it was released, unlike EISAbus and PCIbus which are only needed for the PC.  The fact that the 0.1 netinfo requires the foundation framework shows that none of this was actually tested in any clean room manner.

I should try sheepshaver to see if it can build a Darwin disk, as it is just an image restore... I highly doubt OS X disktool can read it though.  Failing that, anyone have a super old beige G3?[/code]

And here is something else I had in my archives that you might find useful:

https://web.archive.org/web/20011109050008/http://www.publicsource.apple.com:80/news/qa20010925.html#GetStarted

https://web.archive.org/web/20020815122641/http://publicsource.apple.com:80/projects/documentation/howto/html/Building_Darwin_HOWTO.html

libg++ really really wants to do a powerpc build for some reason.  it did do an i386 one though so I cheated, and just copied the i386 onto the empty ppc directory and blamo, a deb was built...

I tried setting RC_ARCHS=i386 in the Makefile but that didn't have the desired outcome.  I guess it's ppc build structure is further burred in there.

Oh well got a few more to build.

I patched more through gdb only to find that it also has roots on the NeXTSTEP side to Foundation Kit.

Off to Darwin 1.0's gdb then..


ugh.


cc -arch i386 -traditional-cpp -c -g -O3 -Wall -Wimplicit  -Dunix -D__unix -D__unix__ -DNX_COMPILER_RELEASE_3_0=300 -DNX_COMPILER_RELEASE_3_1=310 -DNX_COMPILER_RELEASE_3_2=320 -DNX_COMPILER_RELEASE_3_3=330 -DNX_CURRENT_COMPILER_RELEASE=520 -DNS_TARGET=52 -DNS_TARGET_MAJOR=5 -DNS_TARGET_MINOR=2 -DNeXT -D__NeXT -D__NeXT__ -D_NEXT_SOURCE -DTARGET_I386 -DNS_TARGET_MAJOR=5 -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support  -I. -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/config -DHAVE_CONFIG_H -Dfree=xfree -I /System/Library/PrivateFrameworks/opcodes.framework/Headers -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../readline/.. -I /System/Library/PrivateFrameworks/bfd.framework/Headers -I /System/Library/PrivateFrameworks/electric-fence.framework/Headers -I /System/Library/PrivateFrameworks/mmalloc.framework/Headers -I /System/Library/PrivateFrameworks/binutils.framework/Headers -I../intl -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../intl -I/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/tui -DUI_OUT=1 -Wimplicit -Wreturn-type -Wcomment -Wtrigraphs -Wformat -Wparentheses -Wpointer-arith  /private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController.m
In file included from /private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController.m:1:
/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController_Private.h:1: Foundation/Foundation.h: No such file or directory
In file included from /private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController_Private.h:3,
                from /private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController.m:1:
/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/GdbManager.h:1: Foundation/Foundation.h: No such file or directory
/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController.m:3: Foundation/Foundation.h: No such file or directory
In file included from /private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/DebuggerController.m:14:
/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166/src/gdb/../gdb-next/display-support/debug.h:4: Foundation/NSLock.h: No such file or directory
make[5]: *** [DebuggerController.o] Error 1
make[5]: Leaving directory `/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166.obj/i386-apple-rhapsody--i386-apple-rhapsody/gdb'
make[4]: *** [/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166.obj/i386-apple-rhapsody--i386-apple-rhapsody/stamp-build-gdb] Error 2
make[4]: Leaving directory `/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166'
make[3]: *** [build-gdb] Error 2
make[3]: Leaving directory `/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166'
make[2]: *** [build] Error 2
make[2]: Leaving directory `/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166'
make[1]: *** [install-nosrc] Error 2
make[1]: Leaving directory `/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166'
make: *** [install] Error 2
make: Leaving directory `/private/tmp/roots/gdb-4.18-1990707-166.roots/gdb-4.18-1990707-166'
Uncaught exception from user code:
       failed with status 2 at /usr/lib/perl5/Dpkg/Package/Builder.pm line 872.
       Dpkg::Package::Builder::build('dir', '/Local/Users/Administrator/gdb-166', 'ARRAY(0xe3ec)', 'all', '/usr/src/dstroot/', 0) called at /usr/bin/darwin-buildpackage line 41


Quote from: "neozeed"
I should try sheepshaver to see if it can build a Darwin disk, as it is just an image restore... I highly doubt OS X disktool can read it though.  Failing that, anyone have a super old beige G3?

I do! Stored nicely away in the garage.



-Daxziz

Well.. I just checked, and the 'newest' kernel I just build *can* load modules...


Rhapsody darwin 5.5 Kernel Release 5.5: Tue May 30 23:58:50 PDT 2017; root Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.  i386


However... the base system seems to have issues loading modules.

I'm going to try some mass copies around to see what shakes loose.

bigger success!  I got modules to load so my test VM now has things like networking.  8)

[darwin:/etc] root# cd /private/
[darwin:/private] root# ls
Devices  Drivers  cores    dev      etc      tftpboot tmp      var
[darwin:/private] root# ls -al
total 18
drwxr-xr-x   9 root  wheel  1024 May 31 02:50 .
drwxr-xr-x  10 root  wheel  1024 May 31 02:50 ..
lrwxr-xr-x   1 root  wheel    12 May 31 02:50 Devices -> Drivers/i386
drwxr-xr-x   3 root  wheel  1024 May 31 02:45 Drivers


It turns out that the symlink for Devices to point to Drivers/i386 is CRITICAL to get device drivers to work.

I've done this 100% with the debs we've compiled....


[darwin:/private] root# df -k
Filesystem  1K-blocks     Used    Avail Capacity  Mounted on
/dev/hd0a     7960023   117165  7444857     1%    /



So yeah things are looking up again!  :D

You are a mad-man-machine neozeed!

Quote from: "t-rexky"You are a mad-man-machine neozeed!

Yeah im on fire at the moment.

Darwin03_debs_source_31_5_2017.iso (https://sourceforge.net/projects/aapl-darwin/files/Revived/Darwin03_debs_source_31_5_2017.iso/download)

I made an ISO with the binary debs + source.  Sigh if I had something to go on for making the UFS disk, I could do a live CD, but we aren't there.

I've used these binaries to bootstrap a VM, and get it on the internet, so we are certainly on the right path.

The netinfo is sucking badly as it's clearly not married to the machine properly...   Oh well slowly but surely.

I'm not feeling the love for my contributions ;)

NetInfo is likely straightforward enough to package and put in deb form, but we've essentially got a bootable free and open source core system. Well, excluding the network driver unless the NE2000 one is FOSS. I might take a stab at it this weekend.

Still, considering we have the core of crap working. Yay.

Quote from: "NCommander"I'm not feeling the love for my contributions ;)

lol without you I wouldn't have pushed anywhere near as hard.  Actually the NE2K driver does have source, so we are out PCI/EISA, and some other minor ones (maybe major, the floppy!?)

I've just setup a vm with all the debs on my laptop but the build system still had that hard coded powerpc affinity so I think I've "fixed" it, and Ill try to kick off a make everything before I go to sleep...

I tried Qemu 0.11 and it's complaining about some weird DMA modes on the IDE driver before it hangs...  Maybe it's a combo of crappy LBA+DMA being programmed slightly wrong that messes up damned near everything out there.  I'll have to try that later.  I assume you saw that all of /usr is shared out so you can snag stuff easier.  Windows 10 didn't like to WebDAV to this thing for some reason.  Probably the lack of crypto

May be worth forcing the EIDE driver into CHS mode. Notably it works flawlessly on VirtualBox but I haven't been able to fish out the AMDnet PC driver to use it there.

Quote from: "NCommander"May be worth forcing the EIDE driver into CHS mode. Notably it works flawlessly on VirtualBox but I haven't been able to fish out the AMDnet PC driver to use it there.

It doesn't work on so many other machines along with emulators, which is where the problem lies...   It's the heart of the reason why building a white box is such a massive PITA.

bash-2.02# darwin-missing /usr/src/srcroot/ /usr/src/dstroot/
must build gdb_142-1_i386-apple-darwin.deb using dir /usr/src/srcroot//gdb-1
must build rcs_5.7-1_i386-apple-darwin.deb using dir /usr/src/srcroot//rcs-1
must build zsh_3.1.5-1_i386-apple-darwin.deb using dir /usr/src/srcroot//zsh-1


With a little massaging, the AppleTalk stuff is set to PowerPC only, and I think I got the switch to tell libgpp to be i386 only.  

Boot wants some headers from driverkit that aren't actually part of it (pci etc..) and rcs & zsh both fail building in a jail with some weird file handle error running scripts.  But they build find on hte outside, so really the only thing remaining is GDB, with it's hooks into the UI through Foundation kit.  I'll try to look at amputation later on to clear the last hurdle!!

bash-2.02# df -k
Filesystem  1K-blocks     Used    Avail Capacity  Mounted on
/dev/hd0a     7998800  5699325  1899535    75%    /


Thankfully, it all fits in 8GB!  8)

I know when installing custom drivers on NeXTstep or Openstep for intel they always go into the path /private/Drivers/i386

This is a good read on Rhapsody driver tweaks https://www.williamreading.com/2004/04/25/rhapsody.html

This may help jump through some hoops....

I found this as well which may help as it looks like there is some cross over

Creation Date: January 18, 1996
Last Updated: <<Date June 17, 1997>>

Table of Contents

....Overview
....Definitions
....Screen Goes Black
....Convincing the Booter That You Actually Have Drivers
....Bypassing Configure
....Bypassing BuildDisk

Overview

Sometimes, the NEXTSTEP installation programs can get confused about your system configuration, or there may be some incompatibility that prevents you from going through a certain phase of installation.  Often, if you can jump over these hurdles, you can still get NEXTSTEP installed and running on your system.  This NeXTanswer details why the installation process does what it does during certain phases of the installation, the steps to take to bypass them.

Definitions

For the sake of clarity let's define two phases of installation.
Phase 1: This is when you initially boot your system off of the installation floppy and load the software off the CD-ROM.  It goes up to the point when the dots have finished travelling across the screen, and the system reboots.
Phase 2: This is the configuration and package installation phase.  When the system reboots off the hard disk, you still need to have the CD-ROM in the drive to load packages from, but all essential information is already on your hard disk.  It goes up to the end of package installation, and subsequent reboot.
Phase 3: This is when the system is fully up and running, and you are asked to configure the language and keyboard preferences.

Screen Goes Black

When booting some systems, the screen may just go black, and the system will hang.  On some systems this can be prevented by booting in verbose mode.  Type "-v" at the boot: prompt to boot in verbose mode.

Convincing the Booter That You Actually Have Drivers

When rebooting into Phase 2, the booter will determine if you used drivers that were not on the CD-ROM during installation.  If so, it will ask you to insert the driver floppy.  Sometimes, it may have trouble reading the driver floppy.  You can boot up in single-user mode off the CD-ROM and copy the drivers onto the hard drive.

To boot single-user off the CD-ROM:
1. Make a copy of the 3.3 Core Drivers Floppy and the 3.3 Additional Drivers Floppy (see 1921_Creating_FDs_From_Images_Under_NEXTSTEP_and_DOS.rtf)
2. Boot off the installation floppy as if you were going to install, but type "-s" at the boot: prompt.  (You will be prompted for the driver floppy, and everything will proceed the same as a regular installation, but it will stop short of actually installing.)
3. Make sure that you specify the Floppy Driver off of the Core Drivers Floppy as well as the regular boot drivers. You need it so you can use the floppy drive in single user mode.
4. At the # prompt, type "mount -n -o -ro /dev/hd0a /Disk" (use /dev/sd0a if it is a SCSI drive) to mount the hard drive.
5. Insert your floppy with the additional drivers on it that you need to copy over.
6. At the # prompt, type "mount -n /dev/fd0a /Floppy" to mount the floppy.
7. At the # prompt, type "cp -rp /Floppy/private/Drivers/i386/drivername.config /Disk/private/Drivers/i386/" to copy a driver to the hard drive.
8. At the # prompt, type "mv /private/Drivers/i386/System.config/InstallHints /private/Drivers/i386/System.config/InstallHints.old" to rename the InstallHints file.  Both the booter and Configure use the InstallHints file to determine if you used drivers that were not on the CD-ROM.

Bypassing Configure

Configure is the first graphical program that comes up in Phase 2.  For some machines, it can crash the installation process and render your system unbootable.  To bypass Configure, you can boot up in single-user mode off the hard drive and create the Instance0.table file.

To do this:
1. Boot normally off the hard drive, but type "-s" at the boot: prompt
2. At the # prompt, type "cd /private/Drivers/i386/System.config"    :)
3. At the # prompt, type "cp Default.table Instance0.table"
4. At the # prompt, type "exit"

This will exit single-user mode, and continue on with the rest of the configuration.  After you've completed the rest of the configuration, you should be able to log in as root, launch Configure.app, and configure all the drivers for your system.

Note: You should always save and quit in Configure, then log out and reboot your machine for changes to take effect.  Doing the "quick reboot" could cause your disk to become corrupted and unbootable.
Warning: Because you created the Instance0.table by hand, it could become out of sync with what Configure shows you.  After running Configure to set up your drivers, you should check the Instance0.table manally (by viewing it in Edit) to make sure that drivers are not listed twice, and that extra drivers are not listed.

Bypassing BuildDisk

BuildDisk is the second graphical program that comes up in Phase 2.  It is the one that lets you choose what packages to install.  Sometimes, especially with large disks, BuildDisk can become confused and report that you have little or no disk space available to install packages.  It won't let you continue beyond this point.  You can bypass BuildDisk by booting up in in single-user mode off the CD-ROM and creating the /usr/adm/BuildDisk.custom file.

To do this:
1. Boot normally off the hard drive, but type "-s" at the boot: prompt
2. At the # prompt, type "touch /usr/adm/BuildDisk.custom" to create the BuildDisk.custom file.  :)

4. At the # prompt, type "exit"

This will exit single-user mode and allow you to continue with the rest of the installation.  NEXTSTEP has been fully installed and functional at this point, but none of the extra packages have been installed.  You can install them individually by logging in as root and double clicking on the packages in the folder /NeXTCD/Packages on the CD-ROM.

Since I'm in my office, I've loaded up a VMWare instance, and I'm rebuilding everything again to see if my hacks are good enough to 'just work'

I hit the 8GB ceiling hard.  Got a nice 'float' error accessing blocks over 8,000,000 or so.  without GDB not sure what was going on.

stumbling blocks...

driverkit/configTablePrivate.h
driverkit/i386/PCI.h

cp /usr/src/srcroot//driverkit-3/driverkit/configTablePrivate.h /tmp/roots/boot-64-2.roots/boot-64-2.root/System/Library/Frameworks/System.framework/Headers/driverkit/
mkdir /tmp/roots/boot-64-2.roots/boot-64-2.root/System/Library/Frameworks/System.framework/Headers/driverkit/i386
cp /usr/src/srcroot/driverkit-3//driverkit/i386/PCI.h /tmp/roots/boot-64-2.roots/boot-64-2.root/System/Library/Frameworks/System.framework/Headers/driverkit/i386



ipconfig/ipconfig_ext.h

mkdir /tmp/roots/netinfo-1.67-167.roots/netinfo-1.67-167.root/System/Library/Frameworks/System.framework/Headers/ipconfig
cp /tmp/roots/netinfo-1.67-167.roots//netinfo-1.67-167.root/Local/Developer/Headers/ipconfig/* /tmp/roots/netinfo-1.67-167.roots/netinfo-1.67-167.root/System/Library/Frameworks/System.framework/Headers/ipconfig



I don't get why it includes . ..   :roll:

Anyways

bash-2.02# darwin-missing /usr/src/srcroot/ /usr/src/dstroot/
must build ._0_i386-apple-darwin.deb using dir /usr/src/srcroot//.
must build .._0_i386-apple-darwin.deb using dir /usr/src/srcroot//..
must build gdb_142-1_i386-apple-darwin.deb using dir /usr/src/srcroot//gdb-1
must build rcs_5.7-1_i386-apple-darwin.deb using dir /usr/src/srcroot//rcs-1
must build zsh_3.1.5-1_i386-apple-darwin.deb using dir /usr/src/srcroot//zsh-1
must build build.sh_0_i386-apple-darwin.deb using dir /usr/src/srcroot//build.sh


So rcs & zsh build outside ok, and GDB has that foundation kit issue so yeah I got a rebuild! ... time to purge and do it again.

Hello: I found this document  which helps break the differences between NeXTSTEP and openstep and how to pport NeXTSTEP apps to Openstep apps , I'm guessing this may offer some clues  as to why you may be getting certain errors
http://www.nextcomputers.org/NeXTfiles/Docs/Software/OPENSTEP/802-2115.pdf

And this one how to port from NeXTstep to Openstep to Rhapsody to OSX
http://www.rhapsodyos.org/

Quote from: "Rob Blessin Black Hole"Hello: I found this document  which helps break the differences between NeXTSTEP and openstep and how to pport NeXTSTEP apps to Openstep apps , I'm guessing this may offer some clues  as to why you may be getting certain errors
http://www.nextcomputers.org/NeXTfiles/Docs/Software/OPENSTEP/802-2115.pdf

I'm not sure I think that's the PDO over OPENSTEP, although the last thing Im going to do is turn down information!

First thing I see is CORBA & database access... oh dear this is where WebObjects was born.  Back in the day I did have an EOF CD, and I did image it, but I never did get PDO.. What is interesting is that the objectiveC compiler does have PDO hooks so it ought to build for OpenSTEP.  Or at least somewhat build.

I've seen a few mentions of SUN/PDO and HP/PDO along with NT/PDO here and there.

As it mentions:


Foundation Kit OpenStep Foundation Kit is an enhanced
superset of the NEXTSTEP 3.3 Foundation
Kit. The OpenStep Kit contains API
modifications and additions.

While we have no Foundation Kit...

Even with the older Netinfo we had the following un-resolved:


NXFreeObjectBuffer
NXReadArray
NXReadObjectFromBuffer
NXReadTypes
NXWriteArray
NXWriteRootOBjectToBuffer
NXWriteTypes

I don't see any GNUStep equivalents.  Although it does look like an object library... At the surface not that involved, but Im sure it's chock full of edge case exceptions.

I'm fairly badly swamped right now so I can't dig into this, but the CoreFoundation problem is a serious headache that ultimately killed OpenDarwin. That being said, later Darwin's were self-hostable (at least 10.4 was).

PureDarwin reimplemented the wheel here (https://github.com/PureDarwin/PureFoundation) so theorically we have the necessary bits to get the compiler to go, but honestly, I'm debating if we're saner simply grabbing what we need from DR2. That would mean we're dependent on a propertiary blob which kinda sucks.

EDIT: Looking through Darwin 10.4.x86, I was right, CoreFoundation was released, but it looks like Apple took a fairly large bludgen to NetInfo so it could build standalone. Might be worth a shot ...

Quote from: "NCommander"I'm fairly badly swamped right now so I can't dig into this, but the CoreFoundation problem is a serious headache that ultimately killed OpenDarwin. That being said, later Darwin's were self-hostable (at least 10.4 was).

PureDarwin reimplemented the wheel here (https://github.com/PureDarwin/PureFoundation) so theorically we have the necessary bits to get the compiler to go, but honestly, I'm debating if we're saner simply grabbing what we need from DR2. That would mean we're dependent on a propertiary blob which kinda sucks.

EDIT: Looking through Darwin 10.4.x86, I was right, CoreFoundation was released, but it looks like Apple took a fairly large bludgen to NetInfo so it could build standalone. Might be worth a shot ...

It's ok, the netinfo from 1.0 is running!  And I can login to myself.  The only binary blobs is some device drivers.

This is a VM running on my desk.  Check the kernel build date!

# telnet 192.168.1.55
Trying 192.168.1.55...
Connected to 192.168.1.55.
Escape character is '^]'.


4.4 BSD (vdarwin) (ttyp1)

login: root
Password:
tcsh: No entry for terminal type "linux"
tcsh: using dumb terminal settings.
[vdarwin:~] root# hostinfo
Mach kernel version:
        Kernel Release 5.5:
Thu Jun  1 04:15:23 PDT 2017; root
Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.

Kernel configured for a single processor only.
1 processor is physically available.
Processor type: pentium (Intel Pentium)
Processor active: 0
Primary memory available: 384.00 megabytes.
Default processor set: 34 tasks, 71 threads, 1 processors
Load average: 1.56, Mach factor: 0.08
[vdarwin:~] root#



I'll upload an iso later on with all the binaries.  If I could make UFS CD-ROM'S I'd make a bootable distro.  But really the only missing package out of the source is GDB.

I too was occupied with work as well, so don't worry, but Tha KS to those build scripts I could let my PC hammer out a build.

Do you still have the build order to really bootstrap the whole thing?  The build all seems as dumb as me using bash and doing a "for i in *;do ./build.sh $i;done" ....

Ii think I'm at the point where I want to start incorporating some of the bsd kernel changes from 0.3 to 1.0. At a minimum it looks like there is a bunch of NFS fixes.

Build order was only important for bootstrapping. Just make sure the entire set of debs are in repo and then let it rip. It will pull from dstroot over repo if necessary (I think it prefers it).

Quote from: "NCommander"Build order was only important for bootstrapping. Just make sure the entire set of debs are in repo and then let it rip. It will pull from dstroot over repo if necessary (I think it prefers it).

I wanted to rebuild on a bootstrap order to make sure I didn't ******** anything up in some cascading hidden failure...

Otherwise, that VMware box is actually a 3rd generation!

I made it build 2x so it's Deb's are 5th generation.

While poking around I saw cc has a "Grail" target... I386/m68k/hppa/sparc.

--edit

Also I'm scared to say, but after the 1st built kernels failure to load drivers, every one onward I have used, has worked.  It looks like there really was a subtle change from the DR2 to 0.1/0.3 ... I'm on the train.  I'm going to make a 5th gen install and see how it goes.  But yeah TL;DR we are 98% self hosting!

http://mirror.informatimago.com/next/developer.apple.com/documentation/Darwin/Conceptual/howto/booting_on_x86/booting_on_x86.html

QuoteCreate a root filesystem

Now you just need to make a root filesystem for the CD. To do this, put all the things you need into one directory. For instance, assume /cdimage will be the "/" for your CD. You can put whatever you want here.

The list of Darwin projects that are used on Darwin/x86 install CD are:

iokituser
xnu
startup
Libsystem
boot
diskdev_cmds
text_cmds
zsh
basic_cmds
awk
file_cmds
shell_cmds
system_cmds
files
adv_cmds
corefoundation
zlib
cctools
Once you have built yourself a suitable "/" in /cdimage, you'll need to create the actual ISO image of the cdrom. First, copy the boot floppy to an appropriate place on the cd:

cp boot.flp /cdimage/usr/standalone/i386/boot.flp
And then create the image. You'll need the "mkisofs" project installed on your machine.

mkhybrid -r -J -h -c user/standalone/i386/boot.catalog -b   \
     usr/standalone/i386/boot.flp -o darwincd.iso /cdimage
You will now have a bootable Darwin/x86 cd image ready to be burned to disc.

So in the Darwin 1.x days, yeah HFS CD's....

In between circuit failures, and firewall issues I had time to do an overlay

I did a darwin full 'install' and deleted the /private directory,   Then I took a DR2 install and overlaid the darwin on top.

It actually boots up into the UI, and let's me login.  Things mostly work, however there is some expected falling bits.

A *LOT* of applications will not run.

Jun  2 13:12:43 rhapsody /System/Administration/BuildDisk.app/BuildDisk: objc: please link appropriate classes in your program
Jun  2 13:12:44 rhapsody /System/Administration/Configure.app/Configure: objc: please link appropriate classes in your program


This is the most verbose I could get out of clicking like crazy.  A lot of demos run fine.  It's also good to know the Darwin 1.0 netinfo is working fine with DR2.

Jun 02 14:07:10 Terminal[329] Can't connect to the distributed notification server: bootstrap_look_up returned bootstrap unknown service.
dyld: ./Terminal Undefined symbols:
_NXFreeObjectBuffer
_NXReadArray
_NXReadObjectFromBuffer
_NXReadTypes
_NXWriteArray
_NXWriteRootObjectToBuffer
_NXWriteTypes


It looks like these are all in objc-1\typedstream.m so... I'm unsure about the linkage and why these are 'missing'...

As predicted.

Rhapsody:

rhapsody:13# otool -v -T /System/Library/Frameworks/System.framework/Versions/B/System | grep typedstream
typedstream.o    _NXCloseTypedStream
typedstream.o    _NXEndOfTypedStream
typedstream.o    _NXFlushTypedStream
typedstream.o    _NXFreeObjectBuffer
typedstream.o    _NXGetTypedStreamZone
typedstream.o    _NXOpenTypedStream
typedstream.o    _NXOpenTypedStreamForFile
typedstream.o    _NXReadArray
typedstream.o    _NXReadObject
typedstream.o    _NXReadObjectFromBuffer
typedstream.o    _NXReadObjectFromBufferWithZone
typedstream.o    _NXReadType
typedstream.o    _NXReadTypes
typedstream.o    _NXSetTypedStreamZone
typedstream.o    _NXSystemVersion
typedstream.o    _NXTypedStreamClassVersion
typedstream.o    _NXTypedStreamIsSwapped
typedstream.o    _NXWriteArray
typedstream.o    _NXWriteObject
typedstream.o    _NXWriteObjectReference
typedstream.o    _NXWriteRootObject
typedstream.o    _NXWriteRootObjectToBuffer
typedstream.o    _NXWriteType
typedstream.o    _NXWriteTypes
typedstream.o    _getSizeOfType


Darwin:

rhapsody:3# otool -v -T /System/Library/Frameworks/System.framework/Versions/B/System | grep typedstream


More digging to be done...

It looks like on the older DR2 systems, NX* was part of libSystem and in Foundation where it lives now. That would actually explain why NeXTstep software will not run on DR2 due to the library linker not pulling in the right bits even though from an ABI perspective, shat more or less stayed the same until Mac OS X 10.0 and IOKit.

Theoretically, if we could solve that, we would be able to run NS/OS apps under Rhaspody.

I'm sort of tempted to try Darwin 1.0 and see if it will run Rhaspody 5.5. stuff in a chroot. No hope of getting WindowServer to go but just userland compatbility.

Quote from: "NCommander"It looks like on the older DR2 systems, NX* was part of libSystem and in Foundation where it lives now. That would actually explain why NeXTstep software will not run on DR2 due to the library linker not pulling in the right bits even though from an ABI perspective, shat more or less stayed the same until Mac OS X 10.0 and IOKit.

Theoretically, if we could solve that, we would be able to run NS/OS apps under Rhaspody.

I'm sort of tempted to try Darwin 1.0 and see if it will run Rhaspody 5.5. stuff in a chroot. No hope of getting WindowServer to go but just userland compatbility.

I tried 10.12.3 .. it's all killed signal 9.  chroot or not.  Kind of disapointing, but not surprising.  I don't think x64 is that compatible.

http://lisp.gotgeeks.com/Documentation/NEXTSTEP-0.8/RelNotes/OSNotes.txt

QuoteFuture Directions

   The following shared libraries will be merged into one library by
   release 0.9 or 1.0:

       libappkit_s.a
       libdpsclient_s.a
       libstreams_s.a
       libdb_s.a
       libobjc_s.a

Lots of documents on this site, including release notes.  

In the advent we can't build everything that ought to be in System, I guess a 'plan b' would be a way to extract a target's System objects, overlay them with what we do have, and re-link that?  Is that even possible?  I've only done something like this a long long time ago.[/url]

I ran out of space on my VMware build machine, so I thought rebooting would clear tmp... Nope!

It just rename the full directory....

If this rehash goes well I'll post an updated CD with source + binaries.

I'll also finally setup some CVS to work with the tools of the era.

I have to go out to do some chores but I'm still feeling motivated which is good.

Should I do a listserv?

I don't know if I'm just polluting here too much, or if people like to see the (non) progress?  I'm nowhere near as talented as andreas_g, so we shall see how far we get ..

I think it's important to document the non-progress as well. It shows the steps taken and the thoughts behind them.


-Daxziz

Because I'm not good with diff or sed....

change the 1.1 license to 2.0

find . -type f -exec sed -i "s/Copyright (c) 1999 Apple Computer, Inc. All rights reserved./Copyright (c) 2000 Apple Computer, Inc. All rights reserved./g" {} \;
find . -type f -exec sed -i "s/Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights/The contents of this file constitute Original Code as defined in and/g" {} \;
find . -type f -exec sed -i "s/Reserved.  This file contains Original Code and\/or Modifications of/are subject to the Apple Public Source License Version 1\.1 \(the/g" {} \;

find . -type f -exec sed -i "s/Original Code as defined in and that are subject to the Apple Public/\"License\"\).  You may not use this file except in compliance with the/g" {} \;
find . -type f -exec sed -i "s/Source License Version 1.1 (the \"License\").  You may not use this file/License.  Please obtain a copy of the License at/g" {} \;
find . -type f -exec sed -i "s/except in compliance with the License.  Please obtain a copy of the/http:\/\/www.apple.com\/publicsource and read it before using this file./g" {} \;
find . -type f -exec sed -i "s/The Original Code and all software distributed under the License are/This Original Code and all software distributed under the License are/g" {} \;


find . -type f -exec sed -i "/ * License at http:\/\/www.apple.com\/publicsource and read it before using/d" {} \;
find . -type f -exec sed -i "s/ * this file./thislineneedstobedeleted/g" {} \;
find . -type f -exec sed -i "/ *thislineneedstobedeleted/d" {} \;


to get a better handle on changes in the BSD kernel going from Rhapsody to XNU

http://unix.superglobalmegacorp.com/cgi-bin/cvsweb.cgi/kernel-bsd/?cvsroot=DarwinBSD;only_with_tag=MAIN

spot the difference?

[vdarwin:/usr/src/srcroot_lastknowngood] root# diff -ruN cc-1 /tmp/cc-1
[vdarwin:/usr/src/srcroot_lastknowngood] root# du -ks cc-1
25239   cc-1
[vdarwin:/usr/src/srcroot_lastknowngood] root# du -ks /tmp/cc-1
25239   /tmp/cc-1


Sure looks identical, right? except one builds, the other does not.   :x

Quote from: "neozeed"spot the difference?

Code Select Expand

[vdarwin:/usr/src/srcroot_lastknowngood] root# diff -ruN cc-1 /tmp/cc-1
[vdarwin:/usr/src/srcroot_lastknowngood] root# du -ks cc-1
25239   cc-1
[vdarwin:/usr/src/srcroot_lastknowngood] root# du -ks /tmp/cc-1
25239   /tmp/cc-1


Sure looks identical, right? except one builds, the other does not.   :x

How is that even remotely possible?  Maybe file permissions?  Maybe you run out of enough free space to build the second one?  Cannot really see any other possibilities...

Quote from: "t-rexky"

How is that even remotely possible?  Maybe file permissions?  Maybe you run out of enough free space to build the second one?  Cannot really see any other possibilities...

I'm totally at a loss.  In between stuff, I know all the tools love CVS, so I setup a CVS server, committed my source tree, fetched it, and about 10 things fail to build, they all fail in some weird desire for textinfo, meanwhile the originals that diff with zero compares build without issue.

Granted I'm 100% n00b with apt/Deb and without ncommander I know I'd probably skip the package / build stuff all together, but yeah it's .. weird.

This reminds me of the onslaught of voodoo that I have experienced while trying to build gcc and some other gnu tools on NS3.3.

If you look at any of my diffs you will see weirdness in the configure scripts, for example, where I had to add fluff text characters because creating a file on NS3.3 of zero length and then appending to it later sometimes results in random binary garbage being inserted into the file.  This is not supposed to happen unless you are using -posix, but it obviously is happening even without -posix.  So it's either a kernel bug or a library bug in NS3.3 and it is really, really annoying.

This cannot be always reproduced so whenever I tried to identify the root cause I could not reproduce it, but whenever I was running the configure scripts it almost always happened.  So I brute-force "fixed" it by avoiding creation of empty files that are appended to: they are created with some "fluff" content that avoids the bug.

This is just one thing, but the list of seemingly random weirdness goes on-and-on.  I have, somewhere, NS3.3 "porting notes" from few years back.  But one would think that they fixed all that stuff by the time Rhapsody came about...

Quote from: "t-rexky"But one would think that they fixed all that stuff by the time Rhapsody came about...

NeXT really was a small company.  And they did a lot of weirdisims to 'NIH' basically everything they touched.  It sure is apparent in the Darwin source.

It's not like we have access to much older source to compare.  I'm almost tempted to pony up $10k to find out.

QuoteN5515           NeXT 2.0 Mach Source Release   $10,000 (Q1)  $5,000
          on Floppies
          (not in Fall 90 list prices

link for the curious (https://groups.google.com/d/msg/comp.sys.next/jT5FcOv6vPs/d59SKsaQcD4J)

Although in 2017 I'm not sure what I'd even do with them.  At any rate we have what we have, and it really is substantial.  I'm just strungling a little with reproducibility and trying to mash more 1.0 into 0.3 ..

Actually I haven't even looked at the Rhapsody stuff for -posix / -lposix... I've always really thought it really was an overlay link library for SYSV terminal handling...   Now I have the urge to boot up 3.3 and tear a little into -posix .. They at least did put the source to the compiler & ctools on the 3.3 dev cd.

Quote from: "neozeed"Actually I haven't even looked at the Rhapsody stuff for -posix / -lposix... I've always really thought it really was an overlay link library for SYSV terminal handling...   Now I have the urge to boot up 3.3 and tear a little into -posix .. They at least did put the source to the compiler & ctools on the 3.3 dev cd.

Yeah, in 3.3 it was an add-on library that used a different csu to set-up the kernel.  Looking at the historical posts about it it really seems like it was an afterthought for NeXT and they thought it would eventually go away.  So I think they removed it from OPENSTEP because it was somewhat broken and they did not believe in it.  I have not checked for anything POSIX related in Rhapsody, Darwin and OS X, but I assume that by virtue of being based off BSD 4.4 they inherently had some POSIX compliance.  This would perhaps not even need a -posix flag since in NeXT days the flag activated a cludge of stuff in the headers and then used a different csu and added the libposix.a library...

This is an interesting quick read on BSD for those of us who don't know or remember the whole history: http://fhlug.at/wp/wp-content/uploads/2014/04/BSD-Slides.pdf

As far as the NS source is concerned we tried many times with Apple and never had any response.  Big shame really...

Quote from: "t-rexky"
As far as the NS source is concerned we tried many times with Apple and never had any response.  Big shame really...

Actually what I'm curious about is why all evidence of Darwin 0.x has been obliterated, and all of the mailing list posts before October 2001 were deleted.

If I had to bet, back in the day NeXT paid a lot of $$ to AT&T as a late company to the party to get a source license to 32v (and get enough rights to get CSRG work + CMU access + poach CMU staff) that maybe they had though about dumping the BSD aspect for SYSV like all the other 'business' companies out there (SGI/SUN/IBM) but couldn't just justify it.

But that is pure speculation on me.

I'm sure it's needed for something key to building NS, or the proverbial big customer otherwise it'd get dumped.

I think you are probably on to something here.

When you have a moment look at early OS X (10.2 maybe) cctools package source.  I think it was in the Makefiles where there was a comment that it cannot be built for NEXTSTEP or OPENSTEP because of missing proprietary BSD bits.  Or something along these lines.

Of course I conveniently circumvented this by borrowing the missing bits from NS and OS, as required, in my NS3.3 port of cctools.

(EDIT) here it is from https://opensource.apple.com/source/cctools/cctools-435.4/Makefile.auto.html:

# Note: For Darwin developers only building for current MacOS X release is
# supported.  The Openstep target will NOT build outside of Apple as it requires
# 4.3bsd licenced code.


Quote from: "t-rexky"I think you are probably on to something here.

When you have a moment look at early OS X (10.2 maybe) cctools package source.  I think it was in the Makefiles where there was a comment that it cannot be built for NEXTSTEP or OPENSTEP because of missing proprietary BSD bits.  Or something along these lines.

Of course I conveniently circumvented this by borrowing the missing bits from NS and OS, as required, in my NS3.3 port of cctools.

(EDIT) here it is from https://opensource.apple.com/source/cctools/cctools-435.4/Makefile.auto.html:

Code Select Expand

# Note: For Darwin developers only building for current MacOS X release is
# supported.  The Openstep target will NOT build outside of Apple as it requires
# 4.3bsd licenced code.


After 32v was made free per Caldera ( http://www.lemis.com/grog/UNIX/ ) was there hidden secret 4.3BSD?  Or were they just scared and not desiring to possibly embroiled in any SCO suits?

Naturally moving to "cleaner" OSF/mk NetBSD/FreeBSD would sure make sense...  

Also the HFS driver source has a lot of OS 8.1 looking stuff, + a MacOS call emulator.  I've built the AppleTalk stuff, I haven't tried to use it though.

so I just noticed in Darwin that in 0.3 HFS is disabled on the i386 kernel.  Turning it on gives me a panic with

v_usecount

I hammered the 1.0 HFS in there as it does allow mounting root and other fun stuff, and it panics with:

lockmgr: locking against myself

..... non progress..progress...

I'm still around but I'm not checking in on a daily basis. We're nearing a major product release at work and free time and my willingness to work on anything technological in my downtime isn't exactly strong.

For cctools, the underlying problem is GCC has a bootstrapping problem; you need bits of the system its targetting to bootstrap it at all. Generally speaking, that's libc (or libSystem in NeXT/OSX), the CRT (usually called Csu (C Startup Unit), and kernel headers.

Based on what I saw in Mach, the pre-Darwin source almost certainly had BSD 4.4 stuff that was replaced rather rapidly by Apple where possible. Compare kernel headers from OPENSTEP -> Darwin to see what I mean. Based off my memory, the HFS driver port basically came lifted right from classic Mac OS.

32v wasn't free at the time. Originally, the antique UNIX/BSDs were only available under a non-commerical license. We're talking 1999/2000 timeframe here.

You're getting texinfo failures because you changed the modification dates (CVS does not perserve these on import as per my memory). Autotools will NOT rebuild tex files unless the source file is newer than the older. Then you open the path to madness.

EDIT: Darwin 0.x still had BSD code in it. See mach's FFS support from BSD. It would also explain why UFS broke if they were in the process of replacing it from the legacy BSD 4.x version with code from FreeBSD. That's probably the reason it was pulled is that no one realized that was still there since neither NeXT or Mac OS X officially supported FFS as far as I know.

Quote from: "NCommander"I'm still around but I'm not checking in on a daily basis. We're nearing a major product release at work and free time and my willingness to work on anything technological in my downtime isn't exactly strong.

For cctools, the underlying problem is GCC has a bootstrapping problem; you need bits of the system its targetting to bootstrap it at all. Generally speaking, that's libc (or libSystem in NeXT/OSX), the CRT (usually called Csu (C Startup Unit), and kernel headers.

Based on what I saw in Mach, the pre-Darwin source almost certainly had BSD 4.4 stuff that was replaced rather rapidly by Apple where possible. Compare kernel headers from OPENSTEP -> Darwin to see what I mean. Based off my memory, the HFS driver port basically came lifted right from classic Mac OS.

32v wasn't free at the time. Originally, the antique UNIX/BSDs were only available under a non-commerical license. We're talking 1999/2000 timeframe here.

You're getting texinfo failures because you changed the modification dates (CVS does not perserve these on import as per my memory). Autotools will NOT rebuild tex files unless the source file is newer than the older. Then you open the path to madness.

EDIT: Darwin 0.x still had BSD code in it. See mach's FFS support from BSD. It would also explain why UFS broke if they were in the process of replacing it from the legacy BSD 4.x version with code from FreeBSD. That's probably the reason it was pulled is that no one realized that was still there since neither NeXT or Mac OS X officially supported FFS as far as I know.

That makes more sense, I didn't check date time modifications.  I grabbed the nextified textools from 10.3 and other than some .Po files the makefile tries to include, it'll build and I used that to sidestep that issue.

I now need a gperf, but apparently there is no libstdc++ ..  I'll try just retouching everything to the old.  

Speaking of dates, you are right again in 99 32v was still proprietary.  I somehow got on the TUHS list in 97 or so, and bought a SYSIII 'hobbiest' license so I had access.  I guess I'm one of the few lucky personal UNIX ones out there....

What worries me about HFS is that it compiles fine, but the kernel panics with no HFS volumes attached.  I made some changes to get Darwin 1.0 HFS into the kernel and yeah, it crashes too.  I also did FFS, and ... Same thing.

I'll have to break down and see if there is i386 binaries in the Darwin 1.0 dist.  As it too is chicken and egg...  At least Darwin 1.4 has an x86 CD.  It looks like the original Darwin x86 that was 1.2 or so is lost to time save a screenshot.  They were cross compiling from a PowerPC, and using Virtual PC.

I guess the sensible thing is to try to find a level where HFS works on x86 and backport the entire VFS.

I tried disabling stuff like NFS, but it looks like it has hooks for it all over....

I almost wonder if the entire bsd directory could be wholesale moved over........

Now that I think about it I need to check the call table, as 1.0 certainly added a copyfile API call.

I have a contract with a vendor and I have to use supplied equipment.  It crashes hard yesterday.  Nothing like unexpected free time

The earliest x86 Darwin image that I have is darwinx86-141.iso.  I have DarwinOS-0.3.toast, DarwinOS-1.0.toast and DarwinOS-1.0.2.dmg, but those are most certainly PPC binaries...

Quote from: "t-rexky"The earliest x86 Darwin image that I have is darwinx86-141.iso.  I have DarwinOS-0.3.toast, DarwinOS-1.0.toast and DarwinOS-1.0.2.dmg, but those are most certainly PPC binaries...

I have the 141 from Apple's site (its still there but the download page is missing....)  I don't have the 1.0.2 dmg though.. that my have something interesting !?

On HFS segfaulting, do we have the GDB stub in this version of mach to attach a debugger? I believe it requires a kernel command line option to be enabled, but you can then attach gdb over a serial line.

It might want a userland component and that's why its making a sad.

I'll have to learn and bumble through trying to setup a kernel debugger.  I've never done it before, so I'm sure it'll be fun.

I suspect I'll have to debug from Rhapsody as it has a working gdb.

At least Virtual machines are cheap!!

I tried this from Rhapsody's yellowbox on NT

C:\temp\cc-1.obj\x>sh /Apple/proj/cc-1/cc/configure --host=i386-nextpdo-winnt3.5 --target=i386-next-nextstep3 --build=i386-nextpdo-winnt3.5 --srcdir=/Apple/proj/cc-1/cc
Using `/Apple/proj/cc-1/cc/config/i386/next.c' to output insns.
Using `/Apple/proj/cc-1/cc/config/i386/i386.md' as machine description file.
Using `/Apple/proj/cc-1/cc/config/i386/next.h' as target machine macro file.
Using `/Apple/proj/cc-1/cc/config/i386/xm-winnt.h' as host machine macro file.
Merged winnt/x-winnt.
Merged next/t-next.
Merged c++ objective-c++ fragment(s).
Created `./Makefile'.
Merged winnt/x-winnt.
Merged next/t-next.
Created `cp/Makefile'.
Merged winnt/x-winnt.
Merged next/t-next.
Created `obcp/Makefile'.
Links are now set up to build a cross-compiler for i386-next-nextstep3
 from i386-nextpdo-winnt3.5.

sure looks good, right?

/Apple/proj/cc-1/cc/config/next/nextstep.h(928): warning: #warning ****** Disabled objcunique *******
/Apple/proj/cc-1/cc/c-decl.c: In function `grokdeclarator':
/Apple/proj/cc-1/cc/c-decl.c:4771: `stdcallp' undeclared (first use this function)
/Apple/proj/cc-1/cc/c-decl.c:4771: (Each undeclared identifier is reported only once
/Apple/proj/cc-1/cc/c-decl.c:4771: for each function it appears in.)
/Apple/proj/cc-1/cc/c-decl.c:4839: `RID_STDCALL' undeclared (first use this function)
make: *** [c-decl.o] Error 1

So weird.  Guess this stuff wasn't meant to cross compile this way.  Or it just needs more hammering.

I was thinking as this is possible:

C:\temp\cc-1.obj\y>type hi.c
int main(){
printf("hi from Darwin0.3 cc-1\n");
return 0;}

C:\temp\cc-1.obj\y>xgcc -v  -S hi.c
gcc version 2.7.2.1 for Apple PDO
cpp-precomp -smart -lang-c -v -undef -D__GNUC__=2 -D__GNUC_MINOR__=7 -Di386 -DWIN32 -D_WIN32 -Dwinnt -DWINNT -D_M_IX86=300 -D_X86_=1 -D__STDC__=0 -DALMOST_STDC -DNeXT_PDO -D_MT -D_DLL -D__LITTLE_ENDIAN__ -D__ARCHITECTURE__="i386" -D__i386__ -D__WIN32__ -D_WIN32 -D__winnt__ -D__WINNT__ -D_M_IX86=300 -D_X86_=1 -D__STDC__=0 -D__ALMOST_STDC__ -D__NeXT_PDO__ -D_MT -D_DLL -D__LITTLE_ENDIAN__ -D__ARCHITECTURE__="i386" -D__i386 -D__WIN32 -D__winnt -D__WINNT -D__ALMOST_STDC -D__NeXT_PDO -Asystem(winnt) -Acpu(i386) -Amachine(i386) hi.c C:\Users\jason\AppData\Local\Temp\cca00672.i
GNU CPP version 2.6.1 (i386-applepdo-winnt3.51)
#include "..." search starts here:
#include <...> search starts here:
./Library/Frameworks/System.framework/PrivateHeaders
./Library/Frameworks/System.framework/Headers
./Developer/Headers
./Local/Developer/Headers
End of search list.
cc1obj C:\Users\jason\AppData\Local\Temp\cca00672.i -quiet -dumpbase hi.c -version -o hi.s
GNU Obj-C version 2.7.2.1 (i386-applepdo-winnt3.51) compiled by GNU C version 2.7.2.1.


C:\temp\cc-1.obj\y>type hi.s
       .file   "hi.c"
gcc2_compiled.:
___gnu_compiled_c:
       .const
LC0:
       .ascii "hi from Darwin0.3 cc-1\12\0"
.text
       .align 4
.globl _main
_main:
       pushl %ebp
       movl %esp,%ebp
       call ___main
       pushl $LC0
       call _printf
       addl $4,%esp
       xorl %eax,%eax
       jmp L1
       .align 2,0x90
L1:
       leave
       ret

C:\temp\cc-1.obj\y>sh /Apple/proj/cc-1/cc/configure --host=i386-nextpdo-winnt3.5 --target=i386-nextpdo-winnt3.5 --build=i386-nextpdo-winnt3.5 --srcdir=/Apple/proj/cc-1/cc

So it ought to be possible to use OPENSTEP to cross compile, meaning we could hash stuff around so building using NT (aka Windows 10) and using all CPU cores so we can greatly speed stuff up...

This truly is awesome work guys - and as someone who's "had a go" at modernising OpenStep (on black hardware) Id really love a modern bsd 4.4 compatible kernel.

That being said, I don't want to take away from your great work, but would attacking this problem from the other side be more practical use of time and obvious smarts.

I understand the juice is in cracking the problem, but there's an equal opportunity here too. Take the NetBSD kernel and userland and try port the necessary layer required to run NS/OS apps? With the ready available source and dev tools, how easy to open a window from the kernel side?

Quote from: "rooprob"This truly is awesome work guys - and as someone who's "had a go" at modernising OpenStep (on black hardware) Id really love a modern bsd 4.4 compatible kernel.

That being said, I don't want to take away from your great work, but would attacking this problem from the other side be more practical use of time and obvious smarts.

I understand the juice is in cracking the problem, but there's an equal opportunity here too. Take the NetBSD kernel and userland and try port the necessary layer required to run NS/OS apps? With the ready available source and dev tools, how easy to open a window from the kernel side?

To make NetBSD more like NS you'll need to have it loading frameworks and the like... and of course talking to them in Objective C linking...

for now I want to merge in some better filesystem, then try to tackle the missing syscalls for NeXTSTEP to Rhapsody.... that is the other fun thing, they changed a lot of weird stuff... but if I can reliably get NS stuff running it ought to be in a better place.

fleshing out i386 ought to be easier, then looking at something like 68k.  Although the other thing I'm worried about is looking back at the 0.8 release they were all in on the 68030 + 68881 setup.  And there is a very high chance that they emitted a LOT of instructions the 68040 omitted.  I don't know how NetBSD handles it when someone builds something with -m68020 -m68030 -m68881 -mno-soft-float  and then runs it on a 68040.

It's my suspicion that we'll actually need a (partial) 68881 emulator..

Pure speculation of course, but outside of that, I also want to see if I can do drivers in the 'bsd' kernel and dump the driverkit.

It's looking like I'll have my life back come end of June and can pitch in on this.

GCC cross-compiler: ... um, theorically possible, but good look. The ObjC frontend has all sorts of pain and suffering that makes that impractical at best. Apple's fat binary solution is a nifty hack around the problem. Based on what I've seen, NeXT essentially never did "true" cross-compiling, but just fat binary *everything*, and the PBMakefiles suggest they had a NT on NeXT target which was probably how they bootstrapped the thing.

If you compile mach with a debug option and/or boot switch, it should dump a stack track when it goes belly up. You can use gdb on mach_kernel or just check with objtools to determine where it code base it went boom.

EDIT: Re-reading through the backlog, libsystem is dynamically built from several other libraries at once. The exact bits of magic escape me, but I know where to recover them. Dylib files are (fortunately) position independent, and I belive they're also modular, which means they can be disassembled, modified, and reassembled relatively easily. The libsystem buildscript is a fairly decent example of this.

As a potential "option B" however, there might be a cleaner way to handle this. The dynamic link loader is in mach (with another part I think in libc), and we've got some handy fields on the mach-o. We can forcibly include different libraries in the search path, or if we're got a Rhaspody binary, load the Rhaspody sysroot. Got OPENSTEP? Get Openstep, etc.

I've already proven we can run NeXT, OPEN, Rhaspody, and Darwin 0.x binaries all on the same kernel, and you proved from a DisplayServer bit that it should all work.

It's gotten awfully quiet here lately.  I am suffering from withdrawal symptoms.  Hands shaking, body tremors, dreams of magnesium cubes...

Quote from: "t-rexky"It's gotten awfully quiet here lately.  I am suffering from withdrawal symptoms.  Hands shaking, body tremors, dreams of magnesium cubes...

works been kicking my ass.  I did make a slight discovery, the NS CD's actually have a disklabel on them.  I think the filesystem is almost 4.3BSD but with weird sizes.  I think it's a matter of driving mkfs directly to create a filesystem, although you can't just mount one read write, I guess there is a flag to prevent that?  I haven't dived deep enough (time).

I also put up cvs.nextstep33.info with my latest buildable stuff, although clock drift is going to mess stuff up for make.  /debs has binaries though which is an easy way to pull down a release.

I'll try to put together some steps, and a hacked up 'disk' command to chain to the new mkfs/boot blocks instead of the system stuff so it ought to be semi-workable.  Although I don't think Rhapsody can read an 8GB UFS disk on it's own.

Also I see that in NS 3.1 for the x86 there is a fsck diskette, aka a tiny root filesystem that fits onto a floppy.

Poking my head back in. I have very little free time, and very little motivation to work on tech projects on it, but I'll weigh in a bit here.

The UFS format used on the CDs is slightly different than what's used on the HDDs at a 2k block size vs 512; this is noted somewhere in mach's source, and I think I found it originally in the Linux UFS kernel source which has at least some support for NeXTstep volume formats.

From a NeXT perspective, raw disk images exist as the "h" partition of a device, so if you want to dump the entire disk, you can dd if=/dev/srXh or something like that and get the whoel thing

Hello: When I use DD to micro sd cards this works very well
file.iso = your disk image !
you will want to unmount the sd disk as well before using dd
diskutil unmountDisk /dev/disk1

sudo dd if=/dev/rdisk1 of=/file.iso bs=512b then to copy the file back to the disk sudo dd if=/file.iso of=/dev/rdisk1    using /dev/rdisk1 instead of just /dev/disk save significantly on time , also works for making iso files of NeXT CD's

Also the slower 10mb/s sd cards used to take hours to make clone dd , go with extreme pro a 95mb/s 16Gb microsd card takes 5 minutes or less!  What is cool about this is if I hose something up on black or intel  hardware I can just pop in a backup sd and go and repair the hosed sd.

So it may help you all here trying different incarnations of this project ....

Im not dead, work is trying to consume my life, and when I get home, the last thing Ive been wanting to do is touch a compiler.  I've been treking across the wastelands and blowing stuff up.

I'll get in the mood again though.

Quote from: "neozeed"Im not dead, work is trying to consume my life, and when I get home, the last thing Ive been wanting to do is touch a compiler.  I've been treking across the wastelands and blowing stuff up.

I'll get in the mood again though.

I'm unfortunately in the same boat. Being the one full-stack developer at a startup means I lack time.

Looking back through the thread, NetBSD and Linux can both do software emulation of missing opcodes so it's theorically possible to run 68040 and later on newer. That being said, I honestly can't remember off hand how much new was added to the later m68ks from a userland perspective. I think a lot of it came from extensions of the address bus.

Ok, I know all the cool kids use git or whatever, but since Rhapsody has CVS built in, let's use that to pull stuff down.

the date & timestamps will be munged and thusly the build process will want to do further build stuff, and it wont work as there is some missing tools (gprof) which is bombing as I haven't self hosted the c++ stuff just yet.



niutil -create . /machines/cvs.nextstep33.info
niutil -createprop . /machines/cvs.nextstep33.info ip_address 14.136.113.162
cvs -d:pserver:anoncvs@cvs.nextstep33.info:/cvsroot login


There is no passowrd.  And I just created a record for my machine in case you don't have DNS or whatever.

I have 2 things to snag, the source:
cvs -d:pserver:anoncvs@cvs.nextstep33.info:/cvsroot checkout darwin

And the pre-compiled binaries, aka the debs
cvs -d:pserver:anoncvs@cvs.nextstep33.info:/cvsroot checkout debs

The source is 388MB, and the debs are 36MB.

Vintage apple pages....

https://archive.li/uFEzI

https://archive.li/S8eQn

Are you reigniting your enthusiasm for additional tweaking of the Darwin sources?

:wink:

I wish I could be more active but my only free time nowadays is consumed by redesigning my home network including the rack, wiring, firewall, UTM, NAS, etc.  I am drowning...

I have been stepping back through the process...

I need to document it more, it really is a mountain of "special" to be standing on...

That and pull in a whole lot of the rest of Darwin to get a better footing, along with leaning a lot more on Rhapsody......

It's been a year since I posted anything significant, and I'm dying to change that a little......

I also want to check out Visual Studio 2017 and it's "cross Linux" tools and see if it'll just ssh to any other OpenSSL host...

Also I kinda worked around the weird installer thing and got Darwin 0.3 to install!

You need to install OS 9, I used the G4 Cube version, onto a disk.  Then get the Darwin 1.0 CD image, and copy the disk imaging tool to the hard disk.  Then using that tool, restore the image on the Darwin 0.3 CD onto a empty disk.

And then I used Steve Troughton Smith's BootX loader https://github.com/steventroughtonsmith/BootX


qemu-system-ppc.exe -L pc-bios -drive file=..\darwin03.qcow2,index=0,format=qcow2,media=disk -drive file=BootX_custom.dmg,index=2,format=raw,media=disk -prom-env "boot-device=ide1:2,\BootX" -prom-env "boot-args=-v rd=hd0 debug=0xffe kdp=2" -prom-env "boot-file=ide0:8,\mach_kernel" -M g3beige

Yes it does take a while to boot up, but yay now I can test feature parity between x86 and PowerPC

Oh yeah download link:

https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin03-PowerPC_qemu-2.11_04_22_2018.7z/download

Also here is Darwin 1.0

https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin1.00PowerPC_qemu-2.11_04_22_2018.7z/download

At least on Qemu Darwin 1.0 is significantly faster than 0.3 ..

Q. But I heard Darwin OS runs on Intel PCs...

Most of Darwin code is platform-independant, and some of the platform-dependant code includes some support for Intel processor-based systems. There are, however, some parts of the system (eg. HFS+ and AppleTalk) which have not been ported to the Intel platform. The current Darwin release is compiled for PowerPC only, but it should be possible to bring up an Intel version with some additional work.

http://web.archive.org/web/19990821004420/http://publicsource.apple.com:80/projects/darwin/faq.html

Well that certainly explains a lot.

And the original 0.1 certainly was just some source files:

Q. What do I need to compile this code?
Presently, you need to buy and install a copy of Mac OS X Server with the developer tools. We are actively working on putting together a self-hosting binary distribution of Darwin so that you can install that as your development platform. This will include all of the needed development tools.

http://web.archive.org/web/19990508075706/http://www.publicsource.apple.com:80/projects/darwin/faq.html

Something I should have done years ago, but I guess no time like the present...


https://opensource.apple.com//tarballs/

mirror this directory.

# du -hs opensource.apple.com
26G     opensource.apple.com


Ever since the 'great reset' of Darwin with the release of 10.0 it really isn't that much to be honest..

https://opensource.apple.com/release/mac-os-x-100.html

I pulled all the tarballs for 10.0 through 10.5 back in the days when I was fiddling with cctools.  I have them stored locally on my Synology NAS.

Quote from: "t-rexky"I pulled all the tarballs for 10.0 through 10.5 back in the days when I was fiddling with cctools.  I have them stored locally on my Synology NAS.

I wonder if anything went missing during this time..?

Quote from: "neozeed"I wonder if anything went missing during this time..?

I will make a file list when I have a chance...

Ok, here is the source package listings for 10.0 through 10.3:

https://www.dropbox.com/s/612bfsv2ke1xrhd/osx-source-list.zip?dl=0

Excellent work!

I'm not sure how you guys managed to compile the kernel without it panicking! I've applied every patch on the forums and on neozeed's site, it compiles fine. I just can't get it to boot.

The freshly compiled kernel will boot to single user, but will panic as soon as you exit to multiuser after rc starts /etc/startup/0300_Devices (which uses driverLoader to load the Active Drivers in the Instance table)

Any hints on what I could be missing? FWIW, I'm using the darwin-0.3 dpkg-based compile scripts, shoehorning in dpkg/config files to all the source packages. They're pretty complete on the build dependencies, only a few packages fail to build.

EDIT: to clarify, I'm trying to build Darwin 0.1 sources (kernel 5.3)

Quote from: "evolver56k"Excellent work!

I'm not sure how you guys managed to compile the kernel without it panicking! I've applied every patch on the forums and on neozeed's site, it compiles fine. I just can't get it to boot.

The freshly compiled kernel will boot to single user, but will panic as soon as you exit to multiuser after rc starts /etc/startup/0300_Devices (which uses driverLoader to load the Active Drivers in the Instance table)

Any hints on what I could be missing? FWIW, I'm using the darwin-0.3 dpkg-based compile scripts, shoehorning in dpkg/config files to all the source packages. They're pretty complete on the build dependencies, only a few packages fail to build.

EDIT: to clarify, I'm trying to build Darwin 0.1 sources (kernel 5.3)

Sorry I've been locked out for a few months.

Can you show the steps you have done, and where it bombed?

Eventually figured it out, it was the patch to modify the PCI bus lookup in IOPCIDirectDevice.m in driverkit. I'm now running my own build of 5.3, and (ancient) openssh/openssl:

RhapsodyVM01:Administrator root# uname -a ; ssh -V
Rhapsody RhapsodyVM01 5.3 Kernel Release 5.3: Thu Jan 31 13:53:26 EST 2019; root(rcbuilder):kernel/BUILD/RELEASE_I386 Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.  i386
OpenSSH_3.8p1, SSH protocols 1.5/2.0, OpenSSL 0.9.7i 14 Oct 2005
RhapsodyVM01:Administrator root#

Quote from: "evolver56k"Eventually figured it out, it was the patch to modify the PCI bus lookup in IOPCIDirectDevice.m in driverkit. I'm now running my own build of 5.3, and (ancient) openssh/openssl:

Code Select Expand

RhapsodyVM01:Administrator root# uname -a ; ssh -V
Rhapsody RhapsodyVM01 5.3 Kernel Release 5.3: Thu Jan 31 13:53:26 EST 2019; root(rcbuilder):kernel/BUILD/RELEASE_I386 Copyright (c) 1988-1995,1997-1999 Apple Computer, Inc. All Rights Reserved.  i386
OpenSSH_3.8p1, SSH protocols 1.5/2.0, OpenSSL 0.9.7i 14 Oct 2005
RhapsodyVM01:Administrator root#

Good job!

It's a bit of a rough ride to 5.3, but once you get it running it's great.

I should have kept better notes on the whole thing.

I had partial notes on using CVS to download and bootstrap from DR2, although I keep on getting distracted.  :cry:

So in the exciting world of Mach, I was given the disk images to the Mt. Xinu 386 Mach, and I compiled the source to Mach 2.5 both MK83 and some later version.  The catch is that the setup to protected mode & paging is wrong and crashes the processor.  And then I wanted to go back to Darwin to check out how some stuff was done there, and I totally forgot how to use the Debian build process.


This is more so for me, although anyone following on from the future will probably want to know.  Sorry I thought I kept good notes, but apparently I do not. :(

Using the Qemu image on the sourceforge page Darwin03_qemu090_26_5_2017.7z (https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin03_qemu090_26_5_2017.7z/download) (because the networking works!... I forget how to put the modules into place and have them activate to fix the newer one. I think it's all in this thread somewhere, but it turns out that it's easier to update this image).

Mount the CD-ROM Darwin03_debs_source_31_5_2017.iso (https://sourceforge.net/projects/aapl-darwin/files/Revived/Darwin03_debs_source_31_5_2017.iso/download)


mount_cd9660 /dev/sd0a /mnt
cd /tmp
ar x /mnt/debs/buildtools_0.1-2_i386-apple-rhapsody.deb
cd /
tar -zxvf /tmp/data.tar.gz
cd /tmp/
tar -zxvf /mnt/source/dpkg_scriptlib-1.tar.gz
cp dpkg_scriptlib-1/perl5/Dpkg/Package/*.pm /usr/lib/perl5/Dpkg/Package/

I'm not sure if dpkg_1.4.1.0.2-3_i386-apple-rhapsody.deb is needed to be extracted as well.  It's late and I don't want to re-build my VM again.

This should be enough to get the build packages working.  However out of the gate, it's going to try to build for the PowerPC.  And this is an Intel world.  What we are going to do here is shatter the fairy tale of 'one apple employee working from home kept the OS X on Intel dream alive' nonsense by showing that OS X was always buildable on Intel.

We edit the file /usr/lib/perl5/Dpkg/Package/Builder.pm and change the PowerPC crap to i386.

bash-2.02# diff -ruN Builder.pm Builder.pm_i386
--- Builder.pm  Wed Aug  4 17:37:35 1999
+++ Builder.pm_i386    Thu May 25 23:47:16 2017
@@ -60,7 +60,7 @@
  'DSTROOT' => undef,
  'SUBLIBROOTS' => undef,
  'RC_JASPER' => 'YES',
- 'RC_ARCHS' => 'ppc',
+ 'RC_ARCHS' => 'i386',
  'RC_CFLAGS' => '',
  'RC_hppa' => '',
  'RC_i386' => '',
@@ -122,15 +122,15 @@
  }

  if ($target eq 'installhdrs') {
-    $flags->{'RC_CFLAGS'} = '-arch i386 -arch ppc ' . &liststring (@cflags);
-    $flags->{'RC_ARCHS'} = 'i386 ppc';
+    $flags->{'RC_CFLAGS'} = '-arch i386 ' . &liststring (@cflags);
+    $flags->{'RC_ARCHS'} = 'i386';
    $flags->{'RC_i386'} = 'YES';
-    $flags->{'RC_ppc'} = 'YES';
+    $flags->{'RC_ppc'} = '';
  } else {
-    $flags->{'RC_CFLAGS'} = '-arch ppc ' . &liststring (@cflags);
-    $flags->{'RC_ARCHS'} = 'ppc';
-    $flags->{'RC_i386'} = '';
-    $flags->{'RC_ppc'} = 'YES';
+    $flags->{'RC_CFLAGS'} = '-arch i386 ' . &liststring (@cflags);
+    $flags->{'RC_ARCHS'} = 'i386';
+    $flags->{'RC_i386'} = 'YES';
+    $flags->{'RC_ppc'} = '';
  }

  return $flags;
@@ -292,7 +292,7 @@

  $package->{'package'} = $pname;
  $package->{'version'} = "0";
-  $package->{'architecture'} = 'powerpc-apple-darwin';
+  $package->{'architecture'} = 'i386-apple-darwin';
  $package->{'source'} = $package->{'package'};
  $package->{'description'} = "No description available.";
  $package->{'maintainer'} = "Anonymous <darwin-development\@public.lists.apple.com>";
@@ -335,7 +335,7 @@
      $package->{'maintainer'} = "Anonymous <darwin-development\@public.lists.apple.com>";
  }

-  $package->{'architecture'} = 'powerpc-apple-darwin';
+  $package->{'architecture'} = 'i386-apple-darwin';
  $package->{'source'} = $package->{'package'};

  return $package;

Now that we have purged "powerpc-apple-darwin" from the build program, we can get started building stuff.

This simple script will build through all the source on the CD-ROM, and will compile stuff to the /compiled directory.  It tries to keep things 'neat' although it will hammer the hell out of your disk... Not that it matters on modern setups.


cat /build_all.sh
#!/bin/bash

# This is a simple script to compile all of the sources on the CD-ROM
#
# This will use the existing debs on the CD-ROM.  The build script
# from the debian folks will let new debs take precident over the old ones.
# This is basically to prove the build system works

#is the CD mounted? it should be on /mnt
# The CD I'm using is called:
# Darwin03_debs_source_31_5_2017.iso
mount_cd9660 /dev/sd0a /mnt


# Make directories to store the source and the compiled debs
rm -rf /sources
rm -rf /tmp/roots
mkdir /sources
mkdir /compiled
# for each of the source files on the CD
for filename in /mnt/source/*gz; do
cd /sources
# unpack the source
tar -zxf $filename
  for dirname in /sources/* ; do
# run the buildpackage script
      echo "darwin-buildpackage --dir --target all $dirname /mnt/debs /compiled"
      darwin-buildpackage --dir --target all $dirname /mnt/debs /compiled
      rm -rf $dirname
  done
rm -rf /tmp/roots
done

The magic comes from the darwin-buildpackage script.

Anyways Mach back in MK83 days was expecting either COFF or A.OUT not O-MACH.  I don't know if it even really existed yet.  Seems yet another dead end. :(

You can create a 'manifest file' from the extracted tarballs from something like this where you extracted them (if using a bourne shell):
for i in $(ls -1) ; do echo "dir $i all" >> buildmanifest.txt
and using:
darwin-buildall buildmanifest.txt /path/to/debrepo /path/to/debdestination
Will build all the packages it can, and automatically deleting the temp files under /tmp/roots/ on a successful build.
The third field of the manifest is the same as the --target flag for darwin-buildpackage, so it can be any of all|headers|objs|local

There's a Manifest file on the Darwin 0.3 disc (Darwin OS Install CD/Binary Packages/Manifest ... it has mac line endings, so you'll have to change those to unix line endings first before using it with buildall, and change ^cvs to dir) that I believe shows how they built the deb packages for the release

Quote from: evolver56k on July 02, 2019, 01:42:42 PMYou can create a 'manifest file' from the extracted tarballs from something like this where you extracted them (if using a bourne shell):

Code Select Expand

for i in $(ls -1) ; do echo "dir $i all" >> buildmanifest.txt
and using:

Code Select Expand

darwin-buildall buildmanifest.txt /path/to/debrepo /path/to/debdestination
Will build all the packages it can, and automatically deleting the temp files under /tmp/roots/ on a successful build.
The third field of the manifest is the same as the --target flag for darwin-buildpackage, so it can be any of all|headers|objs|local

There's a Manifest file on the Darwin 0.3 disc (Darwin OS Install CD/Binary Packages/Manifest ... it has mac line endings, so you'll have to change those to unix line endings first before using it with buildall, and change ^cvs to dir) that I believe shows how they built the deb packages for the release

This is what I needed to know.

Sigh, pitty the forum has been down for so long.

I need to do a full build.

Yeash, this isn't dead. I actually have all my test VMs from when we did this and I think I have most of my notes as well. I'm not super active here and honestly RL took a serious dump on me about the time I dropped off but I'm around to offer big brain support. a PM works best but I'll set an alert on this thread for the future.

I have had a github repo set up for this for a while now, but recently made it public. I've gotten the entire Darwin 0.3 sources to build fat binaries on both intel and ppc installs, and have a binary tarball from an early build run as a release, which can be used for the binary repo during building.

I've also added the NE2K and VMXNet drivers to the package manifest, so they'll get built (i386 only) during the darwin-buildall run.

https://github.com/evolver56k/Darwin-0.3

The way I use the github repo, is to have it checked out on my modern machine, and have an NFS export of the local repo mounted on the darwin machine, either i386 via VMware, or a B&W G3 running ppc native.

With the repo on an NFS share, I can use modern editors to muck with the packages, and it's instantly reflected on the darwin build box to attempt another build.

My goal is to get other modern packages into the repo like openssh, openssl, etc., all being built with the darwinbuild system natively.

Quote from: evolver56k on February 04, 2021, 11:18:43 AMI have had a github repo set up for this for a while now, but recently made it public. I've gotten the entire Darwin 0.3 sources to build fat binaries on both intel and ppc installs, and have a binary tarball from an early build run as a release, which can be used for the binary repo during building.

awesome job!

Hi guys,

Sorry to bump an old post, I've been very eager to see if I can work on the USB stack for MOSX Server 1.2v3, and I was very excited to see a git repo for something that's at least somewhat close to the mach_kernel!

Long thread, I'm having a bit of trouble working out where it ended up. Were you guys able to compile a kernel from the Darwin 0.3 source that ran in multi-user and "just worked" (ran the loginwindow, with drivers, networking etc.?)

Cheers, keen to keep this train going in 2022.

It's been a good while since I've toyed with this but yeah, everything should "work". There's a couple of caveats though that I have found in my testing:

- The kernel source is roughly equivalent to the kernel in MOSX 1.0, so don't expect it to work any magic on MOSX 1.2 or 1.2v3.
- If you're running RDR2 on i386 or ppc, or MOSX 1.0 on ppc, you can't use System.framework built from these sources, as they're missing some libraries like java that'll break everything.

If you're running Darwin 0.3 though, these should compile fine on either arch.

Has anyone played around with OpenDarwin? I still have the source tree and an ISO from it... though I never hacked on it much.

From Page 4 Post #54 (https://www.nextcomputers.org/forums/index.php?msg=23644)

Quote from: neozeed on April 20, 2017, 11:52:37 PM=== kernel
cd /usr/src
tar -zxvf /mnt/data/darwin/kernel-1.tar.gz
ln -s /usr/src/kernel/BUILD/RELEASE_I386/features.h /usr/include/machdep/i386/features.h

I'm using kernel-7 that came on the darwin source cd of your site?!? But I can't locate machdep/i386/features.h to be able to build drv-EIDE. Does any one have it?

compile errors are on this file /drvEIDE-1/EIDE.drvproj/EIDE.lksproj/AtapiCnt.m (https://github.com/evolver56k/Darwin-0.3/blob/master/drvEIDE-1/EIDE.drvproj/EIDE.lksproj/AtapiCnt.m)

At this location Line 731 at the end of the file (https://github.com/evolver56k/Darwin-0.3/blob/745305afd074437bb660bcf02146a2754eeef5ae/drvEIDE-1/EIDE.drvproj/EIDE.lksproj/AtapiCnt.m#L731)

- property_IODeviceType:(char *)types length:(unsigned int *)maxLen
{
    strcat( types, " "IOTypeATAPI);
    return( self);
}
Edit: 2023-Nov-23 This above code has been solved and was located in the file driverkit/IOProperties.h which is in the driverkit-3 archive.

Thanks

Edit: /machdep/i386/features.h is the last missing file I require, the other missing files were able to locate in either the tarballs of driverkit-3 or kernel-7 that were on the user @neozeed CD. It seems that @neozeed created a file link so I'm interested where he managed to locate it or how he built the file.

It's been a long while...

[darwin:~] root# find / -name 'features.h' -print
/usr/src/kernel-1/mach/features.h
/usr/src/kernel-1/machdep/machine/features.h
/System/Library/Frameworks/System.framework/Versions/B/Headers/mach/features.h
/System/Library/Frameworks/System.framework/Versions/B/PrivateHeaders/mach/features.h
/System/Library/Frameworks/System.framework/Versions/B/PrivateHeaders/machdep/machine/features.h
/System/Library/Frameworks/System.framework/Versions/B/PrivateHeaders/machdep/ppc/features.h
/System/Library/Frameworks/System.framework/Versions/B/PrivateHeaders/machdep/i386/features.h

not sure why you'd be missing it...

/*
 * Copyright (c) 1999 Apple Computer, Inc. All rights reserved.
 *
 * @APPLE_LICENSE_HEADER_START@
 *
 * "Portions Copyright (c) 1999 Apple Computer, Inc.  All Rights
 * Reserved.  This file contains Original Code and/or Modifications of
 * Original Code as defined in and that are subject to the Apple Public
 * Source License Version 1.0 (the 'License').  You may not use this file
 * except in compliance with the License.  Please obtain a copy of the
 * License at http://www.apple.com/publicsource and read it before using
 * this file.
 *
 * The Original Code and all software distributed under the License are
 * distributed on an 'AS IS' basis, WITHOUT WARRANTY OF ANY KIND, EITHER
 * EXPRESS OR IMPLIED, AND APPLE HEREBY DISCLAIMS ALL SUCH WARRANTIES,
 * INCLUDING WITHOUT LIMITATION, ANY WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT.  Please see the
 * License for the specific language governing rights and limitations
 * under the License."
 *
 * @APPLE_LICENSE_HEADER_END@
 */

/*
 * Mach Operating System
 * Copyright (c) 1987 Carnegie-Mellon University
 * All rights reserved.  The CMU software License Agreement specifies
 * the terms and conditions for use and redistribution.
 *
 */
/*
 *  This file is used to pick up the various configuration options that affect
 *  the use of kernel include files by special user mode system applications.
 *
 *  The entire file (and hence all configuration symbols which it indirectly
 *  defines) is enclosed in the (KERNEL && !KERNEL_BUILD) conditional to
 *  prevent accidental interference with normal user applications.  Only
 *  special system applications need to know the precise layout of
 *  internal kernel structures and they will explicitly set the appropriate
 *  flags to obtain the proper environment.
 */

#ifndef _MACH_FEATURES_H_
#define _MACH_FEATURES_H_

#ifdef  KERNEL

#ifdef  KERNEL_BUILD
#import <meta_features.h>

#else   /* KERNEL_BUILD */
#import <machdep/machine/features.h>

#endif  /* KERNEL_BUILD */

#endif  /* KERNEL */

#endif  /* _MACH_FEATURES_H_ */

The emulator package thing Darwin03_qemu090_26_5_2017.7z (https://sourceforge.net/projects/aapl-darwin/files/qemu-images/Darwin03_qemu090_26_5_2017.7z/download) should be a pre-packaged one I used to build stuff...

I ran across this recently: https://github.com/blacktop/darwin-xnu-build

I was able to build XNU easily from this.

Thanks.

I'll extract ALL the files on the CD, at the moment I only had the ones I needed or were interested in which were boot, driverkit, kernel, drvEIDE.



https://ibb.co/dj8sMYN (https://ibb.co/dj8sMYN)

Ignore the illegal function call: found 'IOTypeATAPI' error. This above code has been solved and was located in the file driverkit/IOProperties.h which is in the driverkit-3 archive.

Seems like @neozeed seemed to have another go at this on his blog  8)

Rebuilding Darwin from source: Part 1 Qemu (https://virtuallyfun.com/2024/04/22/rebuilding-darwin-from-source-part-1-qemu/)
Rebuilding Darwin from source: Part 2 Building the kernel (https://virtuallyfun.com/2024/04/23/rebuilding-darwin-from-source-part-2-building-the-kernel/)
Rebuilding Darwin from source: Part 3 Debian makes the world go 'round (https://virtuallyfun.com/2024/04/24/rebuilding-darwin-from-source-part-3-debian-makes-the-world-go-round/)