Greaseweazle and NeXT floppies

NeXT Computer, Inc. -> NEXTSTEP / OPENSTEP Software

Title: Greaseweazle and NeXT floppies
Post by: pl212 on July 17, 2025, 02:24:16 PM
It seems there are two general used strategies for archiving Next software on floppy disks:

A) Image the disks with rawread on a PC.

B) Use DiskCopyII on an actual Next. This has the disadvantage of inserting a spurious header (https://www.nextcomputers.org/forums/index.php?topic=4581.msg26737#msg26737), although there is a workaround (https://www.nextcomputers.org/forums/index.php?topic=4581.msg26713#msg26713).

Now that I have my Greaseweazle working with both 1.44 and 2.88 drives, I'm wondering if there's any need/interest in doing a lower-level form of preservation?

Creating the actual flux dumps is easy enough, and results in a good "forensic" data dump.  What we don't have is a disk format defined in the Greaseweazle software, which would allow us to convert the flux transitions back into a logical disk image. 

I suppose that since exotic forms of copy protection were not common for Next software, there may not be as much of a reason to go this far.  But it would be nice if we could at least have the option.

1.44 Next disks seem to use 18 tracks.  Here is a visualization of such a disk:



2.88 Next disks seem to use 36 tracks:



An example of the IBM 2.88 format, which does exist in Greaseweazle:

disk 2880
    cyls = 80
    heads = 2
    tracks * ibm.mfm
        secs = 36
        bps = 512
        gap2 = 41
        rate = 1000
    end
end







Title: Re: Greaseweazle and NeXT floppies
Post by: user217 on July 17, 2025, 03:12:51 PM
Great timing—I'm working on a master list of all NeXT software that's available on the internet, so we can coordinate on hunting down what still needs to be archived. (Of course, half of it is mostly useless without license keys, but that's another challenge...)

Unfortunately the images in your post did not get attached properly (EDIT: they're working now, I blame Cloudflare) @pl212 but I'm guessing the format defined for "ibm.mfm" has to be modified to match. Is there any other context to this information? It's not clear what software package you're using to archive the images—I don't think there's one simply called "Greaseweazle," is there?

Anyway, these are the most relevant things I can think of/find:

The datasheet for the floppy controller (https://www.nextcomputers.org/NeXTfiles/Docs/Hardware/Datasheets/Floppy_Controller/82077AA_Floppy_Controller_Datasheet.pdf) describes how to control it, and what it can and cannot do.

The 1990 version of the NeXT ROM (https://github.com/itomato/NeXTROM/blob/master/dev/fd_driver.c.c) has floppy driver code in it, including gap size:


// from fd_extern.h:

/*
 * fd_ioreq.density values. Some routines rely on these being in
 * cardinal order...
 */
#define FD_DENS_NONE 0 /* unformatted */
#define FD_DENS_1 1 /* 1 MByte/disk unformatted */
#define FD_DENS_2 2 /* 2 MByte/disk unformatted */
#define FD_DENS_4 3 /* 4 MByte/disk unformatted */

// ...

/*
 * misc. hardware constants
 */
#define NUM_FD_HEADS 2 /* number of tracks/cylinder */
#define NUM_FD_CYL 80 /* cylinders/disk (is this actually a
* constant for all densities?) */

// from fd_vars.h:

/*
 * fd_volume.sect_size values
 */
#define FD_SECTSIZE_MIN 512
#define FD_SECTSIZE_MAX 1024

// from fd_driver.c:

struct fd_drive_info fd_drive_info[] = {
{
    {
"Sony MPX-111N", /* disk name */
{ 0,
  0,
  0,
  0 }, /* label locations - not used */
1024, /* device sector size. This is
* not fixed. */
64 * 1024 /* max xfer bcount */
    },
    SONY_SEEK_RATE,
    SONY_HEAD_LOAD,
    SONY_HEAD_UNLOAD
}
};


/*
 * Constant tables for mapping media_id and density into physical disk
 * parameters.
 */
struct fd_disk_info fd_disk_info[] = {
   /* media_id          tracks_per_cyl  num_cylinders   max_density */
    { FD_MID_1MB, NUM_FD_HEADS, NUM_FD_CYL, FD_DENS_1    },
    { FD_MID_2MB, NUM_FD_HEADS, NUM_FD_CYL, FD_DENS_2    },
    { FD_MID_4MB, NUM_FD_HEADS, NUM_FD_CYL, FD_DENS_4    },
    { FD_MID_NONE, 0, 0, FD_DENS_NONE },
};

struct fd_density_info fd_density_info[] = {
    /* density      capacity    gap3 mfm         */
    { FD_DENS_1, 720  * 1024, 8, TRUE }, /* 720  KB */
    { FD_DENS_2, 1440 * 1024, 16, TRUE }, /* 1.44 MB */
    { FD_DENS_4, 2880 * 1024, 32, TRUE }, /* 2.88 MB */
    { FD_DENS_NONE, 720  * 1024, 48, TRUE }, /* unformatted */
};

We're somewhat lucky as it appears this was possibly the last thing added?

This document (https://www.hermannseib.com/documents/floppy.pdf) explains the mechanics of floppies and what gap2 and gap3 mean.
Title: Re: Greaseweazle and NeXT floppies
Post by: KennyPowers on July 18, 2025, 04:42:10 PM
I've been able to flux-dump 1.44MB NeXT floppies using a Kryoflux and generate logical disk images from those dumps...just MFM sector dumps, 80 tracks-per-side, 18 sectors-per-track...the same settings I would use for a typical 1.44MB PC or Mac MFM-encoded floppy.  I imagine a Greaseweazle or AppleSauce device could do the same (I don't have a Greaseweazle and my AppleSauce is an older model that doesn't work with PC floppy drives).



I've then been able to use BlueSCSI's floppy drive emulation (https://bluescsi.com/docs/Usage#image-directories) to mount those floppy images on real NeXT hardware (an 040 Cube in this case):




As you can see, the BlueSCSI appears to be emulating a TEAC FD235HS, which I think was just a 1.44MB drive.  I don't own a real 2.88MB drive (yet), nor do I know how/if to connect my Kryoflux to one, but apparently you've been able to get flux dumps from a 2.88MB drive with your Greaseweazle?  If you can somehow generate a simple sector-dump image from one, I'd be interested to see what the BlueSCSI makes of it (and a step-by-step of your process :) ).  As you mention, copy protection probably isn't a concern, so more exotic image formats probably aren't necessary.
Title: Re: Greaseweazle and NeXT floppies
Post by: pl212 on July 18, 2025, 11:29:26 PM
Thanks, this is really helpful!

@KennyPowers -- it was quite a journey to get a Greaseweazle talking to a 2.88mb disk.  There are just a few communities online for whom these disks actually matter -- IBM PS/2, NeXT, and -- believe it or not -- IBM's first ThinkPad, which ran PenPoint.

To begin with, the data rate from a 2.88mb drive exceeds the 12Mbps of Full-Speed USB. This instantly eliminates all Greaseweazles except for the F7 Lightning Plus, an uncommon model which supports the 480Mbps of High-Speed.  I see these are now available from a US-based seller on eBay; at the time, I had to import one from the UK.

Next, our job is to find a working 2.88mb drive. But wait: 90% of these use unified power-over-data cables, including the Sony drives made for both NeXT and IBM.  (The drive inside the SCSI PLI Superfloppy included (https://www.nextcomputers.org/forums/index.php?topic=4705.msg27479#msg27479)). Splitting these into a conventional 4-pin Berg can be done, but pinouts are hard to find online and there are many varying models. I was eventually able to find a Sony MP-F40W-18 with separate power and data, and this is the drive I've used to archive 2.88 disks from IBM successfully. 

However, this 2.88mb drive does not work for archiving 720k images -- so I keep a conventional 1.44mb drive on hand for that purpose. I'm pretty certain this is a wiring issue (https://www.retrotechnology.com/herbs_stuff/drive.html#288) -- apparently there were so few vendors who wanted 2.88mb drives that the standard never quite had time to establish itself, and there were lots of possible settings:

QuoteThere is currently no industry-wide standard for the FDC to FDD interface. There are numerous floppy drive vendors, each with their own modes and interface pins to enable 4 MB perpendicular mode. The drive interface not only varies from manufacturer to manufacturer but also within a manufacturer's product line. (https://www.ardent-tool.com/datasheets/Intel_82077SL_AP358.pdf)

Since the drive works for 2.88mb disks, I'm loathe to experiment...

That's great to know about BlueSCSI's floppy emulation as a way to test images! I suspect that even if it doesn't currently support 2.88mb disks, we could work with Eric to do so in the future.

I will do as you suggest and create a sector-dump from a 2.88mb disk to share...

Obligatory ED pic:




Title: Re: Greaseweazle and NeXT floppies
Post by: user217 on July 19, 2025, 06:24:34 AM
@pl212 If I'm not mistaken that version of Mathematica has never been archived—my collection has 2.2, 3.0.0, and 3.0.2, plus of course the antique 1.2 versions that came with early NeXTstep releases. Here's hoping it's got good data on it!
Title: Re: Greaseweazle and NeXT floppies
Post by: pl212 on July 20, 2025, 12:45:48 AM
So these are images for HSD Spell Disk 1 and 2, converted from the raw magnetic dump.  Images of the flux transitions are included as well. (I'm happy to share the raw files; they are about 77 megabytes per disk!)

I've already verified that one of these mounts under Ubuntu's Nextstep UFS support.  If anyone wants to test mounting under real/emulated Nextstep, BlueSCSI's floppy emulation, or other contexts, that would be great!

(HSD Spell software has already been preserved on at least fsck.technology; I haven't done a deep dive to examine version numbers or modification dates... this is more of a proof of concept that we can create useful disk images from flux dumps of 2.88 disks...)
Title: Re: Greaseweazle and NeXT floppies
Post by: user217 on July 20, 2025, 03:46:27 AM
Looks like an unqualified success, @pl212 — here's some imagery to demonstrate:

I have to say, the system for installing additional languages is a little bit clunky, as HSD Spell quits (and spits out the floppy) after installing one, but I was able to install the Danish dictionary with no trouble.

Interestingly, the fsck.technology images won't mount in Previous—they have that spurious 46 byte header you mentioned. I stripped it off and found this:

They're the same version! The file modification dates match, too. (Same goes for Disk 2.) This is perhaps better than starting off with a previously-unarchived release, as we can actually verify that they match. vbindiff says the images are not exactly the same—I think one or both may have been mounted without write protection, causing the OS to write some timestamps, but aside from that they seem to be nearly identical, so you can be confident your methods and media are robust.

Would you like me to archive your copy of the disks on my site?
Title: Re: Greaseweazle and NeXT floppies
Post by: KennyPowers on July 20, 2025, 10:10:04 AM
Quote from: pl212 on July 20, 2025, 12:45:48 AMIf anyone wants to test mounting under real/emulated Nextstep, BlueSCSI's floppy emulation, or other contexts, that would be great!

I can confirm that a BlueSCSI (and therefore probably a ZuluSCSI) happily mounted both of those 2.88MB images:





Was kind of funny seeing this in the BlueSCSI's log file ;D :

=== Configured SCSI Devices ===
ID: 0, BlockSize: 512, Type: Fixed, Quirks: None, Size: 1025730kB
ID: 2, BlockSize: 512, Type: Floppy1.4MB, Quirks: None, Size: 2880kB, Removable
Title: Re: Greaseweazle and NeXT floppies
Post by: pl212 on July 20, 2025, 07:24:46 PM
That's great! Looks like we have a good workflow for low-level archiving.  A few thoughts:

1) I suppose there's value to both the `.diskimage` format (DiskCopyII with extra header) and the `.img` format (no extra header; works with Previous, BlueSCSI and loopback mount).  With most people using either emulators or SCSI-replacement devices, I suspect most future archiving will move away from DiskCopyII.  (Can that app read bare images, without its own generated header? If so, .img files are clearly the best choice going forward.)

2) TechTangents has created a useful set of wrapper scripts (https://github.com/AkBKukU/gw-tools) for the Greaseweazle utilities, which automate creating the flux image, deriving a disk image, visualizing the image, and extracting the files onto a modern file system for casual inspection.

3) I've made a first attempt at extending his work towards Next floppies. Instead of "mtools" for DOS or "cpmtools" for CP/M, I'm using ditool from Previous to do the file extraction.

4) There's one fly in the ointment right now: TechTangent's code uses the hxcfe (https://github.com/latchdevel/HxCFloppyImageConverter) command-line tool to generate the cool green sector overlay visualizations.  On 2.88 disks of any kind (IBM DOS or Next UFS), hxcfe fails to correctly generate the 36 green track overlays. (It appears to do so when you load a floppy image into the gui; it's just the command-line image generation that hiccups.) You still get a visual representation, but there's a red-purple overlay that kind of makes it hard to see:



I've raised a question about this 2.88 disk issue in the HxC forums (https://hxc2001.com/floppy/forum/viewtopic.php?t=4657), hopefully there's some kind of easy fix I'm over looking.

Anyway, attached to this post are two versions of my custom script for TechTangent's gw-tools (https://github.com/AkBKukU/gw-tools).

gw-next.2880 will work in the manner most consistent with TechTangent's intent, and should result in a flux dump, a .img file, and a folder full of the disks's contents.  Until we find a fix, it will visualize the raw flux dump with the unfortunate red-purple overlay.

gw-next-vizfix.2880 cheats by running the visualization step on the decoded image file, which it groks correctly and generates the right sector overlay spokes. But to do this I had to insert the viz step at a later order via a slightly ugly kludge.

Both should have the .txt removed from their filename, and their execution bit (+x) set as they are bash scripts. Be sure to put them into, and run them from, the gw-tools folder. Also make sure you have either Previous or at least ditool (https://github.com/itomato/Previous/tree/master/src/ditool) installed, and that ditool is in your path.

As mentioned above, the gw-tools codebase is great way of getting both forensic and "useful" results in one fell swoop. Running it on, for example, Disk 2 of Mathematica 2.02 results in:

2025-07-20T16-50-05-1-Mathematica_202_Disk2-decode.log
2025-07-20T16-50-05-1-Mathematica_202_Disk2-extract.log
2025-07-20T16-50-05-1-Mathematica_202_Disk2-raw.log
Mathematica_202_Disk2-files/
Mathematica_202_Disk2.img
Mathematica_202_Disk2.png
Mathematica_202_Disk2-raw3/

The two folders, -raw3 and -files, contain the individual flux tracks and the extracted files, respectively.  The .img is a disk image, and the .png is the sector visualization. The three log files chronicle the various stages, and include ascii visualizations such as:

Cyl-> 0         1         2         3         4         5         6         7         
H. S: 01234567890123456789012345678901234567890123456789012345678901234567890123456789
0. 0: ................................................................................
0. 1: ................................................................................
0. 2: ................................................................................
0. 3: ................................................................................
0. 4: ................................................................................
0. 5: ................................................................................
[...]
1.34: ................................................................................
1.35: ................................................................................
Found 5760 sectors of 5760 (100%)

...and logs of the file and folder extractions, complete with ditool dutifully mentioning its failure to re-establish extended attributes and users, which is really just fine for casual inspection. :)


[DIR]   /
[DIR]   /..
[DIR]   /lost+found
[DIR]   /lost+found/.
[DIR]   /lost+found/..
[DIR]   /Mathematica.pkg
[DIR]   /Mathematica.pkg/.
[DIR]   /Mathematica.pkg/..
[FILE]  /Mathematica.pkg/Mathematica.info
[FILE]  /Mathematica.pkg/Mathematica.sizes
[FILE]  /Mathematica.pkg/Mathematica.bom
[FILE]  /Mathematica.pkg/Mathematica.tar.Z.2
[FILE]  /Mathematica.pkg/.last
[DIR]   /.NextTrash
[DIR]   /.NextTrash/.
[DIR]   /.NextTrash/..
---- done.
---- copying Mathematica_202_Disk2.img partition 0 to Mathematica_202_Disk2-files
---- setting file attributes for NFSD
setxattr(Mathematica_202_Disk2-files) failed
setxattr(Mathematica_202_Disk2-files/lost+found) failed
setxattr(Mathematica_202_Disk2-files/Mathematica.pkg/Mathematica.info) failed
setxattr(Mathematica_202_Disk2-files/Mathematica.pkg/Mathematica.sizes) failed
setxattr(Mathematica_202_Disk2-files/Mathematica.pkg/Mathematica.bom) failed
setxattr(Mathematica_202_Disk2-files/Mathematica.pkg/Mathematica.tar.Z.2) failed
setxattr(Mathematica_202_Disk2-files/Mathematica.pkg/.last) failed
setxattr(Mathematica_202_Disk2-files/Mathematica.pkg) failed
setxattr(Mathematica_202_Disk2-files/.NextTrash) failed
---- verifying inode structure
---- verifying file attributes and sizes
uid mismatch (act/exp) 20 != 0 /
gid mismatch (act/exp) 20 != 0 /
uid mismatch (act/exp) 20 != 0 /..
gid mismatch (act/exp) 20 != 0 /..
uid mismatch (act/exp) 20 != 0 /lost+found/.
gid mismatch (act/exp) 20 != 0 /lost+found/.
uid mismatch (act/exp) 20 != 0 /lost+found/..
gid mismatch (act/exp) 20 != 0 /lost+found/..
uid mismatch (act/exp) 20 != 0 /lost+found
gid mismatch (act/exp) 20 != 0 /lost+found
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/.
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/.
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/..
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/..
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.info
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.info
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.sizes
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.sizes
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.bom
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.bom
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.tar.Z.2
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/Mathematica.tar.Z.2
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg/.last
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg/.last
uid mismatch (act/exp) 20 != 0 /Mathematica.pkg
gid mismatch (act/exp) 20 != 0 /Mathematica.pkg
uid mismatch (act/exp) 20 != 0 /.NextTrash/.
gid mismatch (act/exp) 20 != 0 /.NextTrash/.
uid mismatch (act/exp) 20 != 0 /.NextTrash/..
gid mismatch (act/exp) 20 != 0 /.NextTrash/..
uid mismatch (act/exp) 20 != 0 /.NextTrash
gid mismatch (act/exp) 20 != 0 /.NextTrash
---- done.

This script can be easily adapted for 1.44 disks, of course -- and in fact I should try that to see if it solves the incorrect sector visualization problem.
Title: Re: Greaseweazle and NeXT floppies
Post by: pl212 on July 21, 2025, 05:22:25 PM
Quote from: KennyPowers on July 20, 2025, 10:10:04 AM=== Configured SCSI Devices ===
ID: 0, BlockSize: 512, Type: Fixed, Quirks: None, Size: 1025730kB
ID: 2, BlockSize: 512, Type: Floppy1.4MB, Quirks: None, Size: 2880kB, Removable


This makes me think I should attach my 2.88mb PLI SuperFloppy and see what it reports as... then contribute that string as a pull request to BlueSCSI :)
Title: Re: Greaseweazle and NeXT floppies
Post by: KennyPowers on July 21, 2025, 06:00:23 PM
Quote from: pl212 on July 21, 2025, 05:22:25 PMThis makes me think I should attach my 2.88mb PLI SuperFloppy and see what it reports as... then contribute that string as a pull request to BlueSCSI :)

https%3A%2F%2Fmedia.tenor.com%2F7EttrztgjiAAAAAM%2Fpalpatine-star-wars.gif

I should have a PLI SuperFloppy soon too  ;D
Title: Re: Greaseweazle and NeXT floppies
Post by: pl212 on July 22, 2025, 12:24:24 PM
Quote from: KennyPowers on July 21, 2025, 06:00:23 PMhttps%3A%2F%2Fmedia.tenor.com%2F7EttrztgjiAAAAAM%2Fpalpatine-star-wars.gif

I should have a PLI SuperFloppy soon too  ;D

That's great! Be sure to check the output values of the AC adapter -- mine was supposed to be 7.5v but was putting out 10.5v. Maybe that's within range but I decided to run it off a bench power supply to be safe.

Would also be interested in what ROM revision you have -- I have 1.4; somebody else on the board has 1.5...

Looks like the drive reports as:

PLI          SUPER FLOPPY


Go to top  Forum index