NeXTflix

Hi everybody!

A couple(?) of years after macflim (https://www.macflim.com/macflim2), the video player for vintage macs, I'm starting to work on NeXTflix (I did register the domain name a few years ago). (Yep, this is why I wanted network access to my Cube + a working Previous).

Some early progress (see https://oldbytes.space/@fred/111892523851282745), but I am hitting an issue with sound.

Does anyone knows how to play digital sound by chunks without gaps?

I've been trying ``SNDStartPlaying`` followed by another in the begin callback, but there is gaps between the sounds. Any idea?

(If there are motivated people, I can post a shortened version of the sound playing tests)

I don't know if this will help or not but Frank Siegert has a program available with source code that contains audio streaming abilities. There may be some clues that could be picked up from the source code.

Speak-Freely-6.1c - an Internet phone program suite ported to NeXTSTEP/OpenStep. Includes audio stream recording and playing programs for NeXTSTEP (Intel) that really works on the braindead Intel hardware. The archive contains binaries for NS Intel and full sources. However you will notice that NeXT m68k hardware is too slow to support real-time compression. The original Speak-Freely page can be found at Fourmilab. Here is the ported version:

http://www.wizards.de/~frank/download.html (http://www.wizards.de/~frank/download.html)

Wow, very nice project and performance in the video. I'm looking forward to see your project progress.

I'm not sure if this is of any help... or different to what you did audio wise

I was trying to build a small Amiga CDXL player for nextstep some time ago based on an existing player for modern plattforms. It was a bit too "slow" with audio, so it may not be gapless after all, likely because of me doing bad coding and also doing conversion. Here are a few fragments.... perhaps its of any value.



void DoAudio ( CDXLWindowController* c )
{
    if( c->audioActivated &&
            c->isMainPlayer &&
            c->pd.frame != NULL &&
            c->pd.frame->audio_size > 0 )
    {
        NXSoundDeviceError error = NX_SoundDeviceErrorNone;
   
        unsigned int soundDataSize = c->pd.frame->audio_size;
        int channelCount = c->pd.frame->is_stereo ? 2 : 1;

        void* soundData = c->pd.frame->buffer + (c->pd.frame->pan.Size - c->pd.frame->pan.AudioSize);
   
        SNDError result = SND_ERR_NONE;

        char* sourceDataPtr = NULL;
        int size = 0;
        int width = 0;       
        SNDSoundStruct* sourceSound = NULL;
        SNDSoundStruct* destSound = NULL;

        result = SNDAlloc( &sourceSound,
            soundDataSize,
            SND_FORMAT_LINEAR_8,
            c->audioFrequencyMode == 0 ? SND_RATE_LOW_PC : SND_RATE_LOW,   
            channelCount,
             0);
       
         result = SNDGetDataPointer(sourceSound, &sourceDataPtr, &size, &width );
       
        memcpy( sourceDataPtr, soundData, soundDataSize);
       
        // working but slow

        if( [c->audioBufferPool count] > 0 )
        {
            destSound = (SNDSoundStruct*)[c->audioBufferQueue elementAt:0];
             [c->audioBufferPool removeElementAt:0];
        }
        else
        {
            result = SNDAlloc( &destSound, 0, SND_FORMAT_LINEAR_16, SND_RATE_HIGH, channelCount, 0);
        }
       
        result = SNDConvertSound( sourceSound, &destSound);    
        result = SNDFree(sourceSound);
       
        if( SND_ERR_NONE != result )
        {
            NXLogError("Couldn't convert %s", SNDSoundError( result ) );
        }
       
        if( destSound != NULL)
        {
                if( [c->audioBufferQueue count] == 0)
                {
                    result = SNDGetDataPointer(destSound, &sourceDataPtr, &size, &width );

                    error = [c->soundStream playBuffer: (void *)sourceDataPtr
                                              size: destSound->dataSize
                                               tag: (int)destSound ];
                }
                else
                {
                    [c->audioBufferQueue addElement: destSound];
                }
        }
               
        if (error != NX_SoundDeviceErrorNone)
        {
                printf("sound error on schedule: %s\n", [NXSoundDevice textForError: error] );
        }
    }
}

// in controller
-(NXPlayStream*) soundStream
{
    static BOOL cantPlaySounds = NO;
    static NXSoundOut *dev = nil;
   
      NXPlayStream *newStream = nil;

      if (cantPlaySounds)
    {
         return nil;       
      }
   
    if (!dev && !(dev = [[NXSoundOut alloc] init]))
    {   
        NXLogError ("Couldn't create NXSoundOut");
        cantPlaySounds = YES;
           
        return nil;
      }
    
    newStream = [[NXPlayStream alloc] initOnDevice:dev];
   
    if (newStream)
    {
        if (![newStream isActive] &&
                ([newStream activate] != NX_SoundDeviceErrorNone) )
        {
            NXLogError ("Couldn't activate Playstream");

        [newStream free];
           
        newStream = nil;
        }
    }

    [newStream setDelegate:self];
    
    return newStream;
}


- soundStream:sender didCompleteBuffer:(int)tag
{
    int result = 0;
   
    if(tag != 0)
    {
        //result = SNDFree( (SNDSoundStruct*)tag );
        [audioBufferPool addElement: (SNDSoundStruct*)tag];
    }

    if( [audioBufferQueue count] > 0 )
    {
        NXSoundDeviceError error = NX_SoundDeviceErrorNone;
       
        char* sourceDataPtr = NULL;
        int size = 0;
        int width = 0;   
       
        SNDSoundStruct* destSound = (SNDSoundStruct*)[audioBufferQueue elementAt:0];

         [audioBufferQueue removeElementAt:0];

        result = SNDGetDataPointer(destSound, &sourceDataPtr, &size, &width );

        error = [soundStream playBuffer: (void *)sourceDataPtr
                                      size: destSound->dataSize
                                    tag: (int)destSound ];
    }

    return sender;
}

- soundStreamDidAbort:sender deviceReserved:(BOOL)flag
{
    return [self soundStream:sender didCompleteBuffer:0];
}

- soundStreamDidUnderrun:sender
{
    return sender;
}


Wow now that is very cool! Did you see the sound apps in the archives? https://www.nextcomputers.org/NeXTfiles/Software/NEXTSTEP/Apps/Audio/

Is the source file from streaming, CDrom or the hard drive?

I know certain CDrom drives we're able to play sounds better than others using 3rd party apps like CD Evil which worked with Toshibas ....

Digital Ears hooked up through the DSP port and let you interface with your home Stereo
https://www.nextcomputers.org/NeXTfiles/Docs/Hardware/DigitalEars/Also NeXTtime has sound but

what you are doing there is really cool! Does it work in Color as well? 

Quote from: Nitro on February 14, 2024, 01:00:03 AMI don't know if this will help or not but Frank Siegert has a program available with source code that contains audio streaming abilities. There may be some clues that could be picked up from the source code.
http://www.wizards.de/~frank/download.html (http://www.wizards.de/~frank/download.html)

Thanks, this was extremely useful, and proved me that my approach should have worked. I am not 100% sure of what was the exact problem as I basically rewrote the whole thing, but I now have seamless sound, so I am one step closer to the whole thing!

Quote from: GrafZahl on February 14, 2024, 10:18:34 AMWow, very nice project and performance in the video. I'm looking forward to see your project progress.

I'm not sure if this is of any help... or different to what you did audio wise

Thanks. I used Nitro pointer to get the sound running. I think yours would probably have helped me the same :-)

Quote from: Rob Blessin Black Hole on February 14, 2024, 11:36:23 AMDid you see the sound apps in the archives? https://www.nextcomputers.org/NeXTfiles/Software/NEXTSTEP/Apps/Audio/

I didn't try that, as I expected myself getting very distracted with recompiling things and fighting the roadblocks. Anyway, thanks to Nitro, I'm moving forward.

Quote from: Rob Blessin Black Hole on February 14, 2024, 11:36:23 AMIs the source file from streaming, CDrom or the hard drive?

For now, I generate the samples in memory. In the final version it will extract the samples from the a disk-based stream, interleaved with the video. I do preliminary research so I can get the best possible stream format that avoid copies (for instance, thanks to a link above, I learnt that I should be using multiple of 8192 samples, which is very valuable intel).

Quote from: Rob Blessin Black Hole on February 14, 2024, 11:36:23 AMwhat you are doing there is really cool! Does it work in Color as well? 

No, it won't, for a few reasons:

• there is much more data in color and the whole approach would probably have to be changed
• the goal of the project is to showcase the NeXT, so the MegaPixel output is the focus
• it is a companion to MacFlim, which is limited to B&W (next one in the list will probably be the TRS-80 Model 100/200)
• I love the error dithering style of the images
• I dreamt of doing something like that since my first NeXT in 1991...
• I am looking for one, but I don't have a color NeXT station :-)

That said, I think you may be one of the few people on this earth that could help me solve that last issue...

And thanks a lot of finding it cool! Still a lot of work to do, and I am a bit worried on the streaming aspect form the SCSI drive. NeXTstep wasn't known for its real-time capabilities...

Gonna be on holidays, so no progress for the next few days, but I'll post when I'll make progress on the real app.

Also, if people want to test, I'll publish it on github early (normally I'll wait until done, 'cause it is additional work).

• I am looking for one, but I don't have a color NeXT station :-)

That said, I think you may be one of the few people on this earth that could help me solve that last issue...

And thanks a lot of finding it cool! Still a lot of work to do, and I am a bit worried on the streaming aspect form the SCSI drive. NeXTstep wasn't known for its real-time capabilities...

Gonna be on holidays, so no progress for the next few days, but I'll post when I'll make progress on the real app.

Also, if people want to test, I'll publish it on github early (normally I'll wait until done, 'cause it is additional work).
[/quote]  Hello FStark :  I look forward to helping on your project and you know what you have done here is so cool , I indeed have a NeXT Colorstation for you :) If you cover the shipping and let me install this on NeXT's for all my customers as it is really fun! I will offer a Color Station heavily discounted to you for $100 and shipping to show our appreciation fr the Matrix on NeXt lol..... That is for the color station case, motherboard , used floppy and power supply , I'll even include a new bios battery. Then if you would optionally like to upgrade to any of these , a new old stock power supply $15 new old stock floppy drive $10 ...also I have some brand new ram max is 32Mb 4Mb 72 pin simms in sets of 2 for $64 or used $32 = 32Mb, We can do new Austrian caps as well recommended Matt does a great job $75... it'll have an original NeXT Noisy lol hard drive and I would recommend a blue scsi or Zulu scsi I have both and can load them up with hundreds of NeXT 3.3 applications and documentation , developer tools and Y2k patches  on a 32GB Extreme Pro for $199.95. We can talk when you get back. I have a custom cable to flat panel$79.95 as well and a Brian's sound card $199.95 as I'm guessing you would need them ,you will be able to use your NeXT non add keyboard and mouse with it. 

@fstark did you develop this on NeXT or do you use modern tools?

I know a lot of Amiga developers use modern IDE because of tab completion and just fire up WinUAE afterwards.

Do you compile it to a NFS folder and run from there?

That dithering on MacFlim2 sample gifs.  8)

I was on holidays the last few days with no way to answer here.

Quote from: Rob Blessin Black Hole on February 16, 2024, 11:08:18 PMHello FStark :  I look forward to helping on your project and you know what you have done here is so cool , I indeed have a NeXT Colorstation for you :) If you cover the shipping and let me install this on NeXT's for all my customers as it is really fun! I will offer a Color Station heavily discounted to you for $100 and shipping to show our appreciation fr the Matrix on NeXt lol..... That is for the color station case, motherboard , used floppy and power supply , I'll even include a new bios battery. Then if you would optionally like to upgrade to any of these , a new old stock power supply $15 new old stock floppy drive $10 ...also I have some brand new ram max is 32Mb 4Mb 72 pin simms in sets of 2 for $64 or used $32 = 32Mb, We can do new Austrian caps as well recommended Matt does a great job $75... it'll have an original NeXT Noisy lol hard drive and I would recommend a blue scsi or Zulu scsi I have both and can load them up with hundreds of NeXT 3.3 applications and documentation , developer tools and Y2k patches  on a 32GB Extreme Pro for $199.95. We can talk when you get back. I have a custom cable to flat panel$79.95 as well and a Brian's sound card $199.95 as I'm guessing you would need them ,you will be able to use your NeXT non add keyboard and mouse with it.

To quote Neo: "woah". This is beyond awesome. I am interested in the station+power supply+floppy. I have someone local that does recapping+ram. I also have SCSI2SD and a few BluSCSI, so that's not needed either. The custom cable and sound box replacement are a must. And, if I am not wrong, the custom cable would also allow me to connect directly my Dimension to a color screen? In the case, I'll take 2 :-). Should we move that to DM or do you prefer talking about it here?

Second, on the "and let me install this on NeXT's for all my customers as it is really fun!". Of course, this will be MIT license, you'll be able to do whatever you want. We may have to wait until it actaully fully work, though :-) And if you have any special needs / a video that you really-really-really want to perform well on it, let me know.

Third, for playback on the color NeXT, I can't promise anything. I suspect the memory bandwidth will just not be there for a full screen viewing, and, if I remember correctly, NEXTTIME was already capable of playing some windowed videos. Need to look into it, but don't hold you breath.

Thanks for that awesome response. Made my day!

Quote from: pTeK on February 17, 2024, 04:20:19 PM@fstark (https://www.nextcomputers.org/forums/index.php?action=profile;u=3272) did you develop this on NeXT or do you use modern tools?

My workflow is the following:

The encoder is a C++ app, running on Linux (well, it also have a python part running a neural network that does some optional pre-processing. Unsure is this will stay in the final version). This is developped using Visual Code, and run on the Linux itself (it generates the "flix" file and a set of png that I use to look at the result quality without having to run on the NeXT/guess if something seems better or not).

The player is a C app. It is developed on Visual Code, and compiled on Previous (there is a nfs share, @andreas_g (https://www.nextcomputers.org/forums/index.php?action=profile;u=803) helped me for setup). I sometimes check if it works there, but only in a technical way (ie: does it core dump?), as the emulation is not precise enough for video and sound.

I then use an ftp connection to the NeXT to 'put' the nextflix command, and use a telnet to run it (ssh is way too slow). I also 'put' the flix file when I change it.

Anyway, I'm gonna try to get the sound running before the week-end!

So, I dig get sound before the week-end, but it was pretty bad so I didn't post. The good news is that it is now correct.

The bad news is that SNDStartPlaying will never work properly and I need a fresh approach to sync sound and video on the NeXT. The issues is that the sound callbacks are delayed arbitrarily, in general connected to disk activity.

Need some time to find a solution, I have plenty of ideas, slightly pissed-off that it probably mean rewriting a large chunk of the code. Hey, that's life.

Progress video attached to the mastodon post:

https://oldbytes.space/@fred/112005741850816067

Quote from: fstark on February 15, 2024, 10:40:07 AMAlso, if people want to test, I'll publish it on github early (normally I'll wait until done, 'cause it is additional work).

Yes please!  8)

Any news on the project lately?  :) Do you think it would be possible to compile this to run on other platforms in addition to 68k (Risc, Sparc, Intel...)?

Quote from: fstark on February 27, 2024, 04:17:42 PMSo, I dig get sound before the week-end, but it was pretty bad so I didn't post. The good news is that it is now correct.

The bad news is that SNDStartPlaying will never work properly and I need a fresh approach to sync sound and video on the NeXT. The issues is that the sound callbacks are delayed arbitrarily, in general connected to disk activity.

Need some time to find a solution, I have plenty of ideas, slightly pissed-off that it probably mean rewriting a large chunk of the code. Hey, that's life.

Progress video attached to the mastodon post:

https://oldbytes.space/@fred/112005741850816067

So NeXTTime was added to NeXTstep around 3.3 and I could swear was rolled into the OS with OPENSTEP 4.2. I could be wrong, but I would think there are some library calls for playing quicktime media. Perhaps that is a good starting point? And then maybe some transcoding into that?

I noticed Quake2 for Raphsody sound code looked very low-level, may be even it was cut from quicktime?

https://github.com/id-Software/Quake-2/blob/master/rhapsody/snd_next.m