CDDA Won't Loop Last Track on GDEMU/MODE

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Post Reply
User avatar
rpk
DC Developer
DC Developer
Posts: 16
https://www.artistsworkshop.eu/meble-kuchenne-na-wymiar-warszawa-gdzie-zamowic/
Joined: Thu Dec 29, 2016 1:02 pm
Has thanked: 10 times
Been thanked: 12 times

CDDA Won't Loop Last Track on GDEMU/MODE

Post by rpk »

Hi,

I wanted to share the process I've gone through over the last two days trying to understand why the CDDA background music in my game HarleQuest! refused to loop. I came from knowing next to nothing about CDDA or CD layouts and everything I've learned about it has been over the past couple of days, so if I get something wrong please bear with me (and correct me!).

First, I always test HarleQuest! on a Dreamcast that's fitted with a GDEMU clone. This time, I happened not to test it for long enough to hear the music loop on real hardware. I did test that the music loops in an emulator which worked fine.

After releasing a new demo build, multiple people got in touch saying that the music was either stopping or hanging on a single tone after the first playthrough. I tested this on my DC with GDEMU and yep, it didn't loop - just silence.

I'm using the cdrom_cdda_play() KallistiOS function as follows:

Code: Select all

// Start at track 1, stop after track 1, loop 15 times (infinite):
result = cdrom_cdda_play(1, 1, 15, CDDA_TRACKS);
This was taken directly from the KallistiOS examples and it matched all the example code I could find across different DC projects.

I built the disc image with the mkdcdisc tool: https://gitlab.com/simulant/mkdcdisc
Here is the command I used:

Code: Select all

./build/mkdcdisc -a "Kilgariff Technologies Limited" --cdda bgm.wav -e harlequest.elf -n "harlequest" -o /home/ross/harlequest.cdi
This produced a .cdi file with the following layout:
harlequest-demo-3-layout.png
Notice that session 1 is the audio session with a single track, and session 2 is the data session also with a single track. It's the simplest possible .CDI file that correctly boots on the Dreamcast and plays CDDA audio, but the background music wouldn't loop.

At first I thought it was a bug in KallistiOS, so I looked into the implementation of cdrom_cdda_play:
https://github.com/KallistiOS/KallistiO ... rom.c#L388

Code: Select all

int cdrom_cdda_play(uint32 start, uint32 end, uint32 repeat, int mode) {
    struct {
        int start;
        int end;
        int repeat;
    } params;
    int rv = ERR_OK;

    /* Limit to 0-15 */
    if(repeat > 15)
        repeat = 15;

    params.start = start;
    params.end = end;
    params.repeat = repeat;

    mutex_lock(&_g1_ata_mutex);

    if(mode == CDDA_TRACKS)
        rv = cdrom_exec_cmd(CMD_PLAY, &params);
    else if(mode == CDDA_SECTORS)
        rv = cdrom_exec_cmd(CMD_PLAY2, &params);

    mutex_unlock(&_g1_ata_mutex);

    return rv;
}
All this function does is send CMD_PLAY to the bootrom - basically a syscall to initiate CDDA playback. So it's not a bug with KallistiOS.

I noticed there was a second option to play using sector ranges instead of tracks (CDDA_SECTORS) and a KOS function to get the TOC (Table of Contents) via a syscall, so I wrote a small piece of code to display all the tracks in the TOC:

Code: Select all

// Check our CD's table of contents:
{
    CDROM_TOC toc;
    result = cdrom_read_toc(&toc, 0);

    if (result == ERR_OK)
    {
        for (int track = TOC_TRACK(toc.first); track <= TOC_TRACK(toc.last); ++track)
        {
            int idx = track - 1;
            uint32_t entry = toc.entry[idx];
            printf("========= TOC Entry %d\n", idx);
            printf("Entry LBA: %d\n", (int) TOC_LBA(entry));
            printf("Entry ADR: %d\n", (int) TOC_ADR(entry));
            printf("Entry CTRL: %d\n", (int) TOC_CTRL(entry));
            printf("Entry TRACK: %d\n", (int) TOC_TRACK(entry));

            if (TOC_CTRL(entry) == 0)
            {
                printf("This is an audio track\n");
            }
        }
    }
}
This gave the following output:

Code: Select all

========= Entry 0
Entry LBA: 150
Entry ADR: 1
Entry CTRL: 0
Entry TRACK: 0
This is an audio track
========= Entry 1
Entry LBA: 22600
Entry ADR: 1
Entry CTRL: 4
Entry TRACK: 0
By calling cdrom_cdda_play(150, 400, 15, CDDA_SECTORS) where 400 was just a number chosen at random, it successfully looped the first few seconds of the track. However, there was no way using the TOC to determine where the track should end because instead of displaying a separate TOC for each session (including the lead-out time for each), the Dreamcast bootrom appears to just write a single merged TOC with all the entries. The KallistiOS function cdrom_read_toc() says it takes a session number as a parameter, but in practice passing anything other than 0 would result in the function returning ERR_SYS (3) and the TOC being uninitialised memory. So I was at a loss how to solve the problem with sectors, short of using offline tools to generate a static TOC and building it into my game.

On a whim I tried adding a second audio track, and lo and behold, that allowed the first track to loop successfully. It was now the second audio track that wouldn't loop.

Next I thought it might be a bug with mkdcdisc, but it was clearly a valid .CDI image that ImgBurn could parse and display. I went through the mkdcdisc code and there were a couple red herrings due to my own lack of understanding, but in the end it all looked fine. To help with this, Kazade (author of mkdcdisc) booted up Driving Strikers and checked whether the last track looped - it did! So that implied something strange with my setup. He sent me a .CDI of the game to test, but for me the last track did not loop and was just followed by silence.

The only difference between our setups is that I am using a GDEMU clone and he is using a real GDROM drive. So I burnt a HarleQuest! test disc and tried it on a Dreamcast with a real GDROM drive - and the music loops fine.

After looking around, I then found a reddit post in which a user mentions background music not looping on their GDEMU clone: https://www.reddit.com/r/dreamcast/comm ... emu_clone/

All the reports of music not looping had come from people either with GDEMU or MODE installed, so it seems to be issue with both of these. It hasn't been tested on an official GDEMU yet.

If anyone can shed light on why this might be that would be awesome. In the meantime, I'm just going to add an extra blank track to the end of the audio session and move on to other things.

Thanks,
Ross
These users thanked the author rpk for the post:
Ian Robinson
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by TapamN »

The "session" parameter for the TOC function controls if you are reading a CD TOC (on a 0) or a GD TOC (on a 1). Obviously, CDs don't have a GD area, so requesting the GD TOC will fail. For the end of the last track, could you use the leadout sector value as the loop end? You might have to subtract a bit from it; if you do, I would guess something like 1, 75, 76, 150, or 151 would work. I guess if a dummy track is enough to make GDEMU (or clones) happy, that's fine as a work around.

This doesn't exactly help with what you asked right now, but I've been working on a low CPU usage, seamlessly looping, streaming music system, based on the QOA codec (similar to ADPCM, at about 0.4 bytes per sample), with a replacement CD driver. I tried working on streaming compressed music in KOS before, were I found that KOS had horribly inconsistent CPU usage (anywhere from 0 to 60% usage per frame).

It's currently not complete. I've modified QOA to be better suited for the Dreamcast (changing the format from 64-bit big endian to 32-bit little endian). The reference implementation could only decode the entire file at once, so I've also had to rewrite the decoder to handle streaming.

The new CD driver uses DMA (KOS has the CPU copy the data) and will allow for multiple reads to occur, so that you can stream music while loading something else. I'm not planning on adding any real scheduling to it, just run each request to completion, in order, since that's simple to understand and implement, but with a preemptive priority system. High priority requests (like music) can preempt lower priority requests (like level data) so that the music won't skip. Preempted requests will be resumed from where they left off once the higher priority one fully completes.

The CD driver is only partially working at the moment; I can read raw sectors with it, but don't have the file system working yet.

When everything is finished, I think CPU usage for 44100 hz stereo should be no more than about 3% per frame.
These users thanked the author TapamN for the post (total 2):
Ian Robinsonrpk
User avatar
rpk
DC Developer
DC Developer
Posts: 16
Joined: Thu Dec 29, 2016 1:02 pm
Has thanked: 10 times
Been thanked: 12 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by rpk »

That's really interesting, I'll be curious to see how it all works when it's done.

I can't use the leadout to determine the track length because the TOC is merged across both the audio and data sessions when returned by the bootrom syscall. The merged TOC only has the leadout for the last track overall (ie. the data track).

You're right about the "session" being a selector for CDROM/GDROM TOC, I had a suspicion it was something like that.
These users thanked the author rpk for the post:
Ian Robinson
User avatar
Ian Robinson
DC Developer
DC Developer
Posts: 114
Joined: Mon Mar 11, 2019 7:12 am
Has thanked: 206 times
Been thanked: 41 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by Ian Robinson »

TapamN wrote: Thu May 25, 2023 6:44 am The "session" parameter for the TOC function controls if you are reading a CD TOC (on a 0) or a GD TOC (on a 1). Obviously, CDs don't have a GD area, so requesting the GD TOC will fail. For the end of the last track, could you use the leadout sector value as the loop end? You might have to subtract a bit from it; if you do, I would guess something like 1, 75, 76, 150, or 151 would work. I guess if a dummy track is enough to make GDEMU (or clones) happy, that's fine as a work around.

This doesn't exactly help with what you asked right now, but I've been working on a low CPU usage, seamlessly looping, streaming music system, based on the QOA codec (similar to ADPCM, at about 0.4 bytes per sample), with a replacement CD driver. I tried working on streaming compressed music in KOS before, were I found that KOS had horribly inconsistent CPU usage (anywhere from 0 to 60% usage per frame).

It's currently not complete. I've modified QOA to be better suited for the Dreamcast (changing the format from 64-bit big endian to 32-bit little endian). The reference implementation could only decode the entire file at once, so I've also had to rewrite the decoder to handle streaming.

The new CD driver uses DMA (KOS has the CPU copy the data) and will allow for multiple reads to occur, so that you can stream music while loading something else. I'm not planning on adding any real scheduling to it, just run each request to completion, in order, since that's simple to understand and implement, but with a preemptive priority system. High priority requests (like music) can preempt lower priority requests (like level data) so that the music won't skip. Preempted requests will be resumed from where they left off once the higher priority one fully completes.

The CD driver is only partially working at the moment; I can read raw sectors with it, but don't have the file system working yet.

When everything is finished, I think CPU usage for 44100 hz stereo should be no more than about 3% per frame.
How is it going with this format and the CD driver? I think this would be one of the most major updates for Kos and allow sound to not burden as much as it does. Almost all formats are useless and we are forced to use CDDA if we want any speed. The number one thing I'm looking forward to is QOA, and the updated CD driver would bring CPU alignment with the official development kit of 3%. I did a test, and it is 2 ms at 44 kHz mono.

As you found out before KOS had horribly inconsistent CPU usage (anywhere from 0 to 60% usage per frame).
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by TapamN »

I've been distracted with other things, but getting back to the driver and decoder is my next task.

I ran across a post mentioning FFMPEG's ELBG for VQ compression, tried using that for Dreamcast VQ, and it worked very well. I ended up making such good progress on getting a texture compressor/encoder that I wanted to just keep going.

It has several improvements over tvspelsfreak's texconv, compression is 2-4 times faster at equivalent quality (in some places ELBG does better, in others texconv does better, overall it evens out), it has support for dithering (important for ARGB4444 and 4/8 BPP), cartesian normal map input (texconv could only take height maps), better mipmap generation, support for compressed non-power-of-two textures (you can create a 640x480 stride VQ image, takes 77 KB), small codebooks to improve compression ratio and fillrate, and can output .PVR, .VQ, and a new .DT format that's an improvement on both. It also doesn't have the Qt dependency texconv has, so it should be much easier for people on Windows or Mac to compile.
These users thanked the author TapamN for the post (total 2):
|darc|Ian Robinson
User avatar
Protofall
DCEmu Freak
DCEmu Freak
Posts: 78
Joined: Sun Jan 14, 2018 8:03 pm
Location: Emu land
Has thanked: 21 times
Been thanked: 18 times
Contact:

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by Protofall »

TapamN wrote: Mon Aug 07, 2023 5:27 pm I've been distracted with other things, but getting back to the driver and decoder is my next task.

I ran across a post mentioning FFMPEG's ELBG for VQ compression, tried using that for Dreamcast VQ, and it worked very well. I ended up making such good progress on getting a texture compressor/encoder that I wanted to just keep going.

It has several improvements over tvspelsfreak's texconv, compression is 2-4 times faster at equivalent quality (in some places ELBG does better, in others texconv does better, overall it evens out), it has support for dithering (important for ARGB4444 and 4/8 BPP), cartesian normal map input (texconv could only take height maps), better mipmap generation, support for compressed non-power-of-two textures (you can create a 640x480 stride VQ image, takes 77 KB), small codebooks to improve compression ratio and fillrate, and can output .PVR, .VQ, and a new .DT format that's an improvement on both. It also doesn't have the Qt dependency texconv has, so it should be much easier for people on Windows or Mac to compile.
Hey TapamN, I just heard about your project and its really cool. Its always good to have multiple options, especially since not everyone wants to use Qt stuff.

A few things you said stand out to me and other people, so I'd like to ask for more details about them:
- Can you really have a compressed "stride" texture? When I researched PVR texture formats years ago, I came to the conclusion that wasn't legal
- You mention the .PVR, .VQ, and a new .DT formats, as well as stating DT is the best of those 3. What makes the format better? And how does it differ from texconv's .dtex format?
- You mention you can make 640 by 480 stride textures, I understand this works by making a 640 by 512 space in VRAM and fill the last 32 rows with junk data, never to be rendered. However, what if you put this at the end of VRAM, to "remove" those last 32 rows of unused texture. Would this still work, or cause issues when rendering?
These users thanked the author Protofall for the post:
freakdave
Moving Day: A clone of Dr Mario with 8-player support <https://dcemulation.org/phpBB/viewtopic ... 4&t=105389>
A recreation of Minesweeper for the Dreamcast <viewtopic.php?f=34&t=104820>

Twitter <https://twitter.com/ProfessorToffal>
YouTube (Not much there, but there are a few things) <https://www.youtube.com/user/TrueMenfa>
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by TapamN »

Protofall wrote: Fri Aug 25, 2023 6:38 amHey TapamN, I just heard about your project and its really cool. Its always good to have multiple options, especially since not everyone wants to use Qt stuff.

A few things you said stand out to me and other people, so I'd like to ask for more details about them:
- Can you really have a compressed "stride" texture? When I researched PVR texture formats years ago, I came to the conclusion that wasn't legal
- You mention the .PVR, .VQ, and a new .DT formats, as well as stating DT is the best of those 3. What makes the format better? And how does it differ from texconv's .dtex format?
- You mention you can make 640 by 480 stride textures, I understand this works by making a 640 by 512 space in VRAM and fill the last 32 rows with junk data, never to be rendered. However, what if you put this at the end of VRAM, to "remove" those last 32 rows of unused texture. Would this still work, or cause issues when rendering?
Compression seems to work for every possible format (although there is a hardware bug with bi/trilinear when using compressed 8/4BPP with mipmaps). I tried it on real hardware and it works. For a non-power-of-two height stride texture, you tell the PVR that the texture is larger than you allocated space for, then just be careful not to display anything from the unallocated part of the texture when rendering. You don't have to reserve RAM for the area you aren't using. Storing the texture at the end of RAM would not help, it would probably wrap around and try to treat the start of video RAM as the end of the texture.

A more concrete example:

For a 640x480 VQ stride texture, you allocate space for the codebook (up to 2048 bytes) and the indices (640*480*2/8); this is 640*480*2/8+2048=78848 bytes for a full codebook. When you use the texture, you tell the PVR that it's a 640x512 stride texture (640x1024 would work too, if you wanted to be weird), and then you're careful not to draw anything from the Y range 481-512, since the PVR would sample from whatever happens to be stored after the texture. If you're drawing a full screen quad, the UVs for the top left corner would be (0, 0), and the bottom right would be (640/1024.0f, 480/512.0f). It's worth noting that if you enable bilinear, the filter could sample a bit from outside the range specified by the UV values on the top or bottom lines of the texture, and miscolor those pixels a bit.

Here are some sample images of 640x480 VQ. Stride/nontwiddled VQ is a lower quality than normal VQ, since it uses 4x1 blocks instead of 2x2. It's possible to dither VQ indices, which might help with vertical gradients (like "2" in the SA2 logo), but that's not implemented at the moment.

The two biggest improvements my format has over texconv's format is support for small VQ codebook textures (codebook can be any size, from 1-256 entries, .PVR had a fixed codebook size depending on texture dimensions/mipmapping), and 32-byte pixel data alignment to make DMA easier. texconv's format has a 16 byte header, so if you load the file into a buffer with 32 byte alignment, you can't use DMA because the pixel data is off alignment. You either have to do a memcpy to align the data, or go out of your way to load the file offset 16 bytes into a 32-byte aligned buffer.

Another smaller improvement is that a color count is included for palettized textures, to help allow dynamic allocation of palette entries. You can have a 8 bit per pixel texture that only uses 25 colors, then when you load the texture to VRAM, you can remap the values to wherever you allocate the 25 colors in the global palette.

I'm getting very close to releasing the first version soon (then I'll go back to the CD driver). Reading your post actually delayed it a bit, since I just realized I've been calling texconv's format .VQ to myself (since that's what I ended up using as a extension in my wrapper script for my model converter) but it looks like that extension/name isn't used anywhere by texconv, so I'm going to go and fix the name in my source code to something more official. I guess I'll use .TEX, since that's the extension used in texconv's readme.
These users thanked the author TapamN for the post:
Ian Robinson
User avatar
Protofall
DCEmu Freak
DCEmu Freak
Posts: 78
Joined: Sun Jan 14, 2018 8:03 pm
Location: Emu land
Has thanked: 21 times
Been thanked: 18 times
Contact:

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by Protofall »

TapamN wrote: Fri Aug 25, 2023 10:25 pm (although there is a hardware bug with bi/trilinear when using compressed 8/4BPP with mipmaps)
Interesting. I guess tvspelsfreak just didn't test it and accepted the SEGA docs like the rest of us did. Glad you tested it. And yeah, makes sense with that palette+compressed+X-linear bug, since SEGA didn't even intend compressed paletted textures to be usable.
TapamN wrote: Fri Aug 25, 2023 10:25 pm Storing the texture at the end of RAM would not help, it would probably wrap around and try to treat the start of video RAM as the end of the texture.
Ah good. I was mostly concerned about crashes, so even that situations is fine.
TapamN wrote: Fri Aug 25, 2023 10:25 pm It's worth noting that if you enable bilinear, the filter could sample a bit from outside the range specified by the UV values on the top or bottom lines of the texture, and miscolor those pixels a bit.
Yes. I was reminded of this recently, and would explain points where I was confused with video output, back in 2017 - 2020.

TapamN wrote: Fri Aug 25, 2023 10:25 pm texconv's format has a 16 byte header, so if you load the file into a buffer with 32 byte alignment, you can't use DMA because the pixel data is off alignment. You either have to do a memcpy to align the data, or go out of your way to load the file offset 16 bytes into a 32-byte aligned buffer.
Huh, yeah actually that alone is a good reason to make a new format. That would be interesting to see how you implemented the Small-VQ into the file format while still preserving the 32-byte aligned requirement.

I'm looking at my old notes on Small-VQ and tbh I don't really understand it anymore, haha https://github.com/Protofall/Crayon-Uti ... ression%3F Looks like a way to combine many small textures together in VRAM by abusing codebook's empty/predictable space, right?

TapamN wrote: Fri Aug 25, 2023 10:25 pm I'm getting very close to releasing the first version soon (then I'll go back to the CD driver). Reading your post actually delayed it a bit, since I just realized I've been calling texconv's format .VQ to myself (since that's what I ended up using as a extension in my wrapper script for my model converter) but it looks like that extension/name isn't used anywhere by texconv, so I'm going to go and fix the name in my source code to something more official. I guess I'll use .TEX, since that's the extension used in texconv's readme.
Huh, that's strange. I use texconv in my own projects and it outputted `.dtex` and `.dpal` for palettes. The header files also contain 4 bytes with those extension names in them. But I'm now realising my own toolchain sets those names, texconv doesn't really have a standard (Although yes, the readme does use .tex). So I guess you're right, ty for checking.
Moving Day: A clone of Dr Mario with 8-player support <https://dcemulation.org/phpBB/viewtopic ... 4&t=105389>
A recreation of Minesweeper for the Dreamcast <viewtopic.php?f=34&t=104820>

Twitter <https://twitter.com/ProfessorToffal>
YouTube (Not much there, but there are a few things) <https://www.youtube.com/user/TrueMenfa>
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by TapamN »

Protofall wrote: Sat Aug 26, 2023 3:26 amInteresting. I guess tvspelsfreak just didn't test it and accepted the SEGA docs like the rest of us did. Glad you tested it. And yeah, makes sense with that palette+compressed+X-linear bug, since SEGA didn't even intend compressed paletted textures to be usable.
If you don't use mipmaps or filtering at the same time, it's fine, but you'd want both if you're doing 3D.

What happens is the corners of a codebook block can get filtered incorrectly. (See attached screenshots.) The quality on 4bpp mipped VQ is so bad it's already close to unusable on most textures without the bug, but it completely ruins the otherwise promising 8bpp mipped VQ, which gets hit by the bug even worse than 4bpp, since it has smaller blocks.
Protofall wrote: Sat Aug 26, 2023 3:26 amI'm looking at my old notes on Small-VQ and tbh I don't really understand it anymore, haha https://github.com/Protofall/Crayon-Uti ... ression%3F Looks like a way to combine many small textures together in VRAM by abusing codebook's empty/predictable space, right?
That's not how I'm doing it. For small codebooks, I only use the end of the codebook, so the entire texture remains contiguous. Then the texture address passed to the PVR has the unused space subtracted off. So for a texture with a 512 byte codebook, instead of passing the "real" start address to the PVR, you subtract 1536 bytes off it. It's looks like a normal compressed texture to the PVR that just happens to not use the front of the codebook.

I did do something like your link for the PVR accelerated Gens4All. I used a nontwiddled ARGB4444 VQ texture to store the tiles, then selected a different codebook to change palettes. Using the PVR's 4BPP textures would have required twiddling nibbles, but the nontwiddled VQ trick meant that putting Genesis' video RAM into a texture was close to a memcpy.
Attachments
bugged filter 8bpp.png
bugged filter 4bpp.png
User avatar
Protofall
DCEmu Freak
DCEmu Freak
Posts: 78
Joined: Sun Jan 14, 2018 8:03 pm
Location: Emu land
Has thanked: 21 times
Been thanked: 18 times
Contact:

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by Protofall »

TapamN wrote: Sat Aug 26, 2023 4:59 am That's not how I'm doing it. For small codebooks, I only use the end of the codebook, so the entire texture remains contiguous.
Oh. You're a bloody genius mate :lol:
TapamN wrote: Sat Aug 26, 2023 4:59 am I did do something like your link for the PVR accelerated Gens4All. I used a nontwiddled ARGB4444 VQ texture to store the tiles, then selected a different codebook to change palettes. Using the PVR's 4BPP textures would have required twiddling nibbles, but the nontwiddled VQ trick meant that putting Genesis' video RAM into a texture was close to a memcpy.
Fascinating. So if I'm following correctly you had the same compressed texture data, but you just uploaded new codebooks to the same point in memory hence changing the pixel colours and even patterns/entries. Nice
These users thanked the author Protofall for the post:
Ian Robinson
Moving Day: A clone of Dr Mario with 8-player support <https://dcemulation.org/phpBB/viewtopic ... 4&t=105389>
A recreation of Minesweeper for the Dreamcast <viewtopic.php?f=34&t=104820>

Twitter <https://twitter.com/ProfessorToffal>
YouTube (Not much there, but there are a few things) <https://www.youtube.com/user/TrueMenfa>
User avatar
Ian Robinson
DC Developer
DC Developer
Posts: 114
Joined: Mon Mar 11, 2019 7:12 am
Has thanked: 206 times
Been thanked: 41 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by Ian Robinson »

TapamN wrote: Thu May 25, 2023 6:44 am The "session" parameter for the TOC function controls if you are reading a CD TOC (on a 0) or a GD TOC (on a 1). Obviously, CDs don't have a GD area, so requesting the GD TOC will fail. For the end of the last track, could you use the leadout sector value as the loop end? You might have to subtract a bit from it; if you do, I would guess something like 1, 75, 76, 150, or 151 would work. I guess if a dummy track is enough to make GDEMU (or clones) happy, that's fine as a work around.

This doesn't exactly help with what you asked right now, but I've been working on a low CPU usage, seamlessly looping, streaming music system, based on the QOA codec (similar to ADPCM, at about 0.4 bytes per sample), with a replacement CD driver. I tried working on streaming compressed music in KOS before, were I found that KOS had horribly inconsistent CPU usage (anywhere from 0 to 60% usage per frame).

It's currently not complete. I've modified QOA to be better suited for the Dreamcast (changing the format from 64-bit big endian to 32-bit little endian). The reference implementation could only decode the entire file at once, so I've also had to rewrite the decoder to handle streaming.

The new CD driver uses DMA (KOS has the CPU copy the data) and will allow for multiple reads to occur, so that you can stream music while loading something else. I'm not planning on adding any real scheduling to it, just run each request to completion, in order, since that's simple to understand and implement, but with a preemptive priority system. High priority requests (like music) can preempt lower priority requests (like level data) so that the music won't skip. Preempted requests will be resumed from where they left off once the higher priority one fully completes.

The CD driver is only partially working at the moment; I can read raw sectors with it, but don't have the file system working yet.

When everything is finished, I think CPU usage for 44100 hz stereo should be no more than about 3% per frame.
Any news on this i find it very exciting!!.
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by TapamN »

I have the basic CD driver working. You use can use stdio and fs_read to read stuff via DMA, with caching similar to KOS's existing CD cache, and it handles reads from multiple threads at the same time fine (trying that on KOS will hang those threads, though not the entire system). It probably doesn't handle disc changes at the moment, and probably wouldn't work well on modded hardware with additional ATA devices attached, since it currently just takes the G1 lock and never releases it, so there's still some work to do on it before the driver is fully ready.

You won't be able to just replace the existing KOS CD driver with it due to hardware/KOS issues. The main problem is that you can't access the BIOS font during disc DMA (this is what the font mutex syscalls are for), and KOS assumes the font is always available for access. You can't fix this without breaking existing code that uses the font, so the new driver would have to be opt-in.

The biggest hold up has been figuring out a good way to do prefetching for the audio decoder. I want to be able to prefetch the compressed audio so that the decoder will always have enough buffered to play without skipping, even when looping and another thread is also preforming reads (playing music while loading something). I also want the prefetching to support arbitrary seeking (even outside of the buffered data) and be as transparent as possible to the user, so that it would require few or no changes to existing code to use. The first few ideas I had all seemed like they would end up with way too many edge cases that would make writing and debugging a pain, but I finally figured out something that's decent.

I think I'm about 2/3s done to having something working (even if not optimal/finished), but I've been busy with other things, so I don't know how much longer it will take. It's one of my priorities, but not my highest.
These users thanked the author TapamN for the post:
Ian Robinson
User avatar
Quzar
Dream Coder
Dream Coder
Posts: 7497
Joined: Wed Jul 31, 2002 12:14 am
Location: Miami, FL
Has thanked: 4 times
Been thanked: 9 times
Contact:

Re: CDDA Won't Loop Last Track on GDEMU/MODE

Post by Quzar »

TapamN wrote: Fri Feb 23, 2024 5:33 pm You won't be able to just replace the existing KOS CD driver with it due to hardware/KOS issues. The main problem is that you can't access the BIOS font during disc DMA (this is what the font mutex syscalls are for), and KOS assumes the font is always available for access. You can't fix this without breaking existing code that uses the font, so the new driver would have to be opt-in.
Here's a commit against the most recent KOS that adds locking via the font mutex syscalls to the kos bfont system: https://github.com/KallistiOS/KallistiO ... aaeba12708

You should be able to follow the same general pattern used in the g1ata driver to get the cdrom mutex to use and interact with it. If it works out well, then I can see about updating some of the surrounding documentation and the use of the icon data to safeguard that in a similar way.

From some of the old dcdev mailing lists it seems like it may also be required for the flashrom to work correctly:
yamato@20to4.net wrote:btw, the G1 mutex thing is there because when a GD DMA transfer is running,
it is not possible to access the other G1 devices (bios rom / flash rom).
well at least reading will give all 0xFFFF... as far as i remember.
"When you post fewer lines of text than your signature, consider not posting at all." - A Wise Man
Post Reply