New Video Playback Engine For Dreamcast (RoQ)

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Multimedia Mike
DC Developer
DC Developer
Posts: 35
https://www.artistsworkshop.eu/meble-kuchenne-na-wymiar-warszawa-gdzie-zamowic/
Joined: Thu Mar 17, 2011 11:57 pm
Has thanked: 0
Been thanked: 0
Contact:

New Video Playback Engine For Dreamcast (RoQ)

Post by Multimedia Mike »

I see some of you have been following my blog postings about developing decent video playback for the Sega Dreamcast. I have just released a new, KOS-license-compatible playback library for the DC using the old RoQ video format. Code here:

https://github.com/multimediamike/dreamroq

Details here:

http://multimedia.cx/eggs/announcing-dreamroq/
http://multimedia.cx/eggs/roq-on-dreamcast/

It has problems right now but I think it's promising.
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has thanked: 0
Been thanked: 5 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by PH3NOM »

Good news Mike!

Welcome to the forums.

I will look at your work, I have never worked with the RoQ format before, I am curious about the compression/quality of the codec.

Thanks again!
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by BlueCrab »

Good to see that you've gotten something releasable as a first cut. I'll have to take a look at the code at some point. Hopefully someone can find a good use for it! :grin:
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has thanked: 0
Been thanked: 5 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by PH3NOM »

Although looking at the source, I am a little confused.

From the ReadME:
For that matter, the phase which sends the decoded
texture to the graphics hardware via DMA and renders it should also be
in a separate thread.
From your Code:

Code: Select all

    /* send the video frame as a texture over to video RAM */
    pvr_txr_load_ex(buf, textures[current_frame], stride, texture_height,
        PVR_TXRLOAD_16BPP);

    pvr_wait_ready();
    pvr_scene_begin();
    pvr_list_begin(PVR_LIST_OP_POLY);

    pvr_poly_cxt_txr(&cxt, PVR_LIST_OP_POLY, PVR_TXRFMT_RGB565, stride,
      texture_height, textures[current_frame], PVR_FILTER_NONE);
pvr_txr_load_ex() is not using DMA. It think its one of the slowest methods since it also applies Twiddling to the textures.

And if the player is decoding a video that will be scaled using the PVR to fit the screen, the PVR_FILTER does make a noticeable difference. I would change the above code to this:

Code: Select all

    /* send the video frame as a texture over to video RAM */
    pvr_txr_load(buf, textures[current_frame], stride * texture_height * 2);

    pvr_wait_ready();
    pvr_scene_begin();
    pvr_list_begin(PVR_LIST_OP_POLY);

    pvr_poly_cxt_txr(&cxt, PVR_LIST_OP_POLY, PVR_TXRFMT_RGB565 | PVR_TXRFMT_NONTWIDDLED, stride,
      texture_height, textures[current_frame], PVR_FILTER_BILINEAR);
Note the function is still not using DMA, but it should be faster and look better than the previous code.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by BlueCrab »

You could also cache the pvr_poly_hdr_t header you get from pvr_poly_compile to avoid recreating it every frame. You could pretty much just keep two of them around, one for each frame, saving yourself a little bit (although the twiddling thing that PH3NOM mentioned will probably save a lot more than that).
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has thanked: 0
Been thanked: 5 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by PH3NOM »

Haha I tested the code and I did solve your problem Mike.

The speed limitations were not caused by the CD-Rom. They were caused by the twiddling function of pvr_txr_load_ex().

Make the above changes to your code, and it is now as fast as it should be.

If I get some time I will update your player with some enhanced functionality from some code I developed for XviD-DC.
patbier
DC Developer
DC Developer
Posts: 152
Joined: Fri Aug 29, 2003 1:25 am
Has thanked: 0
Been thanked: 0

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by patbier »

Yes, I tested too with the above changes and with sq_copy functions it is now faster than the video length ! Just have a display distorsion with sq_cpy but I guess it would be easy to find the loop problem.

Great job !
ImageAlice Dreams Tournament Dreamcast fans : http://www.facebook.com/alicedreamst
In August 2015, we had to change "Dynamite Dreams" name to "Alice Dreams Tournament"
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by BlueCrab »

patbier wrote:Yes, I tested too with the above changes and with sq_copy functions it is now faster than the video length ! Just have a display distorsion with sq_cpy but I guess it would be easy to find the loop problem.

Great job !
There's really little to no reason to use sq_cpy over pvr_txr_load... After all, here's the entirety of pvr_txr_load:

Code: Select all

/* Load raw texture data from an SH-4 buffer into PVR RAM */
void pvr_txr_load(void * src, pvr_ptr_t dst, uint32 count) {
	if (count % 4)
		count = (count & 0xfffffffc) + 4;
	sq_cpy((uint32 *)dst, (uint32 *)src, count);
}
If you're getting any sort of oddities with sq_cpy (but not with pvr_txr_load), I can only guess its due to the length being not divisible by 4 (although that doesn't make much sense either)...
patbier
DC Developer
DC Developer
Posts: 152
Joined: Fri Aug 29, 2003 1:25 am
Has thanked: 0
Been thanked: 0

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by patbier »

Ok sorry I didn't know about pvr_txr_load
i thought about this post : viewtopic.php?f=29&t=60229
i thought it wasn't the same as pvr_txr_load, but I was wrong
Thanks for the info BlueCrab
ImageAlice Dreams Tournament Dreamcast fans : http://www.facebook.com/alicedreamst
In August 2015, we had to change "Dynamite Dreams" name to "Alice Dreams Tournament"
Multimedia Mike
DC Developer
DC Developer
Posts: 35
Joined: Thu Mar 17, 2011 11:57 pm
Has thanked: 0
Been thanked: 0
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Multimedia Mike »

Thanks for all the interest in this bit of code. A month ago, I would have thought I was the last person on the internet to care about Dreamcast homebrew development.
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has thanked: 0
Been thanked: 5 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by PH3NOM »

I hope this doesn't seem off topic.

Are there any distinct differences on Dreamcast between fopen() and fs_open() for streaming a ~large file?
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by BlueCrab »

fs_open() and its friends are the lowest-level "public" file access functions in KOS. They're the most direct way of dealing with files, and they avoid any sort of buffering that might be done at a higher level (I don't know that newlib does any real buffering though).

In theory, they might be ever so slightly faster (fopen/fread/etc eventually call down to fs_open/fs_read/etc). In practice, there's probably no difference that anyone would notice.
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has thanked: 0
Been thanked: 5 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by PH3NOM »

Using a 512x256 clip, ffmpeg ends up encoding the video @4.5Mbps, which is too fast to be streamed from the /cd/

Attempting to control the bitrate has no effect.

And, even @ 4.5Mbps, the encode is far from lossless.

The compression is not very high, and looks very noticeable.

Not to mention this codec requires an intermediary encoding process, and the final encoder is SLOW. I guess that is OK since this library is likely meant to be hard-coded into a game for FMV or something.

I am starting to think that RoQ may have more limitations...

Mike, if I send you my XviD library, would you have time to look at optimizing the colorspace conversion?
Multimedia Mike
DC Developer
DC Developer
Posts: 35
Joined: Thu Mar 17, 2011 11:57 pm
Has thanked: 0
Been thanked: 0
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Multimedia Mike »

PH3NOM wrote:Using a 512x256 clip, ffmpeg ends up encoding the video @4.5Mbps, which is too fast to be streamed from the /cd/

Attempting to control the bitrate has no effect.
You're right that FFmpeg's bitrate parameter has no effect. However, try -qscale. I've played with it. That can shrink the video a bit.
PH3NOM wrote:Not to mention this codec requires an intermediary encoding process, and the final encoder is SLOW. I guess that is OK since this library is likely meant to be hard-coded into a game for FMV or something.
Yeah, that's the trade-off with vector quantizers like RoQ-- forever to encode, fast to decode. Check out the designer's dev journal, published over 15 years ago:

http://www.wired.com/wired/archive/3.08/shipping.html

He speaks of the super-fast and expensive Pentium boxes they purchased for encoding RoQ video.
PH3NOM wrote:I am starting to think that RoQ may have more limitations...
Try encoding video using the SVQ1 codec in FFmpeg ('ffmpeg -i input.vid -vcodec svq1 output.mov') and see if that can get to acceptable quality within the data rate limitations (and you can test the video in Apple's QuickTime Player). I think SVQ1 is another good candidate for this console.

Then again, I've been analyzing the output of FFmpeg's Switchblade 3 encoder. It's making some odd (apparently suboptimal) decisions. Maybe it can still be improved upon (with SB4, for example).
PH3NOM wrote:Mike, if I send you my XviD library, would you have time to look at optimizing the colorspace conversion?
A few questions:
* Is 512x256 your target resolution? Patbier says he is having success with RoQ on DC @ 320x240.
* Are you sure XviD is actually able to handle decoding 512x256 at whatever framerate on your DC, before the colorspace conversion? That seems like a lot.
* I only have one idea for optimizing the YUV converter, and that's provided that XviD does not do it already.
Multimedia Mike
DC Developer
DC Developer
Posts: 35
Joined: Thu Mar 17, 2011 11:57 pm
Has thanked: 0
Been thanked: 0
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Multimedia Mike »

Just updated the code for Dreamroq at Github:

* I expect the decoding to be correct; I was able to squash the lingering artifact bug.
* Thanks to all the suggestions for making the DC sample player faster. I incorporated some of them.
* Added some other optimizations to the core library.
Chilly Willy
DC Developer
DC Developer
Posts: 414
Joined: Thu Aug 20, 2009 11:00 am
Has thanked: 0
Been thanked: 2 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Chilly Willy »

Is there some way to control the bitrate on ffmpeg when encoding as roq? I've tried virtually every option and nothing changes. It's permanently at q = 0.0.
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has thanked: 0
Been thanked: 5 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by PH3NOM »

Multimedia Mike wrote: Yeah, that's the trade-off with vector quantizers like RoQ-- forever to encode, fast to decode. Check out the designer's dev journal, published over 15 years ago:

http://www.wired.com/wired/archive/3.08/shipping.html

He speaks of the super-fast and expensive Pentium boxes they purchased for encoding RoQ video.
I did read that journal, quite interesting. Made me remember my childhood, life in the mid '90's.
When you consider VHS was the leading format of the time, RoQ compression is not bad, at all.
But we are no longer in the '90's lol.
Multimedia Mike wrote:Try encoding video using the SVQ1 codec in FFmpeg ('ffmpeg -i input.vid -vcodec svq1 output.mov') and see if that can get to acceptable quality within the data rate limitations (and you can test the video in Apple's QuickTime Player). I think SVQ1 is another good candidate for this console.

Then again, I've been analyzing the output of FFmpeg's Switchblade 3 encoder. It's making some odd (apparently suboptimal) decisions. Maybe it can still be improved upon (with SB4, for example).
I tried to use SB4, but it is crashing on my PC. I cant get past the first encode process...

Also, forget QuickTime, I use VLC for general purposes. It supports RoQ BTW.

Attempting some ffmpeg encodes with SVQ1, i do believe it is a better codec to work with, simply due to greater flexibility.
I didn't achieve the desired quality at the target bitrate, but at the very least the target bitrate was attainable. And I am confident that better results could be achieved with better encoder settings.

If you can write/port a SVQ1 decoder for DC, you should. It would be useful for comparison to decide which codec is best, on this platform.
Multimedia Mike wrote:A few questions:
* Is 512x256 your target resolution? Patbier says he is having success with RoQ on DC @ 320x240.
* Are you sure XviD is actually able to handle decoding 512x256 at whatever framerate on your DC, before the colorspace conversion? That seems like a lot.
* I only have one idea for optimizing the YUV converter, and that's provided that XviD does not do it already.
For libXviD, yes, 512x256 is my target resolution. Something like RoQ should be 512x512.
A few days ago I tested some code to make benchmarks at various stages using my current SH4 optimized build of libxvidcore-1.3.0-rc1.
The sample AVI was encoded as DivX@512x256p, 23.976fps, 1Mbps, MP3@48kHz, 2ch, 128kbps
Here is the sample clip I used, from a recent Family Guy episode http://www.megaupload.com/?d=JKY4ZSYQ

52.75fps = read avi packet -> decode video
47.23fps = read avi packet -> decode video -> transfer/draw decoded image
38.30fps = read avi packet -> decode video -> color conversion -> transfer/draw decoded image
30.64fps = read avi packet -> decode video/audio -> color conversion -> transfer/draw decoded image + stream audio
Multimedia Mike
DC Developer
DC Developer
Posts: 35
Joined: Thu Mar 17, 2011 11:57 pm
Has thanked: 0
Been thanked: 0
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Multimedia Mike »

Chilly Willy wrote:Is there some way to control the bitrate on ffmpeg when encoding as roq? I've tried virtually every option and nothing changes. It's permanently at q = 0.0.
Did you try the '-qscale' option?
Multimedia Mike
DC Developer
DC Developer
Posts: 35
Joined: Thu Mar 17, 2011 11:57 pm
Has thanked: 0
Been thanked: 0
Contact:

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Multimedia Mike »

PH3NOM wrote: For libXviD, yes, 512x256 is my target resolution. Something like RoQ should be 512x512.
A few days ago I tested some code to make benchmarks at various stages using my current SH4 optimized build of libxvidcore-1.3.0-rc1.
The sample AVI was encoded as DivX@512x256p, 23.976fps, 1Mbps, MP3@48kHz, 2ch, 128kbps
Here is the sample clip I used, from a recent Family Guy episode http://www.megaupload.com/?d=JKY4ZSYQ

52.75fps = read avi packet -> decode video
47.23fps = read avi packet -> decode video -> transfer/draw decoded image
38.30fps = read avi packet -> decode video -> color conversion -> transfer/draw decoded image
30.64fps = read avi packet -> decode video/audio -> color conversion -> transfer/draw decoded image + stream audio
I'm a bit confused-- this last line implies that this is a solved problem.
Chilly Willy
DC Developer
DC Developer
Posts: 414
Joined: Thu Aug 20, 2009 11:00 am
Has thanked: 0
Been thanked: 2 times

Re: New Video Playback Engine For Dreamcast (RoQ)

Post by Chilly Willy »

Multimedia Mike wrote:
Chilly Willy wrote:Is there some way to control the bitrate on ffmpeg when encoding as roq? I've tried virtually every option and nothing changes. It's permanently at q = 0.0.
Did you try the '-qscale' option?
Yes. The -qscale option works for the svq1 codec, but not the roqvideo codec. Oh, it also only outputs 30 FPS. I can't change that either. I think the ffmpeg encoder is just hardcoded for these things.
Post Reply