pvrtex - PVR Texture Encoder

[TapamN]

After running across a post on using FFmpeg's ELBG for encoding Cinepak (which uses vector quantization), I thought I'd try seeing how well it would work for the Dreamcast. It worked quite well, so I ended up creating a full texture convertor (including uncompressed textures). Source code is attached.

It's designed to work similarly to tvspelsfreak's texconv. I'm not going to repeat the entire contents of the README, but here's the major improvments:

* Faster texture compression and palette generation
* Can generate small codebook VQ textures
* No Qt dependency
* Support for compressed stride textures
* Better mipmap generation
* Dithering
* Support for additional output file types (adds .PVR and .DT)

For what "better mipmaps" means, here's an example. With texconv, the mipmap levels don't line up properly, so the image shifts to the side in smaller mipmap levels. pvrtex corrects for this, and uses a better filter.

Compression is generally much faster using FFmpeg's compressor (but I have found one pathological case where pvrtex is much slower: that test texture above). The results are of similar quality to texconv; you might find one spot in a texture where one compressor does slightly better than the other, but then you'll find a spot where it does slightly worse, and it works out to be about even altogether. Here are some benchmarks versus texconv:

Code: Select all

256x256 RGB565 ($ENCODER -f RGB565 -o result.tex -i tex256x256.png)
  pvrtex 0.02 seconds
  texconv 0.02 seconds

1024x1024 RGB565 Mipped ($ENCODER -f RGB565 -o result.tex -i tex1024x1024.png -m)
  pvrtex 0.80 seconds
  texconv 0.63 seconds

1024x1024 RGB565 Mipped VQ ($ENCODER -f RGB565 -o result.tex -i tex1024x1024.png -m -c)
  pvrtex 1.87 seconds
  texconv 6.75 seconds

1024x1024 ARGB4444 Mipped VQ ($ENCODER -f ARGB4444 -o result.tex -i tex1024x1024.png -m -c)
  pvrtex 1.88 seconds
  texconv 8.53 seconds

1024x1024 PAL8BPP Mipped VQ ($ENCODER -f PAL8BPP -o result.tex -i tex1024x1024.png -m -c -b)
  pvrtex 3.58 seconds
  texconv 11.5 seconds

The new texture format supports compressed texture with small codebooks, which saves RAM on small textures and improves performance a bit. It's also designed to help keep the texture data 32-byte aligned for DMA.

There's still a lot that could be done to improve it, but I think this is enough for now.

Edit: Most recent version (1.01) is in this post

[Ian Robinson]

Thank you so much, TapamN, for sharing this and doing the work. Now I have to learn how to use it and build it into my projects.

[Protofall]

Thanks again TapamN! I'll check this out when I have some time. Btw, do you have a github/gitlab/other? My game framework library auto-downloads its dependencies so it would be nice to add a download for this without the 7-zip requirement.

That said, its not urgent and I'd much prefer to get that cd driver before this small optmisation for my own code

[GyroVorbis]

Oh shit. The original Texture Converter was written by my DC dev mentor, Tvspelsfreak. It's too bad he isn't around anymore, because I think he'd love to hear about this.

Just wanted to also pay my respects and thank you kindly for sharing this with us!

How was getting rid of Qt, btw? I was the one who originally introduced him to Qt, just so he'd get so many source file formats for free.

[TapamN]

Thanks again TapamN! I'll check this out when I have some time. Btw, do you have a github/gitlab/other? My game framework library auto-downloads its dependencies so it would be nice to add a download for this without the 7-zip requirement.

I have SourceForge and GitHub accounts, that I've used for reporting bugs, but I've never uploaded anything to them yet. I'll try to remember to upload it somewhere at some point.

How was getting rid of Qt, btw? I was the one who originally introduced him to Qt, just so he'd get so many source file formats for free.

It's not really getting rid of Qt so much as not using it in the first place. It uses no code at all from texconv. I used stb_image and stb_image_write to handle reading and writing the images, stb_image_resize for mipmaps, and FFmpeg for VQ compression.

[histat]

I tried to build
avconfig.h under libavutil
seems to be missing.
It does not seem to be affected by its .

[Ian Robinson]

I tried to build
avconfig.h under libavutil
seems to be missing.
It does not seem to be affected by its .

You cant build it ?

[TapamN]

I tried to build
avconfig.h under libavutil
seems to be missing.
It does not seem to be affected by its .

Odd, you're right. I'm surprised it compiles on my system. I wonder why? Is it somehow using my system's avconfig.h?

New version with minor fixes:

• Add missing header (I don't think it's used by the compressor, but I want as few changes as possible to any of FFmpeg's files, so that if there are any updates to FFmpeg's compressor, it can be more easily added to pvrtex.)
• Displays error message when there's a problem reading a source image, instead of an assertion.
• Fix -resize down option. It was supposed to only round down non-power-of-two sizes, but would still reduce the size of a texture if it was already a power-of-two.

[histat]

it 's ok.
suggest to add install target

Code: Select all

install: $(TARGET)
	install -s $(TARGET) $(DEST)

Odd, you're right. I'm surprised it compiles on my system. I wonder why? Is it somehow using my system's avconfig.h?

I don't know your environment
but guess
try

Code: Select all

find /opt/local/include  -name avconfig.h

[Ian Robinson]

So do we have examples we can build using this with kos to benchmark the textures compared to the other one ? It would be nice to see the features in build-able kos examples showing the uses of the new features.

[TapamN]

I don't know your environment
but guess
try

Code: Select all

find /opt/local/include  -name avconfig.h

I'm using Xubuntu 20.04. All #includes using avconfig.h are local includes (#include "libavutil/avconfig.h"), so I don't think they should be using the system's headers. When I was first setting up the compressor to run out of FFmpeg, I started with just ELBG's files, and kept copying every missing header over it complained about until it would compile. Why was that single header special and not required?

So do we have examples we can build using this with kos to benchmark the textures compared to the other one ? It would be nice to see the features in build-able kos examples showing the uses of the new features.

I wrote something to view the textures over dcload, but it's very hacked together, has no real UI, and requires recompiling to change most settings, so I didn't include it.

[Ian Robinson]

I don't know your environment
but guess
try

Code: Select all

find /opt/local/include  -name avconfig.h

I'm using Xubuntu 20.04. All #includes using avconfig.h are local includes (#include "libavutil/avconfig.h"), so I don't think they should be using the system's headers. When I was first setting up the compressor to run out of FFmpeg, I started with just ELBG's files, and kept copying every missing header over it complained about until it would compile. Why was that single header special and not required?

So do we have examples we can build using this with kos to benchmark the textures compared to the other one ? It would be nice to see the features in build-able kos examples showing the uses of the new features.

I wrote something to view the textures over dcload, but it's very hacked together, has no real UI, and requires recompiling to change most settings, so I didn't include it.

Yes i have written my own to display code for normal pvr formats. I was looking at your new format and features past what we had before and example with them not the normal pvr textures.

[GyroVorbis]

Jesus, OKAY.

Since I joined the KOS crew, I've been mostly working on language, toolchain, stdlib, kernel, and low-level driver stuff... I'm a bottom-up kind of guy... anyway... I'm working my way back up to graphics and such, and am now playing with the PVR API, GLdc, rendering, textures, etc, and guess what I'm only now trying out...

The fact this even built out-of-the-box for me without a Qt dependency or download is definitely a hell of a plus, but then I see a whole myriad of things yours is doing better?

Apparently Tvspelsfreak's texconv was never even considered for being added to KOS due to the big-ass Qt dependency. I always thought it was a massive shame and that it feels wrong to have a Dreamcast SDK without such a tool built into it... is there any way you would consider letting us put this in the KOS repo? Under utils? I'm sure there will be all kinds of work to ensure the Makefile just builds for all of the platforms we target, but we can handle that... Full credit too, obviously, it's totally your tool... I just feel like the potential good this can do is so great with KOS in terms of user-experience and what people can achieve on DC, that it could really benefit from the distribution channel that is the KOS repo as opposed to just staying a post on these boards.

[GyroVorbis]

Ooops. Double-post. Meant to edit my previous one.

[TapamN]

Apparently Tvspelsfreak's texconv was never even considered for being added to KOS due to the big-ass Qt dependency. I always thought it was a massive shame and that it feels wrong to have a Dreamcast SDK without such a tool built into it... is there any way you would consider letting us put this in the KOS repo? Under utils? I'm sure there will be all kinds of work to ensure the Makefile just builds for all of the platforms we target, but we can handle that... Full credit too, obviously, it's totally your tool... I just feel like the potential good this can do is so great with KOS in terms of user-experience and what people can achieve on DC, that it could really benefit from the distribution channel that is the KOS repo as opposed to just staying a post on these boards.

Go ahead. It might nice to add KMG support to pvrtex, or add DT support to KOS the same way it supports KMG, at some point, but probably not critical.