Loading Media from memory

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Post Reply
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
https://www.artistsworkshop.eu/meble-kuchenne-na-wymiar-warszawa-gdzie-zamowic/
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has thanked: 0
Been thanked: 0
Contact:

Loading Media from memory

Post by JS Lemming »

Ok, in my upcoming project, I plan to have "GamePacks" which would be 1 file that contains everything from png tilesets to mod music to level data and more. The editor I make would allow people to compile all the files needed by their levels into a GamePack. And the game would load the entire pack into ram all at once.

So the question is, is this feesable(sp?)? Do you think it would be too difficult to rewrite the Loading functions of such things as png and music libs to make them just grab the data that's already in ram instead of from the disk? Is it even possible (not sure if it's possible to edit a lib)?

What do you guys think?
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

To my understanding, whenever you send something via the coder's cable it's being loaded into the RAM already. How is this any different than what you've been doing?

When you've been using images and such before, they have been from the RAM not anywhere else.
User avatar
toastman
Iron Fist of Justice
Iron Fist of Justice
Posts: 4933
Joined: Sat Nov 10, 2001 3:08 am
Location: New Orleans
Has thanked: 0
Been thanked: 0
Contact:

Re: Loading Media from memory

Post by toastman »

JS Lemming wrote:Ok, in my upcoming project, I plan to have "GamePacks" which would be 1 file that contains everything from png tilesets to mod music to level data and more. The editor I make would allow people to compile all the files needed by their levels into a GamePack. And the game would load the entire pack into ram all at once.

So the question is, is this feesable(sp?)? Do you think it would be too difficult to rewrite the Loading functions of such things as png and music libs to make them just grab the data that's already in ram instead of from the disk? Is it even possible (not sure if it's possible to edit a lib)?

What do you guys think?
Feasible? It's so feasible there is a cheap and easy way to do it.
Romdisks.
No signature.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

JSL, you're already acquainted with romdisks. I think you just didn't realize that's what you were doing with them.
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

Basically, you build a romdisk using genromfs. Instead of linking that to the program itself, you can load it from the CD, and then mount it from memory. Feet of Fury does this - it uses gzip compressed ROM disks.
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has thanked: 0
Been thanked: 0
Contact:

Post by JS Lemming »

Wow! Why didn't I realize this? But still, in such functions as the loadpng function, it requires a path to that file in the romdisk. I'm a little confused about that and how I would work it to load a file from another file....
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

BlackAura wrote:Basically, you build a romdisk using genromfs. Instead of linking that to the program itself, you can load it from the CD, and then mount it from memory. Feet of Fury does this - it uses gzip compressed ROM disks.
That's a really clever idea. It'd work well in a side project I'm working on (Samba De Amigo clone) as well as in Null.
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

I'm a little confused about that and how I would work it to load a file from another file....
The built-in romdisk is basically a file. It's included in the program file as a block of data (so it's sitting there in memory when your program runs), and KOS finds that, and presents it's contents as part of the filesystem.

You can use the same mechanism to load a romdisk from the CD into memory, and attach that to the filesystem. It shows up wherever you tell it to. If you have a romdisk image that contains the file image.png and you mount the romdisk to /mount, then your full path will be /mount/image.png.
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has thanked: 0
Been thanked: 0
Contact:

Post by JS Lemming »

Sorry for my ignorance, I understand the romdisk deal now, but not the details. Like "/mount", I've never heard of it. I'll try to find some info on this by searching the forum. Thanks.
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

KOS has a filesystem that is (roughly speaking) similar to a Unix system. Probably a little different to what you're used to.

The KOS filesystem is actually a virtual filesystem. That means that the filesystem doesn't literally exist anywhere, at least not in the form that your programs are going to see. It has a single root directory (/). If you try to list the contents of this directory, you'll get a list like:
  • cd
    ram
    rd
    vmu
None of those directories actually exist anywhere. They're mount points.

A mount point is a place in the filesystem (a directory) which you can attach another filesystem to.

Take the /cd directory, for example. If you look in that, you'll see a listing of all the files on the CD. The contents of the /cd directory do actually exist - on the CD. The /ram directory contains a ramdisk (which you can use to store temporary files, which are kept in memory - quite useful), the /rd directory contains the contents of the statically linked romdisk your program uses (if any), and the /vmu directory contains a load of subdirectories, each of which contains files representing the contents of a VMU (even though the VMU files aren't, technically speaking, real files).

Each mount point is handled by a filesystem handler (which is a bit like a filesystem driver in a typical operating system, but a lot simpler). The VFS will automatically call the appropriate drivers, which means that you generally don't need to worry about all the underlying details. You can simply read stuff out of this directory tree, and you don't need to worry about where it came from, or how it got there.

The romdisk filesystem allows you to mount any number of romdisks at a time, each at a different mount point. Once you have the romdisk image stored in memory, you can mount the filesystem using the fs_romdisk_mount function. Tell it the mount point you want, give it a pointer to the romdisk image, and it'll mount it for you.

The built-in romdisk is just a special case. The romdisk is already in memory (it's part of your program file), and KOS automatically mounts it for you.

Anyway, I may as well give you an example. To load a romdisk, and mount it at a given point, you do something like this:

Code: Select all

#include <kos.h>

bool mount_romdisk(char *filename, char *mountpoint)
{
    void *buffer;
    ssize_t size = fs_load(filename, &buffer);

    // Successfully read romdisk image
    if(ssize != -1)
    {
        fs_romdisk_mount(mountpoint, buffer, 1);
        return true;
    }
    else
        return false;
}
That's it, basically. If you have a romdisk containing these files:
  • image.png
    sound.wav
    music.ogg
And mount it to "romdisk", you'll end up with this:
  • /cd
    • (Stuff from CD)
    /rd
    • (Stuff from built-in romdisk)
    /romdisk
    • image.png
      sound.wav
      music.ogg
    /vmu
    • (Stuff from VMU)
So, the filename for image.png inside the romdisk image would be "/romdisk/image.png"

If you want to be really flashy, you can read compressed romdisk images using zlib (make sure you link to zlib using -lz). Same deal as above, but you have to decompress it first. Easiest way is to use zlib's file I/O (note: untested):

Code: Select all

#include <kos.h>
#include <zlib/zlib.h>

...

bool mount_romdisk(char *filename, char *mountpoint)
{
    void *buffer;
    int length = zlib_getlength(filename);

    // Check failure
    if(length == 0)
        return false;

    // Open file
    gzFile file = gzopen(filename, "rb");
    if(!file)
        return false;

    // Allocate memory, read file
    buffer = malloc(length);
    gzread(file, buffer, length);
    gzclose(file);

    // Mount
    fs_romdisk_mount(mountpoint, buffer, 1);
    return true;
}
Since it uses the zlib file input code to read the file, that means that it'll read both uncompressed files, and gzip compressed images.

And how do you compress romdisk images? If you're using Cygwin (or MinGW, or Linux, or any other Unix-y system), it's pretty easy. Right after you create the the romdisk, you run it through gzip. For example, say you have a file named myromdisk.img and you want to compress it:

Code: Select all

gzip -9 myromdisk.img
The result will be a file named myromdisk.img.gz which should be readable by the above function.

Finally, to unmount the romdisk and free the memory it's using, you can do this:

Code: Select all

fs_romdisk_unmount(mountpoint);
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has thanked: 0
Been thanked: 0
Contact:

Post by JS Lemming »

Wow! I'd be dead without you people! Thanks BlackAura.

Only problem is that I have to wait a week before I can play around with this. Going vacationing.

Thanks again for that lesson there.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

BA, is /pc automatically added to the KOS filesystem if a BBA or Coder's Cable is connected?
User avatar
toastman
Iron Fist of Justice
Iron Fist of Justice
Posts: 4933
Joined: Sat Nov 10, 2001 3:08 am
Location: New Orleans
Has thanked: 0
Been thanked: 0
Contact:

Post by toastman »

GyroVorbis wrote:BA, is /pc automatically added to the KOS filesystem if a BBA or Coder's Cable is connected?
Yes, it is.
I now use files from /pc to test with and then change it to /cd, /rd, etc for end releases.
No signature.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

Yeah, I've been using /pc a bit lately. It's really convenient.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

This is really wierd. I'm working on a PC browser ATM. I have it up and running (though not very pretty). When I browse the root directory ("/"), I get this:

Code: Select all

rd
ram
pc
cd
vmu
pty
I open pty and I see two files with a filesize of 0:

Code: Select all

s100
ma00
I'm just curious since I have no clue what in the grill pty or its files are there for. Can anybody fill me in?
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

Yet another little thing copied over from Unix-oid systems. Unix-oid systems allow you to access devices (disk drives, sound hardware, terminals, even the system's physical memory) as if they were files. They have special device files associated with them.

Specifically, a pty is a pseudo terminal. A terminal that's associated with a real hardware terminal (like a text-mode display) lets you send stuff to the screen, and get input from the keyboard, which you can do using standard file descriptors. One of these devices is associated with the standard input and output of each program, which is why you can do things like:

Code: Select all

fprintf(stdout, "Hello!");
A pseudo terminal is used when you want a program to simulate a terminal. One program connects to one end of the pty and pretends to be a real terminal, while another program connects to the other and and uses the pty as a terminal, and doesn't know any difference. It's used for things like GUI terminal emulators, SSH (which basically lets you tunnel a terminal across a network), and so on.

KOS ptys are basically the same thing. One part of your program sets itself up as a pty server (for lack of a better description), and another part can then open the pty file and start sending data to it. That data then shows up at the other end of the pty, and you can read it back out.

I think KOS uses it for shoving debug information around, for routing things like printf to the appropriate console over a network, and probably a few other things. About the only other use for it would be if you were porting a command-line app, and wanted to write a virtual terminal so you could use the Dreamcast's video hardware and keyboard to control it.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1873
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has thanked: 79 times
Been thanked: 61 times
Contact:

Post by GyroVorbis »

Wow, thanks for the info, BA! That's really neat.
Post Reply