Loading Media from memory

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Post Reply
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has liked: 0
Been liked: 0
Contact:

Loading Media from memory

Post by JS Lemming » Wed Jul 06, 2005 5:13 pm

Ok, in my upcoming project, I plan to have "GamePacks" which would be 1 file that contains everything from png tilesets to mod music to level data and more. The editor I make would allow people to compile all the files needed by their levels into a GamePack. And the game would load the entire pack into ram all at once.

So the question is, is this feesable(sp?)? Do you think it would be too difficult to rewrite the Loading functions of such things as png and music libs to make them just grab the data that's already in ram instead of from the disk? Is it even possible (not sure if it's possible to edit a lib)?

What do you guys think?
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Wed Jul 06, 2005 5:21 pm

To my understanding, whenever you send something via the coder's cable it's being loaded into the RAM already. How is this any different than what you've been doing?

When you've been using images and such before, they have been from the RAM not anywhere else.
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
User avatar
toastman
Iron Fist of Justice
Iron Fist of Justice
Posts: 4933
Joined: Sat Nov 10, 2001 3:08 am
Location: New Orleans
Has liked: 0
Been liked: 0
Contact:

Re: Loading Media from memory

Post by toastman » Wed Jul 06, 2005 6:25 pm

JS Lemming wrote:Ok, in my upcoming project, I plan to have "GamePacks" which would be 1 file that contains everything from png tilesets to mod music to level data and more. The editor I make would allow people to compile all the files needed by their levels into a GamePack. And the game would load the entire pack into ram all at once.

So the question is, is this feesable(sp?)? Do you think it would be too difficult to rewrite the Loading functions of such things as png and music libs to make them just grab the data that's already in ram instead of from the disk? Is it even possible (not sure if it's possible to edit a lib)?

What do you guys think?
Feasible? It's so feasible there is a cheap and easy way to do it.
Romdisks.
No signature.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Wed Jul 06, 2005 6:27 pm

JSL, you're already acquainted with romdisks. I think you just didn't realize that's what you were doing with them.
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has liked: 0
Been liked: 0

Post by BlackAura » Thu Jul 07, 2005 12:39 am

Basically, you build a romdisk using genromfs. Instead of linking that to the program itself, you can load it from the CD, and then mount it from memory. Feet of Fury does this - it uses gzip compressed ROM disks.
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has liked: 0
Been liked: 0
Contact:

Post by JS Lemming » Thu Jul 07, 2005 2:37 pm

Wow! Why didn't I realize this? But still, in such functions as the loadpng function, it requires a path to that file in the romdisk. I'm a little confused about that and how I would work it to load a file from another file....
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Thu Jul 07, 2005 6:23 pm

BlackAura wrote:Basically, you build a romdisk using genromfs. Instead of linking that to the program itself, you can load it from the CD, and then mount it from memory. Feet of Fury does this - it uses gzip compressed ROM disks.
That's a really clever idea. It'd work well in a side project I'm working on (Samba De Amigo clone) as well as in Null.
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has liked: 0
Been liked: 0

Post by BlackAura » Thu Jul 07, 2005 7:29 pm

I'm a little confused about that and how I would work it to load a file from another file....
The built-in romdisk is basically a file. It's included in the program file as a block of data (so it's sitting there in memory when your program runs), and KOS finds that, and presents it's contents as part of the filesystem.

You can use the same mechanism to load a romdisk from the CD into memory, and attach that to the filesystem. It shows up wherever you tell it to. If you have a romdisk image that contains the file image.png and you mount the romdisk to /mount, then your full path will be /mount/image.png.
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has liked: 0
Been liked: 0
Contact:

Post by JS Lemming » Fri Jul 08, 2005 9:48 am

Sorry for my ignorance, I understand the romdisk deal now, but not the details. Like "/mount", I've never heard of it. I'll try to find some info on this by searching the forum. Thanks.
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has liked: 0
Been liked: 0

Post by BlackAura » Fri Jul 08, 2005 10:29 am

KOS has a filesystem that is (roughly speaking) similar to a Unix system. Probably a little different to what you're used to.

The KOS filesystem is actually a virtual filesystem. That means that the filesystem doesn't literally exist anywhere, at least not in the form that your programs are going to see. It has a single root directory (/). If you try to list the contents of this directory, you'll get a list like:
  • cd
    ram
    rd
    vmu
None of those directories actually exist anywhere. They're mount points.

A mount point is a place in the filesystem (a directory) which you can attach another filesystem to.

Take the /cd directory, for example. If you look in that, you'll see a listing of all the files on the CD. The contents of the /cd directory do actually exist - on the CD. The /ram directory contains a ramdisk (which you can use to store temporary files, which are kept in memory - quite useful), the /rd directory contains the contents of the statically linked romdisk your program uses (if any), and the /vmu directory contains a load of subdirectories, each of which contains files representing the contents of a VMU (even though the VMU files aren't, technically speaking, real files).

Each mount point is handled by a filesystem handler (which is a bit like a filesystem driver in a typical operating system, but a lot simpler). The VFS will automatically call the appropriate drivers, which means that you generally don't need to worry about all the underlying details. You can simply read stuff out of this directory tree, and you don't need to worry about where it came from, or how it got there.

The romdisk filesystem allows you to mount any number of romdisks at a time, each at a different mount point. Once you have the romdisk image stored in memory, you can mount the filesystem using the fs_romdisk_mount function. Tell it the mount point you want, give it a pointer to the romdisk image, and it'll mount it for you.

The built-in romdisk is just a special case. The romdisk is already in memory (it's part of your program file), and KOS automatically mounts it for you.

Anyway, I may as well give you an example. To load a romdisk, and mount it at a given point, you do something like this:

Code: Select all

#include <kos.h>

bool mount_romdisk(char *filename, char *mountpoint)
{
    void *buffer;
    ssize_t size = fs_load(filename, &buffer);

    // Successfully read romdisk image
    if(ssize != -1)
    {
        fs_romdisk_mount(mountpoint, buffer, 1);
        return true;
    }
    else
        return false;
}
That's it, basically. If you have a romdisk containing these files:
  • image.png
    sound.wav
    music.ogg
And mount it to "romdisk", you'll end up with this:
  • /cd
    • (Stuff from CD)
    /rd
    • (Stuff from built-in romdisk)
    /romdisk
    • image.png
      sound.wav
      music.ogg
    /vmu
    • (Stuff from VMU)
So, the filename for image.png inside the romdisk image would be "/romdisk/image.png"

If you want to be really flashy, you can read compressed romdisk images using zlib (make sure you link to zlib using -lz). Same deal as above, but you have to decompress it first. Easiest way is to use zlib's file I/O (note: untested):

Code: Select all

#include <kos.h>
#include <zlib/zlib.h>

...

bool mount_romdisk(char *filename, char *mountpoint)
{
    void *buffer;
    int length = zlib_getlength(filename);

    // Check failure
    if(length == 0)
        return false;

    // Open file
    gzFile file = gzopen(filename, "rb");
    if(!file)
        return false;

    // Allocate memory, read file
    buffer = malloc(length);
    gzread(file, buffer, length);
    gzclose(file);

    // Mount
    fs_romdisk_mount(mountpoint, buffer, 1);
    return true;
}
Since it uses the zlib file input code to read the file, that means that it'll read both uncompressed files, and gzip compressed images.

And how do you compress romdisk images? If you're using Cygwin (or MinGW, or Linux, or any other Unix-y system), it's pretty easy. Right after you create the the romdisk, you run it through gzip. For example, say you have a file named myromdisk.img and you want to compress it:

Code: Select all

gzip -9 myromdisk.img
The result will be a file named myromdisk.img.gz which should be readable by the above function.

Finally, to unmount the romdisk and free the memory it's using, you can do this:

Code: Select all

fs_romdisk_unmount(mountpoint);
User avatar
JS Lemming
Insane DCEmu
Insane DCEmu
Posts: 202
Joined: Mon Apr 04, 2005 8:08 am
Location: C:\CON\CON
Has liked: 0
Been liked: 0
Contact:

Post by JS Lemming » Fri Jul 08, 2005 11:49 am

Wow! I'd be dead without you people! Thanks BlackAura.

Only problem is that I have to wait a week before I can play around with this. Going vacationing.

Thanks again for that lesson there.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Fri Jul 08, 2005 1:36 pm

BA, is /pc automatically added to the KOS filesystem if a BBA or Coder's Cable is connected?
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
User avatar
toastman
Iron Fist of Justice
Iron Fist of Justice
Posts: 4933
Joined: Sat Nov 10, 2001 3:08 am
Location: New Orleans
Has liked: 0
Been liked: 0
Contact:

Post by toastman » Fri Jul 08, 2005 2:39 pm

GyroVorbis wrote:BA, is /pc automatically added to the KOS filesystem if a BBA or Coder's Cable is connected?
Yes, it is.
I now use files from /pc to test with and then change it to /cd, /rd, etc for end releases.
No signature.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Fri Jul 08, 2005 3:03 pm

Yeah, I've been using /pc a bit lately. It's really convenient.
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Mon Jul 11, 2005 1:45 am

This is really wierd. I'm working on a PC browser ATM. I have it up and running (though not very pretty). When I browse the root directory ("/"), I get this:

Code: Select all

rd
ram
pc
cd
vmu
pty
I open pty and I see two files with a filesize of 0:

Code: Select all

s100
ma00
I'm just curious since I have no clue what in the grill pty or its files are there for. Can anybody fill me in?
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has liked: 0
Been liked: 0

Post by BlackAura » Mon Jul 11, 2005 5:04 am

Yet another little thing copied over from Unix-oid systems. Unix-oid systems allow you to access devices (disk drives, sound hardware, terminals, even the system's physical memory) as if they were files. They have special device files associated with them.

Specifically, a pty is a pseudo terminal. A terminal that's associated with a real hardware terminal (like a text-mode display) lets you send stuff to the screen, and get input from the keyboard, which you can do using standard file descriptors. One of these devices is associated with the standard input and output of each program, which is why you can do things like:

Code: Select all

fprintf(stdout, "Hello!");
A pseudo terminal is used when you want a program to simulate a terminal. One program connects to one end of the pty and pretends to be a real terminal, while another program connects to the other and and uses the pty as a terminal, and doesn't know any difference. It's used for things like GUI terminal emulators, SSH (which basically lets you tunnel a terminal across a network), and so on.

KOS ptys are basically the same thing. One part of your program sets itself up as a pty server (for lack of a better description), and another part can then open the pty file and start sending data to it. That data then shows up at the other end of the pty, and you can read it back out.

I think KOS uses it for shoving debug information around, for routing things like printf to the appropriate console over a network, and probably a few other things. About the only other use for it would be if you were porting a command-line app, and wanted to write a virtual terminal so you could use the Dreamcast's video hardware and keyboard to control it.
User avatar
GyroVorbis
Elysian Shadows Developer
Elysian Shadows Developer
Posts: 1808
Joined: Mon Mar 22, 2004 4:55 pm
Location: #%^&*!!!11one Super Sonic
Has liked: 0
Been liked: 0
Contact:

Post by GyroVorbis » Mon Jul 11, 2005 11:02 am

Wow, thanks for the info, BA! That's really neat.
Elysian Shadows - "Next-Gen" 2D/3D RPG coming to Sega Dreamcast, Steam, OUYA, and Smartphones
Image
http://www.elysianshadows.com
Post Reply