DCEmulation

dreamcast development • homebrew software • hardware hacking • indie games • emulators • and more!
Back to main site
It is currently Sun Sep 21, 2014 11:07 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 56 posts ]  Go to page Previous  1, 2, 3
Author Message
PostPosted: Wed Aug 24, 2011 10:02 pm 
Offline
Insane DCEmu
Insane DCEmu
User avatar

Joined: Wed Mar 10, 2010 12:29 am
Posts: 108
Location: Fort Worth, Texas
Chilly Willy wrote:
There are a number of games I'd like to update on the DC when I get the chance... Doom (my current port uses an older engine) and Duke3D being the two main ones as they're among my favorite games.


Wait, are you saying we might get a port of Duke Nukem 3D to Dreamcast too? I hope I didn't misread that. I know the Dreamcast's 16mb of ram is much less than the 64mb's my PSP 3000 has, and the previous port Bero did was only shareware and had too many problems.

Duke Nukem was one of my favorite games back in the 90's, played so many hours of it I got all Dukes lines memorized.

I do wish you the best of luck if that's the case, cause if anybody could port that game over and have it working, I think it would be someone like you.


Top
 Profile  
 
PostPosted: Sat Aug 27, 2011 3:11 pm 
Offline
DC Developer
DC Developer

Joined: Thu Aug 20, 2009 9:00 am
Posts: 316
Anthony817 wrote:
Chilly Willy wrote:
There are a number of games I'd like to update on the DC when I get the chance... Doom (my current port uses an older engine) and Duke3D being the two main ones as they're among my favorite games.


Wait, are you saying we might get a port of Duke Nukem 3D to Dreamcast too? I hope I didn't misread that. I know the Dreamcast's 16mb of ram is much less than the 64mb's my PSP 3000 has, and the previous port Bero did was only shareware and had too many problems.

Duke Nukem was one of my favorite games back in the 90's, played so many hours of it I got all Dukes lines memorized.

I do wish you the best of luck if that's the case, cause if anybody could port that game over and have it working, I think it would be someone like you.


Yes, DN3D is one of the things I'll work on. The PSP port works fine on the PSP1000, and that only has maybe ~22 MBytes of free ram for the game. KOS is pretty lean, and (stock) DN3D can get by in as little as 8 MB, so I don't think it will be too much trouble. It's not like trying to squeeze it onto a stock PS1 or Saturn with only 2 MB. :grin:

The old JDuke, which was the source of a lot of ports, had lots of problems that did eventually get sorted out. I'm not sure what the DC SW port was based on, but if it had troubles, it was probably an older version of JDuke.


Top
 Profile  
 
PostPosted: Wed Aug 31, 2011 10:50 pm 
Offline
Insane DCEmu
Insane DCEmu
User avatar

Joined: Wed Mar 10, 2010 12:29 am
Posts: 108
Location: Fort Worth, Texas
Awesome news bro, looks like it is totally a possibility then! :mrgreen: Again, good luck! 8-)


Top
 Profile  
 
PostPosted: Thu Sep 01, 2011 5:17 am 
Offline
DCEmu Newbie
DCEmu Newbie

Joined: Thu Sep 01, 2011 5:07 am
Posts: 3
Say, this is just an idea, but isn't it possible someone could make an expansion module for the Dreamcast that will also have a port on it for the Modem? What I mean by this is that it is a RAM expander, and also has an area on it to plug-in the Modem on it.
I know is sounds hard, but it might just work :D


Top
 Profile  
 
PostPosted: Thu Sep 01, 2011 4:34 pm 
Offline
Insane DCEmu
Insane DCEmu
User avatar

Joined: Wed Mar 10, 2010 12:29 am
Posts: 108
Location: Fort Worth, Texas
You must mean something like the Saturn and N64 have? I don't think it would work out, some suggested using the SD cards as ram for some games, but it was deemed to difficult to implement.


Top
 Profile  
 
PostPosted: Thu Sep 01, 2011 9:56 pm 
Offline
DC Developer
DC Developer

Joined: Thu Aug 20, 2009 9:00 am
Posts: 316
Anthony817 wrote:
You must mean something like the Saturn and N64 have? I don't think it would work out, some suggested using the SD cards as ram for some games, but it was deemed to difficult to implement.


Not so much difficult as too slow. For memory swapping on the storage device, said device must be FAST. An SD card connected to the serial port on the DC is VERY slow.


Top
 Profile  
 
PostPosted: Fri Sep 02, 2011 1:45 am 
Offline
Insane DCEmu
Insane DCEmu

Joined: Thu Apr 03, 2008 5:01 am
Posts: 127
The linux kernel has the new "zRAM" module, which allows to dynamically compress the long-time unused RAM pages to save up space. Maybe we could implement something like that on KOS?


Top
 Profile  
 
PostPosted: Fri Sep 02, 2011 12:17 pm 
Offline
DC Developer
DC Developer

Joined: Thu Aug 20, 2009 9:00 am
Posts: 316
Ayla wrote:
The linux kernel has the new "zRAM" module, which allows to dynamically compress the long-time unused RAM pages to save up space. Maybe we could implement something like that on KOS?


That sort of thing is good for a desktop where you'll be using a dozen different things, some of which are idle while you focus elsewhere. It's not so good on a console where you are only doing one thing.


Top
 Profile  
 
PostPosted: Fri Sep 02, 2011 2:56 pm 
Offline
Insane DCEmu
Insane DCEmu

Joined: Thu Apr 03, 2008 5:01 am
Posts: 127
That is not true; we are currently using it on a linux-based chinese handheld console, and it works great.


Top
 Profile  
 
PostPosted: Sat Sep 03, 2011 9:55 pm 
Offline
DC Developer
DC Developer

Joined: Thu Aug 20, 2009 9:00 am
Posts: 316
Ayla wrote:
That is not true; we are currently using it on a linux-based chinese handheld console, and it works great.


Current ARM based handhelds are not really consoles. Not in the usual sense of the word. It's a handheld computer. :grin:

Tell me zRAM is going to work great on an N64. :lol:


Top
 Profile  
 
PostPosted: Mon Sep 05, 2011 9:02 pm 
Offline
DC Developer
DC Developer

Joined: Sun Dec 30, 2001 8:02 am
Posts: 9951
Ayla wrote:
That is not true; we are currently using it on a linux-based chinese handheld console, and it works great.


(Whoops... Long post. Summary - too hard, no benefit. There are much more effective ways to do this on a console like the Dreamcast, games don't really benefit from things like zRAM anyway, and it's much better suited for Live CDs.)

It needs a proper operating system to work. Linux, for example. KOS doesn't have the required bits and pieces (like MMU support, virtual memory, and swapping) to build something like this.

By itself, MMU support and virtual memory is nearly useless on a system like the Dreamcast. Which is why KOS doesn't use it. You have no writable disk, so you can't page unused memory out. You can't really even do things like mapping a file into a program's address space, because the read-only disc you have (a CD drive, basically) is slow, and has a very high seek latency. So, under memory pressure, the CD drive would start to go nuts, the program would slow to a crawl, and would probably take a long time to recover. Hardly any better than just crashing because it ran out of memory.

Plus, you get extra runtime overhead from TLB misses, and having to manage a virtual address space, and it makes it that much more difficult to deal directly with the hardware (for example, you can't use DMA to transfer a buffer over to VRAM if you don't know where the buffer is in physical address space, and you have no guarantee that buffer is contiguous).

Arguably, it might give you a way around memory fragmentation. In practice, it won't - you're more likely to get some degree of heap fragmentation, which will cause you to run out of physical memory anyway. Unless you very carefully manage memory yourself. Which you can do just as easily without an MMU.

So, even if you implemented MMU support in KOS (which is a lot of work, and most of KOS itself would need to be heavily modified), you wouldn't be able to use it for anything. Except, perhaps, something like zRAM.

The problem with zRAM is that it actually needs more memory. You have to have some free memory to compress unused pages into. Which means you need some way to evict pages from memory. So your application has to be using memory mapped files for nearly all read-only data, and code. Since bits of your code have been evicted from memory, you'll end up with the drive thrashing like mad, and everything will slow down to a crawl. Again.

You really don't need any of this stuff on a console like the Dreamcast. All this memory management stuff can be done more efficiently from the game you're running on it. After all, the game knows a lot more about what data it needs than the operating system ever could.

And you can be far more clever with it.

Say, for example, you need to access a model file. And let's assume you have zRAM, and the model's data is sitting in the compressed swap area. Your game tries to access the model, but it can't know that the model isn't there. The OS notices the invalid memory access, and switches to kernel mode, finds the compressed data, allocates some memory for it (which will probably involve either compressing a different memory page, or evicting some of your read-only code or data), decompresses it, and returns back to your game, which carries on as if nothing's happened. Except a huge chunk of CPU time has been used up, and all the caches have been wiped out. If you do this too often (I'm thinking once or twice per frame, maybe), you can kiss any hopes of getting 60FPS goodbye.

Worse - you try to access a model file, and it's actually on the CD. Now, your whole game stalls for ~300ms, while the CD drive grabs the data off the disc. You'd be lucky to get 3FPS like that.

A better approach is to have the game itself control this stuff. Your game needs the model file. It checks, and finds out that it hasn't been loaded yet (or maybe it's been evicted because you've not used it in a while). It sticks the file in a queue of files to load (and if the CD drive isn't busy, starts it reading), and skips over it. The game runs as normal, but the model won't show up for ~300ms or so. When the file's been loaded, it'll be immediately usable, and the game will be able to use it next frame. Better still - your game might know that, in a second or so, you're going to need that model, so you stick it into the queue before you need it. Then, everything just magically works - no loading times, no pauses, no sudden pop-ins.

You can also apply this to textures and sounds, which reside in separate memory on the Dreamcast, and couldn't be managed by an OS even if you wanted to.

Granted, it's much more difficult to actually do that than it would be to let the OS handle everything for you. It also works a hell of a lot better on consoles.

If you have something like an SD card (a fast SD card, not one hooked up using a serial port), the whole equation changes completely. Virtual memory suddenly makes sense. The best way to load (most) resources and code is probably to just map them into a virtual address space, and let the operating system manage it. As an added bonus, initialization code or data that's only used at start-up will eventually get kicked out of main memory, and you won't notice.

You'd still really want to explicitly load things like textures or sounds, but the streaming approach I mentioned earlier is actually even better if you have an SD card, because it'll likely be much faster than a CD drive could ever be.

That's the approach you'd use on something like an iPhone. Or anything powerful enough to run a real operating system, with low-latency flash for storage.

However, streaming + mmap eliminates the need for zRAM (at least, for games). The only stuff you'd ever want to compress is dynamic data, that can't be re-loaded off the SD card by the operating system. In most games, most of the data is static, and could be re-loaded off the disc as needed. The little dynamic data you do need is likely to be needed every single frame, so there's no advantage whatsoever in swapping it out, or compressing it. Everything would slow to a crawl, instead of just crashing.

There are exceptions - this only really works for games. Applications (which we're unlikely to use on a Dreamcast) tend to consist almost entirely of dynamic data. zRAM would probably be useful there (especially if you're running an entire operating system like Linux at the same time). Emulators are almost entirely dynamic data (RAM, internal state, buffers), and ROMs are effectively opaque so there's no way to intelligently load parts of them - you need the whole thing in memory, or performance will tank. Neither MMUs nor zRAM would help.

By the way - the approach I described above for the Dreamcast is actually very common on all kinds of consoles. Systems with limited memory and only an optical disc drive (PS1, Dreamcast, PS2, GameCube, PSP, Xbox 360), and systems with limited memory and a slow cartridge (N64, DS). Even systems with a hard drive (Xbox, PS3, Xbox 360) ultimately have to run off an optical disc, and there's a limited amount of hard drive space (which might be optional, full up, or temporary) which you can really only use as a cache. None of these have much in the way of an operating system either, and where they have an MMU, it's purely for security reasons - you still have to manage memory yourself, and you have no virtual memory or paging to help you out. Streaming's the best bet. Careful memory allocation is needed as well.

Smartphones, tablets, and most of these ARM-based handhelds (real OS, limited memory, fast flash-based storage) are usually better off just using virtual memory to map static data and code, let the OS handle that, keep dynamic data as small as possible, and stream textures and sounds manually.

PCs are... weird. They're really the only place I can think of where zRAM would be useful - running an entire operating system off a read-only optical disk. Lots of dynamic data that isn't being used right now, no writable storage, and that memory would be better used for caching things on the optical disc. That's what zRAM was designed for - Linux Live CDs.

Chilly Willy wrote:
The old JDuke, which was the source of a lot of ports, had lots of problems that did eventually get sorted out. I'm not sure what the DC SW port was based on, but if it had troubles, it was probably an older version of JDuke.


From the readme, it appears to be based on the icculus.org Linux port. Which I seem to remember being very buggy, and just barely enough to get DN3D to run (assembly code replaced with C, platform-specific stuff isolated and replaced with SDL, made compatible with shareware, standard, and atomic edition).

Apparently, Bero's port had some kind of memory restriction, which prevented the full game from working. Then again, it was done using SDL (which is hardly efficient at the best of times), and I think it was abandoned as soon as it was released, so I doubt he put any effort at all into making it use less memory.


Top
 Profile  
 
PostPosted: Tue Sep 06, 2011 3:50 am 
Offline
Insane DCEmu
Insane DCEmu
User avatar

Joined: Wed Mar 10, 2010 12:29 am
Posts: 108
Location: Fort Worth, Texas
BlackAura wrote:
Chilly Willy wrote:
The old JDuke, which was the source of a lot of ports, had lots of problems that did eventually get sorted out. I'm not sure what the DC SW port was based on, but if it had troubles, it was probably an older version of JDuke.


From the readme, it appears to be based on the icculus.org Linux port. Which I seem to remember being very buggy, and just barely enough to get DN3D to run (assembly code replaced with C, platform-specific stuff isolated and replaced with SDL, made compatible with shareware, standard, and atomic edition).

Apparently, Bero's port had some kind of memory restriction, which prevented the full game from working. Then again, it was done using SDL (which is hardly efficient at the best of times), and I think it was abandoned as soon as it was released, so I doubt he put any effort at all into making it use less memory.


Well that gives me hope that we can finally have a perfect port on the Dreamcast. Hearing how buggy the port was from the get go, I can see why it barely works at all. Hell, even with the mouse it can go haywire and move in opposite directions from what you are telling the mouse on Dreamcast.


Top
 Profile  
 
PostPosted: Tue Sep 06, 2011 6:03 pm 
Offline
DC Developer
DC Developer

Joined: Thu Aug 20, 2009 9:00 am
Posts: 316
I've been looking at EDuke32... it's clearly better, but it's also geared towards a modern PC with near infinite CPU/GPU power and ram. JDuke is probably a better match for the DC.


Top
 Profile  
 
PostPosted: Sun Jan 29, 2012 7:24 pm 
Offline
Insane DCEmu
Insane DCEmu
User avatar

Joined: Wed Mar 10, 2010 12:29 am
Posts: 108
Location: Fort Worth, Texas
Any news on this Chilly? What about the Duke port too?


Top
 Profile  
 
PostPosted: Sun Feb 12, 2012 9:40 am 
Offline
DCEmu Newbie
DCEmu Newbie
User avatar

Joined: Sun Jan 22, 2012 11:33 am
Posts: 6
O.O very very good :D


Top
 Profile  
 
PostPosted: Mon Feb 11, 2013 6:54 pm 
Offline
Insane DCEmu
Insane DCEmu
User avatar

Joined: Wed Mar 10, 2010 12:29 am
Posts: 108
Location: Fort Worth, Texas
Big bump. Come on lol, please don't let this project fall by the wayside. Been hoping to see something come from this project for years now. We are still interested in it and I am here to show my support. :mrgreen:


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 56 posts ]  Go to page Previous  1, 2, 3

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group