Duke3d porting team

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
BlackAura
DC Developer
DC Developer
Posts: 9951
https://www.artistsworkshop.eu/meble-kuchenne-na-wymiar-warszawa-gdzie-zamowic/
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

Poor implementation thanks to the Watcom compiler (again) -

Code: Select all

INLINE int moddiv (int i1, int i2)
{
	dmval = i1 % i2;
	return i1 / i2;
}

INLINE unsigned int umin(unsigned int i1, unsigned int i2)
{
	if(i1 < i2)
		return i1;
	return i2;
}

INLINE unsigned int umax(unsigned int i1, unsigned int i2)
{
	if(i1 > i2)
		return i1;
	return i2;
}

INLINE int kmin(int i1, int i2)
{
	if(i1 < i2)
		return i1;
	return i2;
}

INLINE int kmax(int i1, int i2)
{
	if(i1 > i2)
		return i1;
	return i2;
}
That lot should work. Now to go away and test it.

I should only have to boot up Win95 about ten times before it finally gets the idea and doesn't come up with a Windows Protection Error. :x

Now I remember why I said I'd never go back to using Win9x
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

Well, it actually booted first time! Wow... That's never happened before.

Code: Select all

#pragma aux boundmulscale =\
	"imul ebx",\
	"mov ebx, edx",\
	"shrd eax, edx, cl",\
	"sar edx, cl",\
	"xor edx, eax",\
	"js checkit",\
	"xor edx, eax",\
	"jz skipboundit",\
	"cmp edx, 0xffffffff",\
	"je skipboundit",\
	"checkit:",\
	"mov eax, ebx",\
	"sar eax, 31",\
	"xor eax, 0x7fffffff",\
	"skipboundit:",\
	parm nomemory [eax][ebx][ecx]\
	modify exact [eax ebx edx]\

Code: Select all

INLINE int mulscale (int i1, int i2, short i3)
{
  unsigned int retval;
  __int64 scratch1 = (__int64) i1 * (__int64) i2;
  retval = low32 ((scratch1>>i3));
  return(retval);
}

INLINE int boundmulscale(int i1, int i2, int i3)
{
	register long long retval = mulscale(i1, i2, i3);
	if(retval > 0x7FFFFFFF)
		retval = 0x7FFFFFFF;
	if(retval < (int)0x80000000)
		retval = (int)0x80000000;
	return retval;
}
Not totally sure about this one. It seems to run fine with the C version, but I'm not totally sure that it's correct.

Cool, it all seems to work. That's pretty much every non-DOS related function from pragmas.h converted to C.

Now there's just the inline asm from all the other files to look at.
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

Code: Select all

INLINE int moddiv (int i1, int i2)
{
	dmval = i1 % i2;
	return i1 / i2;
}
The asm code uses div as opposed to idiv, so the division is unsigned.
BlackAura wrote:Now I remember why I said I'd never go back to using Win9x
I don't want to go back to using any version of windows. ;)
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

BlackAura wrote:<code snipped>Not totally sure about this one. It seems to run fine with the C version, but I'm not totally sure that it's correct.
EDIT: I think the only thing that's wrong with that code is that it should check for the overflow in the 'original' 64-bit value. The long long that's used is a converted integer, and will never exceed maxint or minint.

Btw, when you said:
BlackAura wrote:Poor implementation thanks to the Watcom compiler (again) -
.. what exactly were you referring to? :)
Last edited by Phantom on Sat Apr 12, 2003 8:47 pm, edited 3 times in total.
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

*bump* (I edited my last post ;)).
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

The compiler seems to be doing weird things when I try code that works in MSVC/GCC. So I'm doing it as inline functions, and trying to be very explicit, so that it works.

It all seems to be working though.

Code: Select all

INLINE unsigned int moddiv (unsigned int i1, unsigned int i2) 
{ 
   dmval = i1 % i2; 
   return i1 / i2; 
}
That should be more correct?
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

BlackAura wrote:

Code: Select all

INLINE unsigned int moddiv (unsigned int i1, unsigned int i2) 
{ 
   dmval = i1 % i2; 
   return i1 / i2; 
}
That should be more correct?
Yeah. It probably won't matter in this case, but it can't hurt to be careful with these things. ;)

What about boundmulscale() though? I think your implementation above doesn't 'bound'. ;) mulscale() returns an integer, so that value can never be greater than maxint (0x7fffffff) or smaller than minint (0x80000000). So the information on whether or not bounding is neccesary is lost at that point and bounding never happens. It simply overflows.

Btw, are you using OpenWatcom or one of the commercial Watcom compilers?
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

What about boundmulscale() though? I think your implementation above doesn't 'bound'. mulscale() returns an integer, so that value can never be greater than maxint (0x7fffffff) or smaller than minint (0x80000000). So the information on whether or not bounding is neccesary is lost at that point and bounding never happens. It simply overflows.
Hmm... Good point. Another random shortcut which doesn't really make any sense.

Code: Select all

INLINE int boundmulscale(int i1, int i2, int i3)
{
	register long long retval = (((__int64) i1 * (__int64) i2) >> i3);
	if(retval > 0x7FFFFFFF) 
		retval = 0x7FFFFFFF; 
	if(retval < (int)0x80000000) 
		retval = (int)0x80000000; 
	return retval;
}
Note to self - Do not take shortcuts.

I've been using Watcom 11.0c, I think. Not sure which type that is, since there was another version on the OpenWatcom site called "OpenWatcom". I think it might be a stripped version of one of the commercial compilers.

It's the one called "Watcom C/C++ 11.0c Binary Patch", which is a variant of the commercial compilers, and isn't OpenWatcom.
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

BlackAura wrote:The compiler seems to be doing weird things when I try code that works in MSVC/GCC. So I'm doing it as inline functions, and trying to be very explicit, so that it works.
Interesting, I'd like to see some examples of that. But nevermind that, it would just be a waste of your time. ;) I haven't seen any weird things in the code you pasted so far. For instance, the typecasting to __int64 is 'normal'. With gcc you'd also have to typecast at least one of the arguments of the multiplication to 'long long' in order to get a 64-bit result.
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

64-bit arithmetic is rather weird.

It probably doesn't help that I can't understand what much of the Build engine's code is actually supposed to be doing. It's just so... weird. I guess that's because it was designed to run on older systems. It uses fixed-point math for everything, all sorts of funky bitshifts, and horrible code layout. It also doesn't help that there are four files with 8000+ lines of code each, and they seem to contain functions that don't relate to the other functions in the program.

The Duke3D game code is a little better, but the engine code... Ugh... I thought that the Doom engine was weird, but this is far worse. No comments, global variables everywhere, multiple statements per line, pointless use of inline assembly, a memory management system which doesn't actually appear to do anything, a compression library that isn't used.

And this is a nice bit to put in a header file:

Code: Select all

#ifdef ENGINE
#define EXTERN
#else
#define EXTERN extern
#endif

EXTERN variablename1;
EXTERN variablename2;
EXTERN variablename3;
...
EXTERN variablename600;
Anyway, rant over. Ken wasn't much younger than I am when he wrote this (I think about a year, maybe two), and my code from even one year ago seems poor compared to my current code.
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

BlackAura wrote:64-bit arithmetic is rather weird.
Yeah, because 32-bit cpu's like the x86 or sh4 cannot handle it. The use of 'long long' will probable generate very inefficient code.
BlackAura wrote:Anyway, rant over.
:) Anyway, let me know if I could help. Although I could only do low-level stuff etc. I know absolutely nothing about 3D. ;)
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

You could try to work out what the heck Ken's memory allocation system is supposed to be doing. It's in the file CACHE1D.C, and I really can't make any sense out of most of it. I wanted to try to do a slightly cleaner re-implementation of some (most?) of Ken's engine code, since it might make porting it somewhat easier, especially if I'm going to try to do an OpenGL rendering engine.

initcache and allocache are obvious, but the rest aren't so obvious. uninitcache isn't anywhere in the source code, nor is it used by Build or Duke3D. agecache is called periodically, but it doesn't seem to do accomplish anything.

The rest of the file CACHE1D.C is quite easy. The GRP file loader and filesystem layer is pretty simple, and I've build code like that many times before. One weird bit is LZW compression functions at the bottom. Unless I'm mistaken, isn't that patented by Unisys? I don't think software patents apply in Australia, but I know they do in the US. Might be worth replacing that with Zlib-based functions, since it only seems to be used for save games.

Code: Select all

// "Build Engine & Tools" Copyright (c) 1993-1997 Ken Silverman
// Ken Silverman's official web site: "http://www.advsys.net/ken"
// See the included license file "BUILDLIC.TXT" for license info.

#include <dos.h>
#include <stdio.h>
#include "pragmas.h"

//   This module keeps track of a standard linear cacheing system.
//   To use this module, here's all you need to do:
//
//   Step 1: Allocate a nice BIG buffer, like from 1MB-4MB and
//           Call initcache(long cachestart, long cachesize) where
//
//              cachestart = (long)(pointer to start of BIG buffer)
//              cachesize = length of BIG buffer
//
//   Step 2: Call allocache(long *bufptr, long bufsiz, char *lockptr)
//              whenever you need to allocate a buffer, where:
//
//              *bufptr = pointer to 4-byte pointer to buffer
//                 Confused?  Using this method, cache2d can remove
//                 previously allocated things from the cache safely by
//                 setting the 4-byte pointer to 0.
//              bufsiz = number of bytes to allocate
//              *lockptr = pointer to locking char which tells whether
//                 the region can be removed or not.  If *lockptr = 0 then
//                 the region is not locked else its locked.
//
//   Step 3: If you need to remove everything from the cache, or every
//           unlocked item from the cache, you can call uninitcache();
//              Call uninitcache(0) to remove all unlocked items, or
//              Call uninitcache(1) to remove everything.
//           After calling uninitcache, it is still ok to call allocache
//           without first calling initcache.

#define MAXCACHEOBJECTS 9216

static long cachesize = 0;
long cachecount = 0;
char zerochar = 0;
long cachestart = 0, cacnum = 0, agecount = 0;
typedef struct { long *hand, leng; char *lock; } cactype;
cactype cac[MAXCACHEOBJECTS];
static long lockrecip[200];

initcache(long dacachestart, long dacachesize)
{
	long i;

	for(i=1;i<200;i++) lockrecip[i] = (1<<28)/(200-i);

	cachestart = dacachestart;
	cachesize = dacachesize;

	cac[0].leng = cachesize;
	cac[0].lock = &zerochar;
	cacnum = 1;
}

allocache (long *newhandle, long newbytes, char *newlockptr)
{
	long i, j, z, zz, bestz, daval, bestval, besto, o1, o2, sucklen, suckz;

	newbytes = ((newbytes+15)&0xfffffff0);

	if ((unsigned)newbytes > (unsigned)cachesize)
	{
		printf("Cachesize: %ld\n",cachesize);
		printf("*Newhandle: 0x%x, Newbytes: %ld, *Newlock: %d\n",newhandle,newbytes,*newlockptr);
		reportandexit("BUFFER TOO BIG TO FIT IN CACHE!");
	}

	if (*newlockptr == 0)
	{
		reportandexit("ALLOCACHE CALLED WITH LOCK OF 0!");
	}

		//Find best place
	bestval = 0x7fffffff; o1 = cachesize;
	for(z=cacnum-1;z>=0;z--)
	{
		o1 -= cac[z].leng;
		o2 = o1+newbytes; if (o2 > cachesize) continue;

		daval = 0;
		for(i=o1,zz=z;i<o2;i+=cac[zz++].leng)
		{
			if (*cac[zz].lock == 0) continue;
			if (*cac[zz].lock >= 200) { daval = 0x7fffffff; break; }
			daval += mulscale32(cac[zz].leng+65536,lockrecip[*cac[zz].lock]);
			if (daval >= bestval) break;
		}
		if (daval < bestval)
		{
			bestval = daval; besto = o1; bestz = z;
			if (bestval == 0) break;
		}
	}

	//printf("%ld %ld %ld\n",besto,newbytes,*newlockptr);

	if (bestval == 0x7fffffff)
		reportandexit("CACHE SPACE ALL LOCKED UP!");

		//Suck things out
	for(sucklen=-newbytes,suckz=bestz;sucklen<0;sucklen+=cac[suckz++].leng)
		if (*cac[suckz].lock) *cac[suckz].hand = 0;

		//Remove all blocks except 1
	suckz -= (bestz+1); cacnum -= suckz;
	copybufbyte(&cac[bestz+suckz],&cac[bestz],(cacnum-bestz)*sizeof(cactype));
	cac[bestz].hand = newhandle; *newhandle = cachestart+besto;
	cac[bestz].leng = newbytes;
	cac[bestz].lock = newlockptr;
	cachecount++;

		//Add new empty block if necessary
	if (sucklen <= 0) return;

	bestz++;
	if (bestz == cacnum)
	{
		cacnum++; if (cacnum > MAXCACHEOBJECTS) reportandexit("Too many objects in cache! (cacnum > MAXCACHEOBJECTS)");
		cac[bestz].leng = sucklen;
		cac[bestz].lock = &zerochar;
		return;
	}

	if (*cac[bestz].lock == 0) { cac[bestz].leng += sucklen; return; }

	cacnum++; if (cacnum > MAXCACHEOBJECTS) reportandexit("Too many objects in cache! (cacnum > MAXCACHEOBJECTS)");
	for(z=cacnum-1;z>bestz;z--) cac[z] = cac[z-1];
	cac[bestz].leng = sucklen;
	cac[bestz].lock = &zerochar;
}

suckcache (long *suckptr)
{
	long i;

		//Can't exit early, because invalid pointer might be same even though lock = 0
	for(i=0;i<cacnum;i++)
		if ((long)(*cac[i].hand) == (long)suckptr)
		{
			if (*cac[i].lock) *cac[i].hand = 0;
			cac[i].lock = &zerochar;
			cac[i].hand = 0;

				//Combine empty blocks
			if ((i > 0) && (*cac[i-1].lock == 0))
			{
				cac[i-1].leng += cac[i].leng;
				cacnum--; copybuf(&cac[i+1],&cac[i],(cacnum-i)*sizeof(cactype));
			}
			else if ((i < cacnum-1) && (*cac[i+1].lock == 0))
			{
				cac[i+1].leng += cac[i].leng;
				cacnum--; copybuf(&cac[i+1],&cac[i],(cacnum-i)*sizeof(cactype));
			}
		}
}

agecache()
{
	long cnt;
	char ch;

	if (agecount >= cacnum) agecount = cacnum-1;
	for(cnt=(cacnum>>4);cnt>=0;cnt--)
	{
		ch = (*cac[agecount].lock);
		if (((ch-2)&255) < 198)
			(*cac[agecount].lock) = ch-1;

		agecount--; if (agecount < 0) agecount = cacnum-1;
	}
}

reportandexit(char *errormessage)
{
	long i, j;

	setvmode(0x3);
	j = 0;
	for(i=0;i<cacnum;i++)
	{
		printf("%ld- ",i);
		printf("ptr: 0x%x, ",*cac[i].hand);
		printf("leng: %ld, ",cac[i].leng);
		printf("lock: %ld\n",*cac[i].lock);
		j += cac[i].leng;
	}
	printf("Cachesize = %ld\n",cachesize);
	printf("Cacnum = %ld\n",cacnum);
	printf("Cache length sum = %ld\n",j);
	printf("ERROR: %s",errormessage);
	exit(0);
}
Phantom
DC Developer
DC Developer
Posts: 1753
Joined: Thu Jan 16, 2003 4:01 am
Location: The Netherlands
Has thanked: 0
Been thanked: 0
Contact:

Post by Phantom »

BlackAura wrote:You could try to work out what the heck Ken's memory allocation system is supposed to be doing. It's in the file CACHE1D.C, and I really can't make any sense out of most of it. I wanted to try to do a slightly cleaner re-implementation of some (most?) of Ken's engine code, since it might make porting it somewhat easier, especially if I'm going to try to do an OpenGL rendering engine.
The code looks pretty portable already. Why not just leave it the way it is? :D

(The rest of this post is slightly off-topic)

The line below is a good example of where macro's without parenthases around the dummies can be dangerous.

Code: Select all

daval += mulscale32(cac[zz].leng+65536,lockrecip[*cac[zz].lock]);
For example, if you would define:

Code: Select all

#define mulscale32(a,b) (((long long) a * (long long) b) >> 32)
this would turn into:

Code: Select all

(((long long) caz[zz].leng+65536 * (long long) lockrecip[*cac[zz].lock]) >> 32)
The precedence of * over + will severely screw things up there.

Defining mulscale32 like this solves the problem:

Code: Select all

#define mulscale32(a,b) (((long long) (a) * (long long) (b)) >> 32)
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

True, it just confuses me, and I don't like using code unless I can at least understand what it's supposed to be doing.
User avatar
bloodite
Insane DCEmu
Insane DCEmu
Posts: 107
Joined: Thu Dec 26, 2002 7:36 pm
Location: Canada
Has thanked: 0
Been thanked: 0
Contact:

Post by bloodite »

icculus has Started porting the code to MacOS. hopefully it will be of use.

http://icculus.org/cgi-bin/finger/finge ... er=icculus
Try Transfusion, the most ambitious TC for Quake! http://www.planetblood.com/qblood
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

Certainly will. That means they're working on removing all of the DOS/x86 specific code.
ReGex
FrNES Creator
FrNES Creator
Posts: 63
Joined: Wed Oct 17, 2001 7:44 pm
Location: Calgary, Alberta Canada
Has thanked: 0
Been thanked: 1 time
Contact:

Post by ReGex »

Hey BlackAura,
Have you made any progress recently? I'm done school now so I'll have a couple of weeks worth of time to help out (you're probably in the midst of exams right now, if I remember correctly). I'll continue working off of your edited version of the code, if that's cool. Maybe we should set up a sourceforge? Anyways, let me know what you want to do when you have a minute.

-ReGex
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

I've still been looking at it, really. I think setting up a sourceforge would be a good idea.

My edited version of the code is still DOS-based though, since I haven't even tried porting it up to anything else.

I think we probably need to decide what we're going to try to do, and how we're going to go about it. Icculus are working on a MacOS X port, which should help remove the remaining asm code. I've got rid of some of it that Icculus hadn't (when last I checked), and I think I know of a couple of ways to do an OpenGL version.
archduke
Insane DCEmu
Insane DCEmu
Posts: 172
Joined: Wed Oct 17, 2001 7:44 pm
Has thanked: 0
Been thanked: 0

Post by archduke »

so, is this project a real possibility on the dc? :? or is it too much to ask of the poor old console?
BlackAura
DC Developer
DC Developer
Posts: 9951
Joined: Sun Dec 30, 2001 9:02 am
Has thanked: 0
Been thanked: 1 time

Post by BlackAura »

The DC should be able to handle it. A P60 could handle it as well as Quake, and we've got a couple of Quake ports up and running on the DC. Of course, it'd probably be slower, since Build was written partly in assembly, and we have to convert it to C.
Post Reply