Trouble burning dcload-ip

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Post Reply
User avatar
showka
DCEmu Freak
DCEmu Freak
Posts: 95
https://www.artistsworkshop.eu/meble-kuchenne-na-wymiar-warszawa-gdzie-zamowic/
Joined: Fri Nov 23, 2001 12:01 pm
Location: Border Town
Has thanked: 0
Been thanked: 0
Contact:

Trouble burning dcload-ip

Post by showka »

Hi everyone,

I have a trusty old CDR containing dcload-ip-1.0.3 which I snagged from a the internets in years past in the form of a Nero image. I believe I have the HIT-0300 "LAN" adapter (though I paid for the HIT-400, thanks ebay!) and never had luck running dcload-ip-1.0.4.

These days, I'm trying to install the entire Dreamcast tool suite from source and actually have everything working in Cygwin, except I haven't burnt a CD yet. I'd like to burn dcload-ip to cement my understanding of it.

I've modified the Makefile in the "make-cd" directory of dcload-ip, switching the "-xa1" argument to cdrecord to "-xa" since cdrecord in Cygwin is actually wodim which uses -xa for 2048 bytes per sector. I also set the recording speed to argument of cdrecord to 4 (1 made discs completely unrecognizable and the DC went to the CD player screen).

Now I'm at the point where the Dreamcast seems to recognize the CD, as I see the "licensed by Sega" screen along with the Napalm-x logo from ip.bin. However, it simply hangs there forever.

My CD drive is fairly new (uses SATA connectors) so that might also be a factor.

Here's the output from the commands I'm running. Any suggestions would be appreciated. :)

First, I re-ran the make file in target-src/1st_read in case this was important. I remember years back having issues because the lib order to the linker was wrong:

Code: Select all

$ cd ~/dreamcast/dcload-ip/target-src/1st_read
$ make
sh-elf-gcc -O2 -ml -m4-single-only -DDCLOAD_VERSION=\"1.0.4\" -I../../target-inc -o crt0.o -c crt0.S
sh-elf-gcc -O2 -ml -m4-single-only -DDCLOAD_VERSION=\"1.0.4\" -I../../target-inc -o 1st_read.o -c 1st_read.c
sh-elf-gcc -O2 -ml -m4-single-only -DDCLOAD_VERSION=\"1.0.4\" -I../../target-inc -o disable.o -c disable.s
sh-elf-gcc -O2 -ml -m4-single-only -DDCLOAD_VERSION=\"1.0.4\" -I../../target-inc -o memcpy.o -c memcpy.S
cp ../dcload/dcload.bin dcload.bin
sh-elf-ld -A sh -b binary --oformat elf32-shl dcload.bin -o dcload.o -r -EL --no-warn-mismatch
cp ../dcload/exception.bin exception.bin
sh-elf-ld -A sh -b binary --oformat elf32-shl exception.bin -o exception.o -r -EL --no-warn-mismatch
sh-elf-gcc -O2 -ml -m4-single-only -DDCLOAD_VERSION=\"1.0.4\" -Wl,-Tdc4.x -nostartfiles -nostdlib crt0.o 1st_read.o disable.o memcpy.o dcload.o exception.o -o 1st_read -lgcc
sh-elf-objcopy -O binary 1st_read 1st_read.bin
sh-elf-objcopy -O srec 1st_read 1st_read.srec
Then, in the make-cd directory:

Code: Select all

$ make
dd if=/dev/zero of=audio.raw bs=2352 count=300
300+0 records in
300+0 records out
705600 bytes (706 kB) copied, 0.00227759 s, 310 MB/s
cdrecord dev=0,0,0 speed=4 -multi -audio audio.raw
wodim: No write mode specified.
wodim: Asuming -tao mode.
wodim: Future versions of wodim may have different drive dependent defaults.
Device type    : Removable CD-ROM
Version        : 0
Response Format: 2
Capabilities   :
Vendor_info    : 'ASUS    '
Identification : 'DRW-24B3ST   c  '
Revision       : '1.01'
Device seems to be: Generic mmc2 DVD-R/DVD-RW.
Using generic SCSI-3/mmc   CD-R/CD-RW driver (mmc_cdr).
Driver flags   : MMC-3 SWABAUDIO BURNFREE FORCESPEED
Supported modes: TAO PACKET SAO SAO/R96P SAO/R96R RAW/R16 RAW/R96P RAW/R96R
Speed set to 2823 KB/s
Starting to write CD/DVD at speed  16.0 in real TAO mode for multi session.
Last chance to quit, starting real write in    0 seconds. Operation starts.
Track 01: Total bytes read/written: 705600/705600 (300 sectors).
touch burn-audio
scramble ../target-src/1st_read/1st_read.bin 1st_read.bin
mkisofs -C `cdrecord dev=0,0,0 speed=4 -msinfo` -o tmp.iso 1st_read.bin
I: -input-charset not specified, using utf-8 (detected in locale settings)
genisoimage: Warning: -C specified without -M: old session data will not be merged.
Total translation table size: 0
Total rockridge attributes bytes: 0
Total directory bytes: 116
Path table size(bytes): 10
Max brk space used 20000
11888 extents written (23 MB)
( cat IP.BIN ; dd if=tmp.iso bs=2048 skip=16 ) > data.raw
170+0 records in
170+0 records out
348160 bytes (348 kB) copied, 0.00103386 s, 337 MB/s
cdrecord dev=0,0,0 speed=4 -xa data.raw
wodim: No write mode specified.
wodim: Asuming -tao mode.
wodim: Future versions of wodim may have different drive dependent defaults.
Device type    : Removable CD-ROM
Version        : 0
Response Format: 2
Capabilities   :
Vendor_info    : 'ASUS    '
Identification : 'DRW-24B3ST   c  '
Revision       : '1.01'
Device seems to be: Generic mmc2 DVD-R/DVD-RW.
Using generic SCSI-3/mmc   CD-R/CD-RW driver (mmc_cdr).
Driver flags   : MMC-3 SWABAUDIO BURNFREE FORCESPEED
Supported modes: TAO PACKET SAO SAO/R96P SAO/R96R RAW/R16 RAW/R96P RAW/R96R
Speed set to 2823 KB/s
Starting to write CD/DVD at speed  16.0 in real TAO mode for single session.
Last chance to quit, starting real write in    0 seconds. Operation starts.
Track 02: Total bytes read/written: 380928/614400 (300 sectors).
rm -f audio.raw data.raw burn-audio
Ex-Cyber
DCEmu User with No Life
DCEmu User with No Life
Posts: 3641
Joined: Sat Feb 16, 2002 1:55 pm
Has thanked: 0
Been thanked: 0

Re: Trouble burning dcload-ip

Post by Ex-Cyber »

It's been years since I've wrangled these issues, but I'm a bit suspicious of the following:
wodim: No write mode specified.
wodim: Asuming -tao mode.
IIRC, there are fewer issues with DAO/SAO mode. Using the -msinfo output like you're doing ought to adjust for the difference, though.
"You know, I have a great, wonderful, really original method of teaching antitrust law, and it kept 80 percent of the students awake. They learned things. It was fabulous." -- Justice Stephen Breyer
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

There actually seems to be some issues in getting dcload to work with GCC 4.x and/or newer versions of Binutils (I'm not really sure where the problem lies at the moment).

I've actually been working on dcload-ip today, and I have a new binary that if you would like, you can try to burn to a CD and see if it works. I don't actually have any blank CDs handy, so I can't test it by CD, however when I upload it with dc-tool to a working dcload-ip disc, it seems to work. I'd actually be interested to know if it does work properly when burned to a disc, so if someone could test it, I'd appreciate it. The file is in the zip file attached to this post. It is an unscrambled raw binary, so make sure to scramble it before burning it to a disc (if you just drop the extracted 1st_read.bin in the 1st_read directory over top any existing 1st_read.bin files, it should hopefully pick it up for the make-cd Makefile).

In case anyone is interested in what changes are in this, versus the git repository dcload code, here's what I changed:
1) Replaced the 1st_read loading code with a very small assembly version thereof (adapted from my pso_patcher).
2) Changed the background color when dcload is running to blue (just so I could make sure I was running the version I thought I was running).
3) Changed the SR value set by the go() function to 0x500000f0 instead of 0x600000f0. For whatever reason, if I left interrupts enabled (which the 0x600000f0 value does), dcload crashes when it tries to execute the binary. I suppose it could also be the other bit that is different (the register bank bit), but that makes less sense than the interrupt bit causing the issue. :lol: Of course... this might screw with programs that assume that interrupts are enabled when dcload loads them, but I don't know of any code that should have any major issues with them being disabled.

If I get confirmation that this works, I'll try to get the code in the git repo updated soonish.
Attachments
1st_read.bin.zip
(11.48 KiB) Downloaded 114 times
User avatar
showka
DCEmu Freak
DCEmu Freak
Posts: 95
Joined: Fri Nov 23, 2001 12:01 pm
Location: Border Town
Has thanked: 0
Been thanked: 0
Contact:

Re: Trouble burning dcload-ip

Post by showka »

Alas, it didn't work.

I changed the make-cd directories Makefile 1ST_READ variable to the path to your attached 1st_read.bin file.

Unfortunately it didn't work. Strangely enough, it wasn't even recognized as a playable disc. The ones I had burnt myself at least got to the "licensed by Sega" screen before freezing.

To make sure I could burn any CD, I compiled the pvrmark_strips_direct example, run it through "sh-elf-objcopy -O binary -R .stack pvr_marks_strips_direct.elf 1st_read.bin" and put that in place of 1ST_READ in the dc-tool make-cd Makefile and the CD worked.

Tangent: My original dcload-ip disc boots up extremely fast, but the pvrmark_strips_direct CD takes at eight times as long. Seems like it's really stressing out the motor... anyone know what might cause this?

I also was able to run your 1st_read.bin script from dc-tool using my existing dcload-ip disc, and from there I was able to run other things. I'm a bit stumped why it only fails when being burned to a disc.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

showka wrote:Alas, it didn't work.

I changed the make-cd directories Makefile 1ST_READ variable to the path to your attached 1st_read.bin file.

Unfortunately it didn't work. Strangely enough, it wasn't even recognized as a playable disc. The ones I had burnt myself at least got to the "licensed by Sega" screen before freezing.
That's certainly odd... :?

I guess I'll have to try to find some CD-Rs to test it out tomorrow or something...
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

Hrm, well... I just burned a disc of it (admittedly, not using the make-cd Makefile), and it seems to work (other than not showing any status on the screen (it looks as though it is frozen on the Sega license screen, but it actually does work other than that).

I'll look at it further later today and see if I can fix the issue I described up above and to see if I can't get it to burn the disc with the Makefile setup.
User avatar
showka
DCEmu Freak
DCEmu Freak
Posts: 95
Joined: Fri Nov 23, 2001 12:01 pm
Location: Border Town
Has thanked: 0
Been thanked: 0
Contact:

Re: Trouble burning dcload-ip

Post by showka »

Success! I got your version of dcload-ip to run. It froze at the license screen, but I was able to use dc-tool to run stuff.

Here's how I discovered this.

This morning I decided to try the dcload-ip disc I'd built from source to see if dc-tool would work once it froze at the license screen like you described. It didn't.

On a lark, I then put in the pvr mark strips direct example CD I'd burnt, and found it would not work. Knowing I'd seen it running, I kept trying and had success on my fifth attempt.

I then tried using one of the discs I'd burnt with your modified dcload-ip and after several attempts the disc was read.

Maybe this is just a myth, but I'm concerned these discs could be stressing the Dreamcast's motor. I actually bought a second Dreamcast years ago after my original one seemed to get progressively worse at playing burnt CDs. The dcload-ip I burnt from a Nero image on my old laptop starts up 100% of the time and much quicker than the CDs I've burnt, so something about the process I'm using must be sub-par. I'd be interested in knowing if you find any differences between the dcload-ip makefile and your burn process.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

Honestly, it could all boil down to the quality of the disc itself. If you look at the output from cdrecord/wodim, it should give some measure of the quality of the disc and such. I've found that often if that quality measure output by cdrecord/wodim isn't a B or an A, the disc just plain won't be readable in the Dreamcast or will stress the drive more if it does eventually read.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

Well, I've finished up on fixing everything, and I know that what is in the git repository for dcload-ip now does work properly when built with GCC 4.7.0. Other than a few minor changes since the last binary I posted here, there isn't all that much new on the Dreamcast side of things.

I also fixed some problems with 64-bit pointers in dc-tool-ip. Basically, doing anything with /pc that involved a directory (opendir/readdir/closedir) would cause it to segfault on 64-bit Mac OS X. I fixed that and committed the changes.

Also, I updated the make-cd Makefile to work with wodim, while I was at it.

I've attached a new dcload-ip binary here (although, you can easily build the source and get a working version now). Once again, it is unscrambled, just like last time. I'd appreciate it if people could test out the new code (either the binary or the source itself) and see if they find any new bugs. If no bugs are found, I'll probably package it up as a new version sometime.
Attachments
1st_read.bin.zip
(11.49 KiB) Downloaded 130 times
User avatar
showka
DCEmu Freak
DCEmu Freak
Posts: 95
Joined: Fri Nov 23, 2001 12:01 pm
Location: Border Town
Has thanked: 0
Been thanked: 0
Contact:

Re: Trouble burning dcload-ip

Post by showka »

Sorry, but it's not working for me.

First, I pulled down the latest changes from git. I had to burn three CDs before it worked (thanks to the spindle of relics I'm working with), but I was eventually able to see a nice green screen (not blue like before).

Unfortunately, when I ran the pvrmark_strips_direct.elf example, before I could see anything the screen flickered and the Dreamcast itself rebooted. Unfortunately the same example worked with my much older working dc-tool CD as well as the "blue" screen version I'd burnt earlier.

In all of these case I was running a freshly compiled version of host-src/tool/dc-tool.exe from the latest code.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

I changed the screen color to blue in the git repository (and the version number to 1.0.5). If it was green or still said 1.0.4, it was not the correct version.

You should be able to grab it with this command:

Code: Select all

git clone git://cadcdev.git.sourceforge.net/gitroot/cadcdev/dcload-ip
Any other repositories are not the official one and are completely unsupported (by me anyway).
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: Trouble burning dcload-ip

Post by TapamN »

I've just upgraded from a serial cable to a BBA (uploading >1 MB of data/program was getting real tiresome, and it won't stop growing) and have been working on converting over to it. While doing so, I've found a bug in the newest version of the host program when running on Cygwin on Windows 7.

The bug is in the lseek syscall, and prevents seeking in reverse with open files. The bug only happens on host systems where the offset parameter type (off_t) is defined as being 64 bits. off_t is a signed 32-bit integer on the Dreamcast, but the host's buffer treats the number as an unsigned int. If the host's off_t is 32-bits, it just reinterprets the offset as signed and it works like it should. But with a 64-bit off_t (like in Cygwin), the offset passed to lseek is zero extended to 64-bits, making it impossible to seek in reverse.

To fix it, in host-src/tool/syscalls.c, change this...
retval = lseek(ntohl(command->value0), ntohl(command->value1), ntohl(command->value2));
...to this...
retval = lseek(ntohl(command->value0), (int)ntohl(command->value1), ntohl(command->value2));
While not quite directly related to this thread's topic, I also have some other issues with using the BBA that I'll mention here:

While the initial program upload is fast (~800 KB/s), there seem to occasionally be delays (up to about 5 seconds) before when I start the upload program and when it actually starts transferring. A bit annoying, but not as bad as this: Reading files from the PC over BBA is actually SLOWER than what I got on serial! (Serial: ~32 KB/s (384000 bps connection) BBA: ~20 KB/s) Interestingly, writing to files with the BBA is much faster, getting around 100 KB/s. Writing to console is slower than serial, but that's not as important to me since I have a graphical console setup on the DC that I use for most messages.

It looks like, when transferring a large file over the BBA with a precompiled VMU Tool (which claims to use KOS 1.2.0 from 2006), I get a much better transfer rate at around 160 KB/s. So the slow read speed probably isn't caused by the host or loader, but either by something with my code or some change to KOS since then. I don't think my code is the problem; it just opens the file, calls fread onto a buffer a few times, then closes the file. Not much for me to do wrong, although I have tried changing the size of the buffer (to make fewer fread calls), but that seems to have no effect. Any ideas?

In addition to the slow speed, I've also had some reliability problems with dc-tool though the BBA, with the Dreamcast randomly freezing at the start of doing any BBA syscall (anything from accessing a file to console I/O); it's happened both on things I've compiled on my system and older, precompiled programs (VMU Tool). The reset function of dc-tool still works when this happens, so it's probably not a total lockup, but just the DC sitting and waiting for something.

It took me a while, but I eventually figured out the trigger for the freeze: 60 seconds without network activity. After a bit of searching ("a bit of searching" means looking at the first result from Google for, "windows udp timeout 60 seconds"), I found that the problems are apparently caused by how the Windows firewall handles UDP connections. If a UDP connection is idle for more than 60 seconds, it's removed from the NAT and blocked, even if it's still kept open. Adding an exception to the firewall seems to have fixed this.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

TapamN wrote:I've just upgraded from a serial cable to a BBA (uploading >1 MB of data/program was getting real tiresome, and it won't stop growing) and have been working on converting over to it. While doing so, I've found a bug in the newest version of the host program when running on Cygwin on Windows 7.

The bug is in the lseek syscall, and prevents seeking in reverse with open files. The bug only happens on host systems where the offset parameter type (off_t) is defined as being 64 bits. off_t is a signed 32-bit integer on the Dreamcast, but the host's buffer treats the number as an unsigned int. If the host's off_t is 32-bits, it just reinterprets the offset as signed and it works like it should. But with a 64-bit off_t (like in Cygwin), the offset passed to lseek is zero extended to 64-bits, making it impossible to seek in reverse.

To fix it, in host-src/tool/syscalls.c, change this...
retval = lseek(ntohl(command->value0), ntohl(command->value1), ntohl(command->value2));
...to this...
retval = lseek(ntohl(command->value0), (int)ntohl(command->value1), ntohl(command->value2));
Sensible change. I'll make sure it doesn't break anything else and probably commit it tomorrow.
While not quite directly related to this thread's topic, I also have some other issues with using the BBA that I'll mention here:

While the initial program upload is fast (~800 KB/s), there seem to occasionally be delays (up to about 5 seconds) before when I start the upload program and when it actually starts transferring. A bit annoying, but not as bad as this: Reading files from the PC over BBA is actually SLOWER than what I got on serial! (Serial: ~32 KB/s (384000 bps connection) BBA: ~20 KB/s) Interestingly, writing to files with the BBA is much faster, getting around 100 KB/s. Writing to console is slower than serial, but that's not as important to me since I have a graphical console setup on the DC that I use for most messages.

It looks like, when transferring a large file over the BBA with a precompiled VMU Tool (which claims to use KOS 1.2.0 from 2006), I get a much better transfer rate at around 160 KB/s. So the slow read speed probably isn't caused by the host or loader, but either by something with my code or some change to KOS since then. I don't think my code is the problem; it just opens the file, calls fread onto a buffer a few times, then closes the file. Not much for me to do wrong, although I have tried changing the size of the buffer (to make fewer fread calls), but that seems to have no effect. Any ideas?
Not sure on that one. I can't think of any changes to KOS since 1.2.0 that would affect that, but there have been so many that it would probably be pretty difficult to put a finger on off the top of my head anyway. :?
In addition to the slow speed, I've also had some reliability problems with dc-tool though the BBA, with the Dreamcast randomly freezing at the start of doing any BBA syscall (anything from accessing a file to console I/O); it's happened both on things I've compiled on my system and older, precompiled programs (VMU Tool). The reset function of dc-tool still works when this happens, so it's probably not a total lockup, but just the DC sitting and waiting for something.

It took me a while, but I eventually figured out the trigger for the freeze: 60 seconds without network activity. After a bit of searching ("a bit of searching" means looking at the first result from Google for, "windows udp timeout 60 seconds"), I found that the problems are apparently caused by how the Windows firewall handles UDP connections. If a UDP connection is idle for more than 60 seconds, it's removed from the NAT and blocked, even if it's still kept open. Adding an exception to the firewall seems to have fixed this.
Well, I'm glad to hear that at least that wasn't my fault. :lol:
TapamN
DC Developer
DC Developer
Posts: 104
Joined: Sun Oct 04, 2009 11:13 am
Has thanked: 2 times
Been thanked: 88 times

Re: Trouble burning dcload-ip

Post by TapamN »

I've mostly managed to work out the performance problems, speeding up file uploads, downloads, and console I/O.

The first change was in how the host program handles requests from the Dreamcast, which improved the console and DC-to-PC upload speeds. The host uses a rather questionable method to poll for data with timeouts. Instead of using blocking I/O and specifying a timeout, the program using non-blocking I/O and polls in a loop until something comes in. To prevent 100% CPU usage during the poll loop, the program sleeps for half a second between polls. This adds 0.0-0.5 seconds to every request the Dreamcast makes, because the program will normally be in the middle of sleeping and can't respond immediately.

I modified the console loop to switch to blocking I/O, so the host can just wait in the recv call for something and respond much quicker, and removing the call to nanosleep. The top of the do_console loop now looks like this:

Code: Select all

	fcntl(dcsocket, F_SETFL, 0);	// Disable non-blocking I/O (i.e. enable blocking I/O)
	while (1) {
		fflush(stdout);
		recv(dcsocket, (void *)buffer, 2048, 0);
This seems perfectly stable over several hours of use and much more responsive, with all communications getting a speed boost.

(Is there any specific reason that the host handle timeouts so strangely? You can use blocking UDP I/O with a specified timeout just fine by using the setsockopt call; there's no need for the funny poll loops. Maybe it's a problem when using MinGW or something?)

The second change was modifying the PACKET_TIMEOUT constant, which not only defines the packet timeout length, but ratios of it are also used elsewhere for timing. Decreasing the constant would cause the initial upload speed to drop, but increase the speed of file transfers. Decreasing it too much causes transfer errors. With how the value is reused everywhere, I haven't yet figured out what specific host delays are responsible for the changes in speed. I wouldn't be surprised if the best value depended on the hardware setup (e.g. the distance between the host and Dreamcast, what kind of ethernet controller the host has, etc.). Ideally, the host should probably try to figure out what the idea timing values are and dynamically adjust as necessary. I don't really have any useful code regarding this at this point; it's just an observation that other people using BBAs might want to try out.

The third problem I found was that fread is slower than read. I haven't looked into why, but switching over to read made things around 10 times (!!!) faster, even when things like read sizes are identical. I don't know if fwrite and write have different speeds, but fwrite is already plenty fast.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

TapamN wrote:I've mostly managed to work out the performance problems, speeding up file uploads, downloads, and console I/O.

The first change was in how the host program handles requests from the Dreamcast, which improved the console and DC-to-PC upload speeds. The host uses a rather questionable method to poll for data with timeouts. Instead of using blocking I/O and specifying a timeout, the program using non-blocking I/O and polls in a loop until something comes in. To prevent 100% CPU usage during the poll loop, the program sleeps for half a second between polls. This adds 0.0-0.5 seconds to every request the Dreamcast makes, because the program will normally be in the middle of sleeping and can't respond immediately.

I modified the console loop to switch to blocking I/O, so the host can just wait in the recv call for something and respond much quicker, and removing the call to nanosleep. The top of the do_console loop now looks like this:

Code: Select all

	fcntl(dcsocket, F_SETFL, 0);	// Disable non-blocking I/O (i.e. enable blocking I/O)
	while (1) {
		fflush(stdout);
		recv(dcsocket, (void *)buffer, 2048, 0);
This seems perfectly stable over several hours of use and much more responsive, with all communications getting a speed boost.

(Is there any specific reason that the host handle timeouts so strangely? You can use blocking UDP I/O with a specified timeout just fine by using the setsockopt call; there's no need for the funny poll loops. Maybe it's a problem when using MinGW or something?)
Honestly, I haven't ever looked at most of the dcload code. I suspect that any sort of blocking/non-blocking I/O stuff has been the way it has been for a long time. If it were written for any sort of speed improvement, I'd suspect it was on some old version of Linux, not for MinGW, Cygwin, or anything else Windows related.
The second change was modifying the PACKET_TIMEOUT constant, which not only defines the packet timeout length, but ratios of it are also used elsewhere for timing. Decreasing the constant would cause the initial upload speed to drop, but increase the speed of file transfers. Decreasing it too much causes transfer errors. With how the value is reused everywhere, I haven't yet figured out what specific host delays are responsible for the changes in speed. I wouldn't be surprised if the best value depended on the hardware setup (e.g. the distance between the host and Dreamcast, what kind of ethernet controller the host has, etc.). Ideally, the host should probably try to figure out what the idea timing values are and dynamically adjust as necessary. I don't really have any useful code regarding this at this point; it's just an observation that other people using BBAs might want to try out.
Once again, I haven't looked at any of this, as I've never been particularly dissatisfied with the transfer speeds from my Mac to my Dreamcast(s).
The third problem I found was that fread is slower than read. I haven't looked into why, but switching over to read made things around 10 times (!!!) faster, even when things like read sizes are identical. I don't know if fwrite and write have different speeds, but fwrite is already plenty fast.
That is surprising that it would be that much faster. It really shouldn't be that much faster. Yes, fread/fwrite tend to do cacheing that read/write do not, but there shouldn't be anywhere near that large of a difference. That is most likely a problem with the library setup on the host system. :?
patbier
DC Developer
DC Developer
Posts: 152
Joined: Fri Aug 29, 2003 1:25 am
Has thanked: 0
Been thanked: 0

Re: Trouble burning dcload-ip

Post by patbier »

TapamN wrote:
It took me a while, but I eventually figured out the trigger for the freeze: 60 seconds without network activity. After a bit of searching ("a bit of searching" means looking at the first result from Google for, "windows udp timeout 60 seconds"), I found that the problems are apparently caused by how the Windows firewall handles UDP connections. If a UDP connection is idle for more than 60 seconds, it's removed from the NAT and blocked, even if it's still kept open. Adding an exception to the firewall seems to have fixed this.
Waoh, thanks a lot. You can't imagine how much time I lost with that. A the beginning, I thought it was a problem with my program. But when I tested on a burned cd version, the "freeze" wasn't there. So i always thought it was a dc-tool bug. To deal with it, when I test my program with dc-tool, I add a printf every second.
I've just tested : I've just added an exception for dc-tool-ip in the windows firewall, and no freeze ! Thanks again.
ImageAlice Dreams Tournament Dreamcast fans : http://www.facebook.com/alicedreamst
In August 2015, we had to change "Dynamite Dreams" name to "Alice Dreams Tournament"
User avatar
showka
DCEmu Freak
DCEmu Freak
Posts: 95
Joined: Fri Nov 23, 2001 12:01 pm
Location: Border Town
Has thanked: 0
Been thanked: 0
Contact:

Re: Trouble burning dcload-ip

Post by showka »

@BlueCrab: I finally found time to figure out what I was doing wrong.

What I needed to do was add the line "TARGETCCVER = 4" to Makefile.cfg. Ironically, I found this info by searching all my Dreamcast files for "-Tdc" and finding TARGETCCVER defined in an old copy of the dcload-ip source which I'd moved to a different directory.

Without that I'd get the following error:

Code: Select all

sh-elf-gcc -O2 -ml -m4-single-only -DDCLOAD_VERSION=\"1.0.5\" -Wl,-Tdc.x -nostartfiles -nostdlib crt0.o dcload-syscall.o dcload-syscalls.o memcpy.o console-test.o -o console-test -lgcc
/opt/toolchains/dc/sh-elf/lib/gcc/sh-elf/4.7.0/../../../../sh-elf/bin/ld: cannot open linker script file dc.x: No such file or directory
collect2: error: ld returned 1 exit status
Makefile:31: recipe for target `console-test' failed
make[1]: *** [console-test] Error 1
I assumed this part of the makefile was just to create an example or something else unnecessary for the CD, since it didn't seem to affect the make process for make-cd directory. Being confused later was my just desserts for blithely ignoring an error of any kind in trying to set this up.
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5652
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has thanked: 9 times
Been thanked: 69 times
Contact:

Re: Trouble burning dcload-ip

Post by BlueCrab »

Ah... I thought I removed all mention of those linker scripts from the code. Obviously I missed a reference though. You shouldn't need that stuff at all, though.

Either way, the example stuff shouldn't be necessary to build a new CD of dcload-ip...
User avatar
showka
DCEmu Freak
DCEmu Freak
Posts: 95
Joined: Fri Nov 23, 2001 12:01 pm
Location: Border Town
Has thanked: 0
Been thanked: 0
Contact:

Re: Trouble burning dcload-ip

Post by showka »

Hmm... well, either way, I'm set now. Thanks!
Post Reply