dcload-ip with DHCP support!

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

dcload-ip with DHCP support!

Post by Moopthehedgehog » Sat Nov 23, 2019 3:30 pm

Update 12/31/2019:
All done! See the GitHub page for burnable CDIs and the latest source.
https://github.com/Moopthehedgehog/dcload-ip
CDIs are in the "Releases" section of the GitHub repository. They are lightly padded (about 64MB of padding).

Update 12/10/2019:
On github now: https://github.com/Moopthehedgehog/dcload-ip
CDIs are in the "Releases" section of the GitHub repository.

Old post (attachment explanations are in here, though as of 12/10/2019 all attachments to this post are old. The perf counter timing thing is still not figured out completely, but I have a way to work around it until the true meaning behind it is understood):
Spoiler!

Alrighty, so here is what I have so far since it was requested that I post the current status. No, it is not quite ready yet for actual release, so consider it very alpha. I think it's more like beta now with the 12/2/2019 source.

Some notes:
  • Everything from 1.0.5 should be there and still work as expected. (Ideally)
  • The seconds countdown for the DHCP lease is not exactly 1 second. It's veeeerrrrrryyyy close, but I don't know what the actual frequency of the performance counters in elapsed time mode is. My "stopwatch" calculation shows somewhere around 2.4GHz, but as I mentioned here I have a hard time believing that's correct. No, it's off by like 11.96988, see later in this thread. Still trying to figure this out; it's the last thing.
  • Resetting dcload with -r from DC-TOOL's command line interface causes the DHCP countdown display to stop working. I don't know why this is. It also happens after executing a command. I would like the DHCP countdown to persist. Fixed.
  • There's just over 1.4kB free memory space left, shared with the stack. Try to keep it over 700 bytes, ideally over 1024 bytes, if adding anything new. I'm trying to make this 2kB by converting the command buffer into using the packet transmit buffer, but it's not working for some reason. Well, that was actually pretty silly. I fixed it. BSS has 2.4KB freed up now. Unfortunately it looks like there's a more significant constraint on the file size that I didn't realize was there. There's like 3.6kB left free now as of 12/2/2019. Space is no longer an issue.
  • There appears to be an upper limit to the absolute filesize of dcload.bin. I don't know what it is; somewhere in the mid-20kBs? Needs more looking into. Linkerscript problems, as far as I can tell. I believe this is fixed now.
  • The makefile and build options for dcload had to be modified, so there are some new options I've added there to help with future debugging and to make things just work better in general. All the .asm files are GCC's produced assembly for the .c files, and output.map is, well, a map of the dcload elf file (dcload.elf is just the extensionless dcload file that appears after running make).
  • draw_progress() now outputs decimal digits instead of hexadecimal ones (lol), but nothing uses it in 1.0.5. It's there, nonetheless, and would take up about 144 bytes if it were used, so keep that in mind.
  • Using the command string "PMCR" allows for complete control over the SH4's two 48-bit performance counters. I still don't know how to send this data, so I haven't been able to test it, but I've gone over the code several times and see no reason for it not to work. You'll get textual feedback if it's working right. Got it, see https://dcemulation.org/phpBB/viewtopic ... 0#p1057140 and https://dcemulation.org/phpBB/viewtopic ... 4#p1057114
  • I just discovered right before posting this that I broke the DHCP countdown timer. The seconds are wrong. It was working earlier today... Oh no, they're not wrong. Not wrong at all. This is hilarious: my router decided to put my Dreamcast on probation because it was asking for the same IP too much, so it says my DC's lease expires "never" now, but everything else has an expiry time.
Important Changes:
  • Static IPs in Makefile.cfg should be written with leading zeros. I had to do this so that the second-most-common IP address range I've seen in the US (192.168.0.x) can work, in addition to many other IP address ranges that have zeros in them.
  • Specifying 0.0.0.0 in Makefile.cfg activates DHCP mode!! (Well, it's really 0.x.x.x, since the whole 0.0.0.0/8 range is reserved for that purpose anyways). The ARP trick will still work, but you should really be using 169.254.x.x IP addresses for linking two machines together and doing that. 0.0.0.0/8 is mine now. :P
  • If DHCP fails, the IP address will be shown as 0.0.0.0, and in this situation the "arp with IP of 0.0.0.0" trick will be necessary like "old times."
Requires KOS build environment set up. I'm using Windows Subsystem for Linux v2 (Windows Insider program), but it might work with WSLv1. Don't use Ubuntu if you go this route, as KOS won't build. Use Debian instead.

EDIT: Updated source attachment.

Another edit: So I'm wondering some more about these performance counters, and I found this: http://www2.lauterbach.com/pdf/debugger_user.pdf
So at least there's something that exists that uses them. However it very explicitly states to use the CPU clock frequency, which we know to be 200MHz, not the absurd 2.4GHz number I've had to use. I'm wondering if this isn't one of those things where like the bottom 16 bits are reserved and the "elapsed time" is only in the top 32 bits, and you divide that value by the clock frequency to get time in... milliseconds? seconds? Really not sure. Maybe an Lxdream dev knows? Might need to go track down that debugger software... -- found it, it's in the demo downloads for the SH supported processor cores: https://www.lauterbach.com/frames.html?home.html, but sadly that one screenshot in the pdf showing PMCTR1 and PMCTR2 is more helpful than the software itself... :(

Also, I believe I've fixed DHCP renewal. Updated the source attachment. Will be sure in about 20-25 minutes. -- Yep, it's fixed. I'll even put up a CDI image so you can see what it looks like so far!

Edit 12/2/2019: updated source, updated post. It's messy here, sorry.

12/3/2019: Newer source, see https://dcemulation.org/phpBB/viewtopic ... 0#p1057160

12/5/2019: updated source, QoL improvements. Also binaries are in the source package (and target-src/1st_read/1st_read.bin is scrambled).
Attachments
dcload-ip-master2--DHCP-12-5-2019.zip
Old source, see GitHub link for latest. I use GCC 9.2.0 now. See GitHub for newer binaries.
(462.45 KiB) Downloaded 24 times
GCC-binutils config lines.zip
Hastily written notes about compiling sh4 elf gcc 9.2.0
(1.05 KiB) Downloaded 20 times
dcload-ip-master2--DHCP-12-3-2019.zip
Old source. Probably works with GCC 4.7.x, but I use GCC 9.2.0 now
(231.53 KiB) Downloaded 18 times
dcload-ip-master2--DHCP-12-2-2019.zip
Old source. Probably works with GCC 4.7.x, but I use GCC 9.2.0 now
(207.51 KiB) Downloaded 17 times
dcload-ip-master2--DHCP-11-26-2019.zip
Old source code, requires KOS build environment set up
(207.31 KiB) Downloaded 19 times
Last edited by Moopthehedgehog on Thu Jan 02, 2020 10:27 am, edited 13 times in total.
These users liked the author Moopthehedgehog for the post:
mrneo240
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
kazade
Insane DCEmu
Insane DCEmu
Posts: 140
Joined: Tue May 02, 2017 3:11 pm
Has liked: 2 times
Been liked: 16 times

Re: dcload-ip with DHCP support!

Post by kazade » Sun Nov 24, 2019 7:17 am

This is very exciting!! Great work!

There are a bunch of people on the Simulant Discord* who I'm sure will be excited to try this, feel free to drop in and chat :)

*https://discordapp.com/invite/TRx94EV
mrneo240
DCEmu Freak
DCEmu Freak
Posts: 73
Joined: Wed Mar 14, 2018 12:22 am
Has liked: 9 times
Been liked: 10 times

Re: dcload-ip with DHCP support!

Post by mrneo240 » Sun Nov 24, 2019 8:21 am

Quick sidenote:

Using ubuntu under WSL v2 works fine for dreamcast and Kos.
Used it daily since the first version of wsl v2 in June, zero issues.
User avatar
lerabot
Insane DCEmu
Insane DCEmu
Posts: 123
Joined: Sun Nov 01, 2015 8:25 pm
Has liked: 0
Been liked: 3 times

Re: dcload-ip with DHCP support!

Post by lerabot » Sun Nov 24, 2019 9:58 am

Thank you! I'll keep an eye on this project!
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Sun Nov 24, 2019 10:44 am

Just a note--you might be able to use that CDI as it is in its current state. The main purpose of DCLOAD has been working for me thus far for everything I've tried except the exception-test, and I would really appreciate someone explaining the exception binary so I can understand why this is happening and fix it.

There are really only 3 things left that need to be done before it's finished:

1. Understand the file size constraint. How does the exception handler binary work? What are the size limitations concerning dcload-ip because of it? How come the exception-test is the only example program that doesn't work? The CDI I posted above actually has the remote-PMCR functionality disabled/commented out and is using a stripped down "disable command only" version because I was trying to find some kind of filesize limit. With the full remote-PMCR functionality implemented, dcload.bin is about 21,920 bytes. With reduced PMCR it's 21,644 bytes, and with all remote-PMCR functions removed it's 21,184 bytes. Stock dcload.bin is 21,000 bytes, and exception.bin is 2048 bytes. There's 1024 bytes of space between the end of exception.bin and 0x8c010000, and I don't know what the purpose of that space is or what the importance of 0x8c00f400 might be.

2. Why doesn't the DHCP lease display persist when sending a cmd_reboot via dc-tool's -r flag or after executing some binary? It's very odd. It's like an entire if() block is just getting completely skipped on re-entry to main() for no rhyme or reason.

3. [Not strictly necessary, but would be nice to know for accuracy] How does the elapsed time measurement mode of the SH4 performance counters actually work? The way I'm using them for timing is just a best guess.

If I knew the answers to these it probably would've been done by now... I might be able to find the answer to 1 in some hardware documentation (maybe), 2 might require some sleuthing in the output ASM if it's a GCC thing, and 3 I just have no idea.
mrneo240 wrote:
Sun Nov 24, 2019 8:21 am
Quick sidenote:

Using ubuntu under WSL v2 works fine for dreamcast and Kos.
Used it daily since the first version of wsl v2 in June, zero issues.
Following these instructions only worked with stock WSLv2 Debian for me without any extra work, which is why I'm recommending it.

They've all been updated since WSLv2 first came out, though, so maybe you have a slightly older version? The latest version of Ubuntu (as of last week) didn't work for me without some extra finagling, unlike Debian which "just worked(tm)". If you run uname -r it's possible you might see you're using an older version... unless you did some extra work to get it up and running? The Windows Store downloads get updated with no fanfare, and the only way to really "update" is to uninstall and reinstall from the store. The Debian kernel I'm on is 4.19.79, for reference. I mean, I don't doubt Ubuntu can be made to work; Debian was just a seamless install and go.
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Sun Nov 24, 2019 10:03 pm

I finally figured out how to send commands to DCLOAD! It requires netcat/ncat and a raw payload file.

Use this on Windows (requires downloading ncat, which is part of the nmap package here): ncat -4 -u [Dreamcast's IP] < [path to payload]

Using a tool like Wireshark to view the response packets, the 64-bit data is stored in the last 8 bytes of the response when a read command is sent. It's in little endian, as I didn't byteorder-flip it for transmission across the network, and my reasoning is so it could be directly fed into a program on x86 without any extra conversion steps.

So I took some measurements of performance counter 1 and found that it appears to run at about 11.96988 x 200MHz. It would be great if someone double-checked my calculations here. I'm not sure how there's even any circuitry capable of running at 12x per clock on the Dreamcast, but here it is.

...And then I came across this, in the Renesas SH7750 hardware manual, page 67:
https://www.renesas.com/us/en/products/microcontrollers-microprocessors/superh/sh7750/sh7750r.html wrote:Note: The SH-4 does not support endian conversion for the 64-bit data format. Therefore, if double-precision floating-point format (64-bit) access is performed in little endian mode, the upper and lower 32 bits will be reversed.
So now I'm off to check whether result_high and result_low need to be flipped. -- Edit: Nope, doesn't look like I got it wrong. If I unmask the upper bits they're just 0, and checking the byte order in the packet for small numbers (like <100 secs) has zeros in the expected places. The elapsed time counters are just incrementing way faster than the clock speed, so the max lease time that can be supported is actually 117576 seconds, or about 1 day, 8 hours, and 40 minutes. I think that's fine, especially since the most I've ever seen in practice is one day, anyways. :)

I am still curious to know how the designers pulled that ~12x multiplier off, though. 2393976245 is just a weird number. I didn't see any obvious patterns in the bits comparing it to 200000000, either.

EDIT: Regarding remaining thing #1, well, wow, what a wild goose chase. The exception-test example program doesn't even work with stock dcload, so I'm just not going to bother with it. And I think #2 is caused by go.s clearing all the registers and resetting the stack. GCC probably doesn't like that!

12/3/2019: Removed CDI image. It's way old now. The other attachments are still applicable.
Attachments
Perf counter 1 measurements.xlsx
My perf counter 1 measurements
(11.25 KiB) Downloaded 7 times
PMCR_R1.zip
Payload for ncat to read PMCR1
(174 Bytes) Downloaded 19 times
Last edited by Moopthehedgehog on Wed Dec 04, 2019 11:50 am, edited 1 time in total.
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
User avatar
ThePerfectK
Insane DCEmu
Insane DCEmu
Posts: 127
Joined: Thu Apr 27, 2006 10:15 am
Has liked: 21 times
Been liked: 19 times

Re: dcload-ip with DHCP support!

Post by ThePerfectK » Tue Nov 26, 2019 7:25 am

I'm loving this topic, please keep posting as you find more information, this is pretty great!

Your perf results -- is that the perf for your windows program trying to communicate with the DC, or are you some how running a perf test on the DC itself?
Still Thinking!~~
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Tue Nov 26, 2019 10:42 am

ThePerfectK wrote:
Tue Nov 26, 2019 7:25 am
I'm loving this topic, please keep posting as you find more information, this is pretty great!

Your perf results -- is that the perf for your windows program trying to communicate with the DC, or are you some how running a perf test on the DC itself?
Perf test on the DC itself. I added the ability to query the DC's counters, disable them, restart them, and even change their modes (there are like 30) from over the network. That's what I meant by remote-controlled PMCR. :)

It just took me a while to figure out how the heck to send arbitrary data over the network since dc-tool doesn't let you do what I was trying to do. If you send a specially made command packet as per my above post, you can access the functions in commands.c in dcload-ip's source. Ncat/netcat and a hex editor to make the payload data was the answer I was looking for.

You can apparently also use this syntax:
ncat -4 -u [Dreamcast's IP] < [path to payload] > [path to some file to store the response in]
And you won't need to use Wireshark to read the Dreamcast's response packet, although it won't be timestamped (there's definitely a way to append timestamping via Windows command line, though I don't know it off the top of my head). Also worth noting: this only works on the Windows command line, not PowerShell. I don't know how to do < and > on PowerShell. On Linux I assume these might be | pipes, unless < and > do the same thing there. I know > is the same.

In other news:
I also think I have a better lead on #2, and it's not go.s like I thought. I think GCC might be very unhappy with triple-nested if() statements... Working on this now. There are also MUCH better ways to implement the whole exception handler, and long story short it's do it like x86. There's no need to use a separate binary that has to rely on dcload C functions when the vector table can just be set up inside dcload's binary itself. That way loader.s only needs to copy one binary over, and dcload.c can then set up the exception region as it pleases making full use of all available internal functionality. This way GCC optimization won't destroy things. It would be particularly easy in this case to do it this way because the hardware is fixed and the addresses are always the same for everybody. Basically it's incorporating exception.s into dcload.bin instead of having it separate.
These users liked the author Moopthehedgehog for the post:
ThePerfectK
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
User avatar
ThePerfectK
Insane DCEmu
Insane DCEmu
Posts: 127
Joined: Thu Apr 27, 2006 10:15 am
Has liked: 21 times
Been liked: 19 times

Re: dcload-ip with DHCP support!

Post by ThePerfectK » Tue Nov 26, 2019 6:51 pm

Moopthehedgehog wrote:
Tue Nov 26, 2019 10:42 am
ThePerfectK wrote:
Tue Nov 26, 2019 7:25 am
I'm loving this topic, please keep posting as you find more information, this is pretty great!

Your perf results -- is that the perf for your windows program trying to communicate with the DC, or are you some how running a perf test on the DC itself?
Perf test on the DC itself. I added the ability to query the DC's counters, disable them, restart them, and even change their modes (there are like 30) from over the network. That's what I meant by remote-controlled PMCR. :)

It just took me a while to figure out how the heck to send arbitrary data over the network since dc-tool doesn't let you do what I was trying to do. If you send a specially made command packet as per my above post, you can access the functions in commands.c in dcload-ip's source. Ncat/netcat and a hex editor to make the payload data was the answer I was looking for.

You can apparently also use this syntax:
ncat -4 -u [Dreamcast's IP] < [path to payload] > [path to some file to store the response in]
And you won't need to use Wireshark to read the Dreamcast's response packet, although it won't be timestamped (there's definitely a way to append timestamping via Windows command line, though I don't know it off the top of my head). Also worth noting: this only works on the Windows command line, not PowerShell. I don't know how to do < and > on PowerShell. On Linux I assume these might be | pipes, unless < and > do the same thing there. I know > is the same.

This is outstanding! I've actually been struggling to get some perf tools on the DC itself! I'd resorted to writing simple benchmarking tools that could be placed around code, but this sounds much more like what I'm looking for. So these perf tools are rolled directly into your copy of DCTool? I'm sorry if that's an ignorant question, I'm just trying to wrap my head around how you specifically generated that perf readout. Do you happen to go to that discord server posted above? I'd love to pick your brain in real time.

Excellent work so far, you've got me all excited lol

EDIT: Ahh, I see the perfctr.c/h files and changes to command.c in your source now, I'm setting everything up on my end to test this out. I'm unreasonably excited over this lol
Last edited by ThePerfectK on Tue Nov 26, 2019 7:22 pm, edited 1 time in total.
Still Thinking!~~
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Tue Nov 26, 2019 7:21 pm

Well, oh boy do I have some interesting news.
I was trying to figure out what I could do to solve #2 today, and I discovered something very interesting.

Using the -fno-strict-aliasing flag SLOWS DOWN THE PERF COUNTERS BY 11-12x!!!!!!!!!!
Evidently GCC turns on strict-aliasing by default or something, which does this insane thing. So I'm using no-strict-aliasing from now on because not having it produces downright wrong behavior. Good, so I'm not going crazy. When I typecast I don't mess around, thank you very much. :spazzy: Now I gotta go and fix everything.
ThePerfectK wrote:
Tue Nov 26, 2019 6:51 pm
So these perf tools are rolled directly into your copy of DCTool? I'm sorry if that's an ignorant question, I'm just trying to wrap my head around how you specifically generated that perf readout. Do you happen to go to that discord server posted above?
1. Yes, but your terms are mixed up: dc-load-ip (the part that runs on the Dreamcast), not dc-tool (the part that runs on the PC). I'm able to query dcload-ip without using dc-tool by using ncat/netcat. I don't know if you're familiar with x86 performance counters, but the DC's SH4 has essentially the same thing. I was initially looking for an invariant TSC, which the elapsed time mode of the perf counters basically is. And there are TWO of them.
You can find all the modes they support listed here: http://git.lpclinux.com/cgit/linux-2.6. ... l_sh7750.c
2. Nope! I do get e-mail notifications of PM's, though. If there were a Dreamcast community Slack or something I'd probably be on that. Slack actually saves conversations and has notifications, which are like the big downfalls of standard IRC.
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
User avatar
ThePerfectK
Insane DCEmu
Insane DCEmu
Posts: 127
Joined: Thu Apr 27, 2006 10:15 am
Has liked: 21 times
Been liked: 19 times

Re: dcload-ip with DHCP support!

Post by ThePerfectK » Tue Nov 26, 2019 7:25 pm

Moopthehedgehog wrote:
Tue Nov 26, 2019 7:21 pm
Well, oh boy do I have some interesting news.
I was trying to figure out what I could do to solve #2 today, and I discovered something very interesting.

Using the -fno-strict-aliasing flag SLOWS DOWN THE PERF COUNTERS BY 11-12x!!!!!!!!!!
Evidently GCC turns on strict-aliasing by default or something, which does this insane thing. So I'm using no-strict-aliasing from now on because not having it produces downright wrong behavior. Good, so I'm not going crazy. When I typecast I don't mess around, thank you very much. :spazzy: Now I gotta go and fix everything.
ThePerfectK wrote:
Tue Nov 26, 2019 6:51 pm
So these perf tools are rolled directly into your copy of DCTool? I'm sorry if that's an ignorant question, I'm just trying to wrap my head around how you specifically generated that perf readout. Do you happen to go to that discord server posted above?
1. Yes, but your terms are mixed up: dc-load-ip (the part that runs on the Dreamcast), not dc-tool (the part that runs on the PC). I'm able to query dcload-ip without using dc-tool by using ncat/netcat. I don't know if you're familiar with x86 performance counters, but the DC's SH4 has essentially the same thing. I was initially looking for an invariant TSC, which the elapsed time mode of the perf counters basically is. And there are TWO of them.
You can find all the modes they support listed here: http://git.lpclinux.com/cgit/linux-2.6. ... l_sh7750.c
2. Nope! I do get e-mail notifications of PM's, though. If there were a Dreamcast community Slack or something I'd probably be on that. Slack actually saves conversations and has notifications, which are like the big downfalls of standard IRC.
Sorry for the shorthand confusion with dctool/dctool-ip. I'm getting this all setup on my end to test. I'm familiar with x86 perf counters, so I'm pretty excited to try this out.

Only bummer is that it would seem this would be impossible to use with dctool-serial. Or... hm...

EDIT: wait, I'm dumb lol, I could just cat instead of netcat. Oh man, I gotta try this out, I'm going to try rolling your perf header and command.c changes into the dctool-serial source and see if it works there, too!
Still Thinking!~~
kazade
Insane DCEmu
Insane DCEmu
Posts: 140
Joined: Tue May 02, 2017 3:11 pm
Has liked: 2 times
Been liked: 16 times

Re: dcload-ip with DHCP support!

Post by kazade » Wed Nov 27, 2019 12:16 am

Moopthehedgehog wrote:
Tue Nov 26, 2019 7:21 pm
If there were a Dreamcast community Slack or something I'd probably be on that. Slack actually saves conversations and has notifications, which are like the big downfalls of standard IRC.
Discord is basically Slack - click the link :grin:
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Wed Nov 27, 2019 1:29 am

ThePerfectK wrote:
Tue Nov 26, 2019 7:25 pm

Sorry for the shorthand confusion with dctool/dctool-ip. I'm getting this all setup on my end to test. I'm familiar with x86 perf counters, so I'm pretty excited to try this out.

Only bummer is that it would seem this would be impossible to use with dctool-serial. Or... hm...

EDIT: wait, I'm dumb lol, I could just cat instead of netcat. Oh man, I gotta try this out, I'm going to try rolling your perf header and command.c changes into the dctool-serial source and see if it works there, too!
Here, use the latest versions. I thought I'd have to change more, but it was just a couple comments.
Might as well attach the latest source, too.

Unfortunately, I went and changed everything to account for the GCC 12x speed up bug, and then... IT CAME BACK! I have no idea what the heck GCC is doing, but this basically makes using 4.7.3 impossible. So I'm going to have to upgrade to GCC 8 or 9, I think. There's an SH4-ELF-GCC 8.3.0 and 9.2.1 (Debian Sid only) available it seems. Really annoying. :/

--source removed, still available in first post, but there's newer source there also--
Last edited by Moopthehedgehog on Mon Dec 02, 2019 10:27 pm, edited 1 time in total.
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
User avatar
ThePerfectK
Insane DCEmu
Insane DCEmu
Posts: 127
Joined: Thu Apr 27, 2006 10:15 am
Has liked: 21 times
Been liked: 19 times

Re: dcload-ip with DHCP support!

Post by ThePerfectK » Fri Nov 29, 2019 11:45 pm

Could you please post an example of a raw payload file with a few example commands? I'd like to make sure I'm generating a payload file correctly by comparing to one you used to talk to dcload, as I'm having a bit of trouble sending commands through netcat.
Still Thinking!~~
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Sun Dec 01, 2019 12:14 pm

I did; it's a couple posts up. PMCR_R1.zip has one in it.
I wrote up instructions inside the commands.c file about usage, as well. It's right above the definition of the cmd_pmcr() function:

Spoiler!
commands.c wrote:

Code: Select all

// The 6 perfctr functions are:
/*
void init_pmctr(int which, unsigned short mode); // Clear existing counter and enable
void enable_pmctr(int which, unsigned short mode); // Enable one or both of these "undocumented" performance counters. Does not clear counter(s).
void restart_pmctr(int which, unsigned short mode); // Disable, clear, and re-enable with new mode (or same mode)

unsigned long long int read_pmctr(int which); // 48-bit value needs a 64-bit storage unit

void clear_pmctr(int which); // Only when disabled
void disable_pmctr(int which); // Remember to disable before leaving DCLOAD to execute a program. Does not clear counter(s).
*/
// The command is the first letter of the function (capitalized) plus the counter number, plus a byte with the mode (if applicable)
// Except restart--read is 'R', so restart is 'B' (think reBoot)
// Sending command data of 'D' 0x1 (2 bytes) disables perf counter 1
// Sending command data 'I' 0x3 0x23 (3 bytes) inits both perf counters to elapsed time mode
// ...
// etc.

static char * ok_message = "OK";
static char * invalid_read_message = "PMCR read: 1 or 2 only.";
static char * invalid_function_message = "PMCR: I, E, B, R, C, or D only.";
static char * invalid_option_message = "PMCR: 1, 2, or 3 only.";
static char * invalid_mode_message = "PMCR modes: 0x1-0x29.";


The payload data needs to conform to the command_t structure:
Spoiler!
commands.c again wrote:

Code: Select all

/*
// command_t struct here For reference

typedef struct __attribute__ ((packed)) {
	unsigned char id[4];
	unsigned int address;
	unsigned int size;
	unsigned char data[];
} command_t;
*/


Available modes are in perfctr.h, which, as mentioned in the file, are also listed in a couple places like in the Linux kernel's implementation:
Spoiler!
perfctr.h wrote:

Code: Select all

 *	Event Code	Description
 *	----------	-----------
 *
 *	0x01		Operand read access
 *	0x02		Operand write access
 *	0x03		UTLB miss
 *	0x04		Operand cache read miss
 *	0x05		Operand cache write miss
 *	0x06		Instruction fetch (w/ cache)
 *	0x07		Instruction TLB miss
 *	0x08		Instruction cache miss
 *	0x09		All operand accesses
 *	0x0a		All instruction accesses
 *	0x0b		OC RAM operand access
 *	0x0d		On-chip I/O space access
 *	0x0e		Operand access (r/w)
 *	0x0f		Operand cache miss (r/w)
 *	0x10		Branch instruction
 *	0x11		Branch taken
 *	0x12		BSR/BSRF/JSR
 *	0x13		Instruction execution
 *	0x14		Instruction execution in parallel
 *	0x15		FPU Instruction execution
 *	0x16		Interrupt
 *	0x17		NMI
 *	0x18		trapa instruction execution
 *	0x19		UBCA match
 *	0x1a		UBCB match
 *	0x21		Instruction cache fill
 *	0x22		Operand cache fill
 *	0x23		Elapsed time
 *	0x24		Pipeline freeze by I-cache miss
 *	0x25		Pipeline freeze by D-cache miss
 *	0x27		Pipeline freeze by branch instruction
 *	0x28		Pipeline freeze by CPU register
 *	0x29		Pipeline freeze by FPU


A perf counter number of "3" means apply to both 1 and 2, and is supported in all functions except "read," which only takes 1 or 2 as valid perf counter options.

Note: In making this post I found a couple typos, which I've just fixed in posting this. I also recommend just using a hex editor like HxD to edit that payload data. ncat only sends ASCII, which is why a payload file is needed in the first place.

Edit: Just a note, if you send a bad payload, you should get an error message. If you do it right, ncat will probably respond with "PMCR" followed by what looks like some garbage. That garbage is actually the data, which is stored in raw format. You'll need to open the response in a hex editor or look at the raw packet in Wireshark--I designed remote PMCR stuff so that if someone wants to make a program around it, they can. Doing a remote read, the command->data in the response is transmitted in little endian "host" format, in contrast to the rest of the packet which is in big endian "network" format. Ncat just displays everything in ASCII and doesn't have a "raw" mode...
Last edited by Moopthehedgehog on Mon Dec 02, 2019 10:58 pm, edited 2 times in total.
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Mon Dec 02, 2019 10:03 pm

Just a quick update:
I've managed to build GCC-9.2.0 SH4 and it works! :)
Let me tell you, it's soooo much nicer than GCC 4.7.3, since you can use modern compiler options and directives, plus the warning display is much better. Attaching a rather minimal and hastily written text file of the command line options I used to build it. It should be enough for anyone who's compiled GCC before to do it. Note: my paths are hardcoded to /mnt/c/DreamcastKOS, as I'm using WSLv2 and my dev folder is C:\DreamcastKOS.

I was able to fix some more bugs, this time in the linker script, and between that, adding some flags to the makefile, and changing some global variables, I got the lease time countdown to display across resets! The exception handler actually works, despite example-src's exception-test not working, as that's how I was able to track down alignment problems in the linkerscript. So that's cool. Turns it wasn't a case of "triple-nested if()s" like I mused about earlier--certain variables just need to be in .data since .bss gets wiped by dcload-crt0.s and the registers and stack get totally reset by go.s (and the stack is reset on top of that by dcload-crt0.s).

The only problem left to tackle, since I'm not really intending on overhauling the exception handler binary as part of dcload.bin, is the darn perfcounters still going too quickly. I really don't know what is happening here, or why I once saw it go at what I think should be the "correct" speed. I've not been able to replicate that. I even overhauled the read_pmctr function to use assembly to read the perf counters directly because I caught GCC doing some nasty things (an unconditional branch smack in between reads of the two halves of a uint64?????? REALLLLLYY??????), so I'm wondering if there's some undocumented configuration thing. Hopefully it's not a case of the "it only works right in big endian mode" or something stupid like that.

EDIT: removed source, it's still available in the first post, but there's newer source available now.
Attachments
GCC-binutils config lines.zip
Hastily written notes about compiling sh4 elf gcc 9.2.0
(1.05 KiB) Downloaded 20 times
Last edited by Moopthehedgehog on Wed Dec 04, 2019 11:46 am, edited 1 time in total.
These users liked the author Moopthehedgehog for the post:
mrneo240
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
mrneo240
DCEmu Freak
DCEmu Freak
Posts: 73
Joined: Wed Mar 14, 2018 12:22 am
Has liked: 9 times
Been liked: 10 times

Re: dcload-ip with DHCP support!

Post by mrneo240 » Mon Dec 02, 2019 11:20 pm

Moopthehedgehog wrote:
Mon Dec 02, 2019 10:03 pm
Just a quick update:
I've managed to build GCC-9.2.0 SH4 and it works! :)
Let me tell you, it's soooo much nicer than GCC 4.7.3, since you can use modern compiler options and directives, plus the warning display is much better.

I've recently been using gcc 7.1.3 and its excellent.

Is C++ working with yours? i wasnt able to get that working sadly, so C only at the moment
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Mon Dec 02, 2019 11:47 pm

No clue, sorry; I don't really use C++.

They've actually upgraded the warning system in GCC 9. I wasn't expecting it since I've been using GCC 7 & 8 for other things, but it's a really nice change. A cool unrelated thing about GCC is that on x86-64 it can even regularly beat Intel's C compiler, although it takes longer to compile GCC from source than it does the Linux kernel. :grin:
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
mrneo240
DCEmu Freak
DCEmu Freak
Posts: 73
Joined: Wed Mar 14, 2018 12:22 am
Has liked: 9 times
Been liked: 10 times

Re: dcload-ip with DHCP support!

Post by mrneo240 » Tue Dec 03, 2019 9:27 am

I agree with you on the c++ front, but it's important for many people that c++ be working
User avatar
Moopthehedgehog
DCEmu Freak
DCEmu Freak
Posts: 84
Joined: Wed Jan 05, 2011 4:25 pm
Has liked: 4 times
Been liked: 33 times

Re: dcload-ip with DHCP support!

Post by Moopthehedgehog » Tue Dec 03, 2019 10:51 am

No arguing there. Although because I explicitly compiled with enable-languages=c, I don't have g++ to test, let alone any Dreamcast-related projects that make use of C++ to any real capacity. I don't see offhand why it wouldn't work, but in this situation I can't really test it...

In other news, I came across something: the PMCR_CLKF definition (0x100), which I've never seen used, is decimal value 256. I don't know what it's actually for, but I found some interesting relations:
256/11.96988 = 21.38.., which is extremely close to the number of seconds it's [supposed to] take for the high longword of the performance counters to increment by 1 (in other words, the number of sets of 200,000,000 needed to overflow the low longword). The actual number is 21.47..., and 256/21.47 = 11.920928, which is an interesting number. It's about 1% off from 11.96988, it's apparently a clock frequency for an RTC in ARM according to Google, and also equals 100,000,000 / (8 * 1024^2). That's 200,000,000/2^24 = 11.920928..., where 2^24 is 16,777,216, a very common number and also the square root of 2^48 (perf counter is 48-bit).

There's also 2^28/200,000,000 = 1.34217728, where 2^28 is the smallest power of 2 over 200MHz, and it's 256*1024^2. ((1.34217728)^-1)*16 = 11.920928.

No idea currently if any of this is relevant or a red herring, but that's a lot of overlap going around in circles!
I'm sure Aleron Ives feels weird with his postcount back to <10668
:D
Post Reply