New gcc 12 and gcc-rs (Rust) toolchains

[TapamN]

These aren't merged into KOS yet as there's still some way to go for testing, but I've made repos for two experimental toolchain revisions for anyone who is interested.
(Edit: This GCC 12 toolchain config has been merged into KallistiOS main.)
We have been looking at supporting newer GCC versions for better language support to bring C++20, C23, ObjC, and Rust support for Dreamcast.

First, GCC 12, which has been used by GyroVorbis over the past few months for his libGimbal library development. However, we ran into some snags when reconciling his codebase against the latest KOS and patches, so I'm not entirely sure of the stability yet. It also includes Newlib 4.1.0 (edit: now Newlib 4.3.).
To build this toolchain, follow the typical directions, but check out KOS using the following repo:
git clone -b gcc12 https://github.com/darcagn/KallistiOS.git
and when setting up the toolchain config, choose the new config.mk.latest.sample file as your configuration template:
mv config.mk.latest.sample config.mk




Second, gcc-rs, the experimental GCC Rust compiler. Cepawiel did an excellent job of getting a raw gcc-rs toolchain built with libronin, which he put in a Docker container and uploaded to his repo here. He was able to include a libronin example to print "hello, world" over serial cable. It's just calling C, but it's a breakthrough proof of concept. Following his example, I was able to get a Hello, World! example going on the screen:

However, we ran into snags getting KOS to work instead of the simpler libronin. We were able to break through, tonight, however, and get a KOS toolchain built using gcc-rs. Cepawiel is working on a much better example for this toolchain, but in the mean time, here's how you can build it yourself. Keep in mind there's no standard library support yet, so you're pretty much on your own making bindings to existing stuff.

To build this toolchain, follow the typical directions, but check out KOS using the following repo:

Code: Select all

git clone -b gcc-rs https://github.com/darcagn/KallistiOS.git

when setting up the toolchain config, choose the new config.mk.latest.sample file as your configuration template:

Code: Select all

mv config.mk.latest.sample config.mk

after running ./download.sh and ./unpack.sh, open config.mk once again and change the gcc version from

Code: Select all

sh_gcc_ver=12.2.0

to

Code: Select all

sh_gcc_ver=rs

and finally, clone the gcc-rs repo into the dc-chain folder:

Code: Select all

git clone https://github.com/Rust-GCC/gccrs gcc-rs

then run make to build your gcc-rs toolchain.

[GyroVorbis]

Omg this was an absolute pain in the ass and took 3 of us quite a lot of time, but I'm so stoked. GCC12 was a must have for me as a C++20 concept/constraint abuser for my templates. Not only that, but the freaking SIZE of the binaries created by GCC12 is WAY smaller across the board, even with all debug information and -Og turned on. It's insane!

I've confirmed source level debugging works fine. Using GDB within Qt Creator and can step through code just fine.

I'm currently in the process of testing/validating the toolchain. I have a PR up to get it into KOS here: https://github.com/KallistiOS/KallistiOS/pull/90/files which I will take out of just being a draft once I'm convinced of the toolchain's stability.

I'm in the process of running my massive libGimbal test suite (Github: https://github.com/gyrovorbis/libgimbal), and the output is looking fantastic. Should really be flexing the toolchain:

Code: Select all

KallistiOS Git revision 63d9ff4-dirty:
  Thu Feb  2 12:30:20 AM EST 2023
  falco@slutserv:/opt/toolchains/dc/kos
[...] blah blah
* [GblTestScenario] Beginning Scenario: [libGimbalTests]
        * Local Time          : Tuesday, 12/01/1998 12:22:24 AM                                         
        * LibGimbal Info
                * Version             : 0.1.0 
* Compiler Info
                * C
                        * ID                  : GNU                                                     
                        * Version             : 12.2.0                                                 
                        * Target              : sh-elf                                                 
                        * Language Standard   : 17                    
* ********* Starting TestSuite [GblSortTestSuite] *********
* [ INIT      ]: GblSortTestSuite
* [ RUN       ]: GblSortTestSuite::bubbleSort
* [      PASS ]: GblSortTestSuite::bubbleSort (175.000 ms)
* [ RUN       ]: GblSortTestSuite::selectionSort
* [      PASS ]: GblSortTestSuite::selectionSort (44.000 ms)
* [ RUN       ]: GblSortTestSuite::insertionSort
* [      PASS ]: GblSortTestSuite::insertionSort (81.000 ms)
* [ RUN       ]: GblSortTestSuite::shellSort
* [      PASS ]: GblSortTestSuite::shellSort (26.000 ms)
* [ RUN       ]: GblSortTestSuite::mergeSort
* [      PASS ]: GblSortTestSuite::mergeSort (20.000 ms)
* [ RUN       ]: GblSortTestSuite::quickSort
* [      PASS ]: GblSortTestSuite::quickSort (14.000 ms)
* [ FINAL     ]: GblSortTestSuite
* Totals: 6 passed, 0 failed, 0 skipped, 360.000ms, 0/0 leaked (0/0 bytes)
* ********* Finished TestSuite [GblSortTestSuite] *********
[....] SHITLOADS LATER:
* ********* Finished TestSuite [GblCmdParserTestSuite] *********
* [GblTestScenario] Ending Scenario: [libGimbalTests]
        * Runtime Statistics
                * Total Time (ms)     :             1040.000
                * Max Allocations     :                  108
                * Max Allocation Size :                 6144
                * Max Allocated Bytes :                12960
                * Remaining Allocs    :                   29
                * Remaining Bytes     :                 9812
                * Seed                :            912471746
        * Test Suite Totals
                * Total               :                   40
                * Passed              :                   40
                * Skipped             :                    0
                * Failed              :                    0
        * Test Case Totals
                * Total               :                  810
                * Passed              :                  810
                * Skipped             :                    0
                * Failed              :                    0
* ********************* [   PASS   ] *********************

Funnily enough another thing on the agenda for this toolchain is supporting Objective-C fully with the Foundation runtime (from GNUStep). Got the author of GNUStep offering to help us personally even! (Shut up, Rust people, it's a beautiful language!)

[TapamN]

Oh, wow. I've been having trouble with GCC 9.3.0 being unable to compile my collision detection code (but sh4-linux-gnu-gcc-10, installed though Xubuntu 20.04's apt, seemed to work correctly), so hopefully this will help. And I was looking into what would need to be done to do a Descent port, but one version that looked promising, D2X-XL, required a lot of modern C++ stuff that's too knew for my x64 GCC version to handle. Maybe this GCC version will work...

Does the new version of Newlib fix the clock handling errors? KOS creates garbage timestamps for VMU saves, and I think it's Newlib's fault, since localtime() would produce random output for a flash inspection program I wrote.

[GyroVorbis]

Oh, wow. I've been having trouble with GCC 9.3.0 being unable to compile my collision detection code (but sh4-linux-gnu-gcc-10, installed though Xubuntu 20.04's apt, seemed to work correctly), so hopefully this will help. And I was looking into what would need to be done to do a Descent port, but one version that looked promising, D2X-XL, required a lot of modern C++ stuff that's too knew for my x64 GCC version to handle. Maybe this GCC version will work...

I can 100% tell you that there are several bugfixes between 9.3.0 and 12.2.0 for things like ICE and other issues that you can see by looking at the repo. I was able to confirm that an ICE scenario found by MrNeo was fixed in 12 for SH. HOPEFULLY this is the case.

Does the new version of Newlib fix the clock handling errors? KOS creates garbage timestamps for VMU saves, and I think it's Newlib's fault, since localtime() would produce random output for a flash inspection program I wrote.

Whoah, I just wrote a whole Date/Time utility for libGimbal and ran it on DC for the first time, so I have some information and can help validate this.

First when the test suites start up, I'm printing the DC timestamp using localtime():

Code: Select all

* Local Time          : Tuesday, 12/01/1998 09:45:19 AM

I tried again a few minutes later and got this timestamp:

Code: Select all

* Local Time          : Tuesday, 12/01/1998 09:50:40 AM   

which looks like it's correct based on my mostly dead-ass LR2032 BIOS battery. It's consistent so it's not garbage. I'll set it to something more obvious and change the battery and verify, though.

Then I have a bunch of test cases for random stuff:

Code: Select all

* ********* Starting TestSuite [GblDateTimeTestSuite] *********
* [ INIT      ]: GblDateTimeTestSuite
* [ RUN       ]: GblDateTimeTestSuite::dateIsLeapYear
* [      PASS ]: GblDateTimeTestSuite::dateIsLeapYear (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateMonthDays
* [      PASS ]: GblDateTimeTestSuite::dateMonthDays (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateMonthString
* [      PASS ]: GblDateTimeTestSuite::dateMonthString (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateWeekDayString
* [      PASS ]: GblDateTimeTestSuite::dateWeekDayString (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateToJulian
* [      PASS ]: GblDateTimeTestSuite::dateToJulian (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateWeekDay
* [      PASS ]: GblDateTimeTestSuite::dateWeekDay (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateYearDayAndWeek
* [      PASS ]: GblDateTimeTestSuite::dateYearDayAndWeek (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateFromJulian
* [      PASS ]: GblDateTimeTestSuite::dateFromJulian (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateFromOrdinal
* [      PASS ]: GblDateTimeTestSuite::dateFromOrdinal (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::timeStr
* [      PASS ]: GblDateTimeTestSuite::timeStr (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::timeInitDefault
* [      PASS ]: GblDateTimeTestSuite::timeInitDefault (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::timeIsPm
* [      PASS ]: GblDateTimeTestSuite::timeIsPm (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeInitDefault
* [      PASS ]: GblDateTimeTestSuite::dateTimeInitDefault (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeIsUtc
* [      PASS ]: GblDateTimeTestSuite::dateTimeIsUtc (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeNormalizeDefault
* [      PASS ]: GblDateTimeTestSuite::dateTimeNormalizeDefault (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeEquals
* [      PASS ]: GblDateTimeTestSuite::dateTimeEquals (3421.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeFromUnix
* [      PASS ]: GblDateTimeTestSuite::dateTimeFromUnix (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeFromLocal
* [      PASS ]: GblDateTimeTestSuite::dateTimeFromLocal (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeFromUtc
* [      PASS ]: GblDateTimeTestSuite::dateTimeFromUtc (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeNowLocal
* [      PASS ]: GblDateTimeTestSuite::dateTimeNowLocal (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeNowUtc
* [      PASS ]: GblDateTimeTestSuite::dateTimeNowUtc (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeParse
* [      PASS ]: GblDateTimeTestSuite::dateTimeParse (1.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeToUnix
* [      PASS ]: GblDateTimeTestSuite::dateTimeToUnix (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeToLocal
* [      PASS ]: GblDateTimeTestSuite::dateTimeToLocal (1.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeToUtc
* [      PASS ]: GblDateTimeTestSuite::dateTimeToUtc (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeFormat
* [      PASS ]: GblDateTimeTestSuite::dateTimeFormat (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeDiff
* [      PASS ]: GblDateTimeTestSuite::dateTimeDiff (1.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddDays
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddDays (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddHours
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddHours (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddMinutes
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddMinutes (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddSeconds
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddSeconds (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddWeeks
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddWeeks (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddMonths
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddMonths (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeAddYears
* [      PASS ]: GblDateTimeTestSuite::dateTimeAddYears (0.000 ms)
* [ RUN       ]: GblDateTimeTestSuite::dateTimeToIso8601
* [      PASS ]: GblDateTimeTestSuite::dateTimeToIso8601 (0.000 ms)
* [ FINAL     ]: GblDateTimeTestSuite
* Totals: 35 passed, 0 failed, 0 skipped, 3424.000ms, 0/2 leaked (0/261 bytes)
* ********* Finished TestSuite [GblDateTimeTestSuite] *********

Everything seems to be working fine, but I would note something important here about Newlib's time stuff:

Code: Select all

* [ RUN       ]: GblDateTimeTestSuite::dateTimeEquals
* [      PASS ]: GblDateTimeTestSuite::dateTimeEquals (3421.000 ms)

Look how goddamn SLOW this test case is. One of the slowest tests of everything I've run on the DC. The test case code is:

Code: Select all

GBL_TEST_CASE(dateTimeEquals) {
    GblDateTime dt = { 0 };
    GBL_TEST_VERIFY(GblDateTime_equals(&dt, &dt));

    GBL_TEST_VERIFY(GblDateTime_normalize(&dt));
    GBL_TEST_VERIFY(GblDateTime_equals(&dt, &dt));

    gblRandBuffer(&dt, sizeof(dt));
    GBL_TEST_VERIFY(GblDateTime_equals(&dt, &dt));

    GBL_TEST_CASE_END;
}

GblDatetTime_equals() is basically just calling standard C's "difftime" using the unix epoch representation of the time. Notice the last thing I'm doing is generating a random timestamp to make sure it's always handled properly. Newlib does properly return the -1 error for unix time of an invalid timestamp and never crashes or does anything bad, but OH MY GOD, it's SO SO SO slow when you have bad timestamps. This one test case took a whopping 7 seconds straight on one run and just froze the DC for the entire time... so beware, the implementation seems robust, but it can be fucking slow.

[TapamN]

That sounds promising. I'm going to try GCC 12 tomorrow.

For reference, the error I've been getting in my collision detection code is this. (I can't include it directly in this post because the forum gives me server errors when I try to submit)

What values cause such long freezes on difftime? It sounds like it's worth profiling it to figure out what's going on.

Does kos-cc still pass -fno-builtin to GCC 12?

On the version of KOS I have setup, kos-cc passes KOS_CFLAGS (from environ.sh) to sh-elf-gcc, and KOS_CFLAGS has -fno-builtin in it. This prevents GCC from doing things like precalculating calls to math functions with constant inputs, converting 1/sqrtf(x) to an FSRRA instruction with -mfsrra, or inlining memcpy/memset with small sizes.

Presumably, there was a reason fno-builtin was required (Maybe just to work around a compiler bug? Does someone here know for sure?), but has anyone checked if KOS on GCC 12 can work without it? Even with 9.3, manually calling the __builtin_ versions of library functions seems to work correctly where I've tried it, but I haven't tried compiling a whole KOS without fno-builtin. It'd be nice if kos-cc would allow built-ins by default.

[GyroVorbis]

Update: Just pushed new patches and new "latest" config file to the GCC 12 branch.

'latest' toolchain config versions bumped: binutils to 2.40, newlib to 4.3.0, gdb to 12.1, binutils (ARM) to 2.40; toolchain patches for newlib 4.3.0
This is the latest version of everything.
It builds KOS and runs 2ndmix example. More testing needed.

with a raw non-kos toolchain with the same versions, I can also compile libronin with openlara, it runs and works fine.


Update 2: same version bumps have now been applied to the gcc-rs branch.
spencer723 was able to get a "hello, world" Rust example working with KOS and debug it with gdb using CLion.

[GyroVorbis]

I'm going to look deeper into the difftime issue, and I'll let you know what specifically causes the slowdown.

On the version of KOS I have setup, kos-cc passes KOS_CFLAGS (from environ.sh) to sh-elf-gcc, and KOS_CFLAGS has -fno-builtin in it. This prevents GCC from doing things like precalculating calls to math functions with constant inputs, converting 1/sqrtf(x) to an FSRRA instruction with -mfsrra, or inlining memcpy/memset with small sizes.

Presumably, there was a reason fno-builtin was required (Maybe just to work around a compiler bug? Does someone here know for sure?), but has anyone checked if KOS on GCC 12 can work without it? Even with 9.3, manually calling the __builtin_ versions of library functions seems to work correctly where I've tried it, but I haven't tried compiling a whole KOS without fno-builtin. It'd be nice if kos-cc would allow built-ins by default.

HOLY SHIT. Good to know!!! I'm working on a PR to add a dreamcast.toolchain.cmake file to KOS, and one of the things it does do for you is set up the correct flag incantation for SH4 math... but we're literally right now debating on whether to directly set the flags and call the compiler ourself or to use the KOS gnu_wrappers... I had no idea that's what -fno-builtin was doing. This absolutely should get looked into. Thanks!

[GyroVorbis]

For reference, the error I've been getting in my collision detection code is this. (I can't include it directly in this post because the forum gives me server errors when I try to submit)

Daaaaamn.That's a serious ICE! I'm not positive if that's fixed or not, but I'd love to see. Definitely there are a few commits in the delta between 9.3.0 and 12.2.0 addressing things like this.

I'd love to see wtf your collision algorithm looks like that owned the compiler so badly.

[GyroVorbis]

OKAY. I've been throwing everything I possibly could at GCC12 tonight + latest and greatest binutools and newlib--making my DC build toolchain more modern than my main desktop.

I tested with libGimbal, I tested with nearly every example in KOS, I built the KOS-ports, I grabbed GLdc and built that, I built Ian Michael's SDL/GLdc interop fork, I grabbed the game VVVVV and built that. I played with various optimization levels, I tried out stepping through the debugger, etc...

The ONLY thing is that I found one goddamn ICE. It's an unfortunate one because it happens to be in the SDL codebase, so I confirmed it happens when building SDL from 3 different codebases (kos-ports, Ian Michael, version bundled with VVVVV). When you get to compiling events/SDL_mouse.c, the compiler will just hang indefinitely, utilizing increasing amounts of RAM until it's killed by the system.

The workaround is so damn trivial. Multiple ways. Instead of building with -O3 or or -O4, build with anything else. The other way is to modify the code. I isolated it to this code, an assignment on line #231:

Code: Select all

/* These are static for our mouse handling code */
static Sint16 SDL_MouseX = 0;
static Sint16 SDL_MouseY = 0;
static Sint16 SDL_DeltaX = 0;
static Sint16 SDL_DeltaY = 0;
static Sint16 SDL_MouseMaxX = 0;
static Sint16 SDL_MouseMaxY = 0;
static Uint8  SDL_ButtonState = 0;

/* These are global for SDL_eventloop.c */
int SDL_PrivateMouseButton(Uint8 state, Uint8 button, Sint16 x, Sint16 y)
{
    //..............
       /* Figure out which event to perform */
	buttonstate = SDL_ButtonState; //<======= OFFENDING LINE =========
	switch ( state ) {
		case SDL_PRESSED:
			event.type = SDL_MOUSEBUTTONDOWN;
			buttonstate |= SDL_BUTTON(button);
			break;
		case SDL_RELEASED:
			event.type = SDL_MOUSEBUTTONUP;
			buttonstate &= ~SDL_BUTTON(button);
			break;
		default:
			/* Invalid state -- bail */
			return(0);
	}

    //..........

Commenting out the offending line gets rid of the ICE. Changing the type of SDL_ButtonState to anything sized larger than a byte fixes the ICE. Compiling with -O2 or lower fixes the ICE. This is literally the only issue I've found, and I was not even able to reproduce it in a separate project trying to construct something similar, but it still sucks. Just wanted to make everyone aware of it.

If anybody else finds an ICE, please post it so we can track these things. Also, GCC13 is about to come out, which will unify GCC12 and the experimental Rust branch, so perhaps this is gone there.

Does the new version of Newlib fix the clock handling errors? KOS creates garbage timestamps for VMU saves, and I think it's Newlib's fault, since localtime() would produce random output for a flash inspection program I wrote.

Okay, so this time I let my BIOS battery die and set it to something more interesting and obvious to inspect:

Code: Select all

        * Local Time          : Sunday, 07/23/1989 07:17:45 AM   

I got back exactly what I set to the minute, so it appears to be completely working.

[|darc|]

This was pointed out in chat, but for those reading along in the thread here, -O2 is actually the default, so if we have no examples of -O2 causing problems and can only find one example of -O3 tripping up like that, that's damn fine showing for the new toolchain on GCC 12, newlib 4.3.0, and binutils 2.40.

KOS and dc-chain will likely be moving to 4.7.4 as legacy, 9.3.0 as stable, 12.2.0 as testing, and potentially new future bleeding edge versions of everything as latest.

[TapamN]

I'd love to see wtf your collision algorithm looks like that owned the compiler so badly.

It's for a K-d tree. The part that has problems ids comparing a single float against either X, Y, or Z of a 3D vector. But which axis it compares to is selected by the lowest 2 bits of the float. A simplified version of the problem code is this:

Code: Select all

int WhichSide(float kd_plane, float *point) {
	uint32_t raw_plane_bits = get_raw_bits(kd_plane);
	return point[plane_bits & 0x3] > kd_plane;
}

I tried compiling my 3D game engine and Gens4All with GCC 12. It all works fine, compiled with -O2. VMU save files finally have correct timestamps! I tried Gens4All with -O3, and that also works, and one part got a 25% speedup, and another got a 5% slowdown.

I tried compiling without fno-builtin, and got an interesting error:

Code: Select all

fs_ext2.c: In function ‘fs_ext2_symlink’:
fs_ext2.c:1586:9: error: ‘strncpy’ specified bound 60 equals destination size [-Werror=stringop-truncation]

Whoops. I guess some warnings get bypassed with fno-builtin. That looks like a fairly important warning, too, since it writes to the FAT! But it's involved in making symlinks, which few (if any) people would try to do in KOS. Maybe the code is correct, and it's OK if the null terminator gets dropped? I don't know the specs of symlinks for ext2.

I replaced that line with assert(0) on my system, just to get it to compile, and finished recompiling KOS, then tested Gens4All and my engine. Both ran without problems.

I also experimented with compiling with -mrelax, which converts MOV.L/JSR combinations to faster, smaller BSR instructions. Given that what looks like to be the last attempt at getting mrelax to work in GCC didn't fully succeed, I wasn't optimistic. And it didn't fully succeed for me, but it didn't fully fail, either.

KOS and my engine compiled and ran correctly, although linking generated several warnings. It seems like it was a warning for each place it converted a JSR to a BSR. Gens4All would crash.

Where it crashed is interesting.

Code: Select all

	case VDP_SET2_REG:
		Update_IRQ_Line();
		break;
	case VDP_SET3_REG:
		H_Scroll_Mask = H_Scroll_Mask_Table[val & 3];

It was inside a switch statement, which was converted to a branch table, where a function call higher up in the switch statement (Update_IRQ_Line) was converted to a BSR. The crash was caused by the address loaded for the H_Scroll_Mask_Table being... 0x7, causing an unaligned read exception.

The generated code in the switch statement was correct. It loaded the correct address for H_Scroll_Mask_Table, but the register dump listed the wrong value. It seems like it was jumping too far into the code, so H_Scroll_Mask_Table was never loaded. I guess GCC has a bug involving relaxation and branch tables. When the code shrinks after relaxation, branch tables don't get updated, or something like that? I didn't try to work out exactly what address it jumped to.

In short, it's best to keep mrelax off, but you can maybe get away with it sometimes if you're feeling lucky.

[BlueCrab]

I tried compiling without fno-builtin, and got an interesting error:

Code: Select all

fs_ext2.c: In function ‘fs_ext2_symlink’:
fs_ext2.c:1586:9: error: ‘strncpy’ specified bound 60 equals destination size [-Werror=stringop-truncation]

Whoops. I guess some warnings get bypassed with fno-builtin. That looks like a fairly important warning, too, since it writes to the FAT! But it's involved in making symlinks, which few (if any) people would try to do in KOS. Maybe the code is correct, and it's OK if the null terminator gets dropped? I don't know the specs of symlinks for ext2.

Symlinks don't require the NUL terminator since the size of the field is always 60... That said, I guess I do need to fix that at some point...

[GyroVorbis]

I'd love to see wtf your collision algorithm looks like that owned the compiler so badly.

It's for a K-d tree. The part that has problems ids comparing a single float against either X, Y, or Z of a 3D vector. But which axis it compares to is selected by the lowest 2 bits of the float. A simplified version of the problem code is this:

Code: Select all

int WhichSide(float kd_plane, float *point) {
	uint32_t raw_plane_bits = get_raw_bits(kd_plane);
	return point[plane_bits & 0x3] > kd_plane;
}

Dude, what in the *actual*... That's both hardcore C and hardcore math in terms of conceptualizing wtf that code is even doing.

I tried compiling my 3D game engine and Gens4All with GCC 12. It all works fine, compiled with -O2. VMU save files finally have correct timestamps! I tried Gens4All with -O3, and that also works, and one part got a 25% speedup, and another got a 5% slowdown.

That's freaking awesome! I was hoping the toolchain would work for you after all the work we did to get it here!!!

Also right before the check-in, Crabby spotted an old bug that was the reason why the timestamps wouldn't have worked correctly on 9.3.0. The #define which was supposed to give it the proper size, long long, was never enabled called (it was checking for newlib >3 when it was newlib 3), and instead the default newlib size (which must've been wrong) was used. The GCC12 checkin should've also fixed that issue for GCC9.

I also experimented with compiling with -mrelax, which converts MOV.L/JSR combinations to faster, smaller BSR instructions. Given that what looks like to be the last attempt at getting mrelax to work in GCC didn't fully succeed, I wasn't optimistic. And it didn't fully succeed for me, but it didn't fully fail, either.

KOS and my engine compiled and ran correctly, although linking generated several warnings. It seems like it was a warning for each place it converted a JSR to a BSR. Gens4All would crash.

Where it crashed is interesting.

Code: Select all

	case VDP_SET2_REG:
		Update_IRQ_Line();
		break;
	case VDP_SET3_REG:
		H_Scroll_Mask = H_Scroll_Mask_Table[val & 3];

It was inside a switch statement, which was converted to a branch table, where a function call higher up in the switch statement (Update_IRQ_Line) was converted to a BSR. The crash was caused by the address loaded for the H_Scroll_Mask_Table being... 0x7, causing an unaligned read exception.

The generated code in the switch statement was correct. It loaded the correct address for H_Scroll_Mask_Table, but the register dump listed the wrong value. It seems like it was jumping too far into the code, so H_Scroll_Mask_Table was never loaded. I guess GCC has a bug involving relaxation and branch tables. When the code shrinks after relaxation, branch tables don't get updated, or something like that? I didn't try to work out exactly what address it jumped to.

In short, it's best to keep mrelax off, but you can maybe get away with it sometimes if you're feeling lucky.

Thank you for this! I've never even hard of -mrelax, and I want to be more rigorous about documenting the impact of various flags and issues with this version of the toolchain, so that we can be on the lookout for patches to pick up from the repo or even bugs to file (ICEs do actually get taken care of for SH still). I'd love to put some of this on the wiki once we get more data coming in.

By the way, not sure if you knew about it, but we have everything in compiler explorer now, for easily checking out disassembly for Dreamcast on the go: https://dreamcast.wiki/SH4_in_Compiler_Explorer. GCC12.2.0 is the exact latest SH toolchain on there too, so it's very nice for us. That wiki page might be where I compile more of these notes...

OH, another thing to report. I haven't been doing the kind of hardcore runtime ASM optimization stuff like you have, but I've been very interested in specifically the size differences in binaries output by GCC12 versus the older toolchains. Ian Michael keeps detailed notes on what compiler configurations produce what library sizes for KOS at which flags, and it looks as though GCC12 beats the best we had (GCC4, IIRC) at every optimization level. Here's a table I made for libKallisti.a's sizes in 12:

Code: Select all

-O2: 3.4MB
-O3: 3.6MB
-Os: 3.2MB

Note that GCC4 was producing 3.5MB at -O2. Perhaps Ian can share the rest of his data for a complete table.

Another thing that can be looked into/played with now that we have the newest toolchain in KOS is the fact that Newlib actually has different configuration options now for libc. You can evidently build it for minimal size. Perhaps this should be added as a flag to the configuration script for GCC12 for people to play around with? It also lets you enable full C11 sized printf formatters, which probably nobody except my printf()-happy, C17 ass would probably care about...

[GyroVorbis]

Omg you want to see some epic, crazy shit you can do with this toolchain? I built the Objective-C runtime with it (I'll add it once we get the stdlib ported), and I can do this:

Code: Select all

#include <objc/objc.h>
#import <objc/Object.h>
#import <objc/runtime.h>
#include <concepts>
#include <iostream>

template <typename T>
concept ILabel = requires(T v)
{
    {v.toString()} -> std::convertible_to<std::string>;
};

template<typename... Args>
class Stringifier { 
    int i = 0;
public:
    std::string toString(void) {
        return "Test String";
    }
};

@interface Person: Object 
 {
    int douchewezel;
 }
- (void)test;
+ (void)initialize;
@end

@implementation Person
+ (void) initialize {
    printf("Objective-C static constructors work!\n\n");
    if constexpr(ILabel<Stringifier<int, float, char>>) { 
        std::cout << "C++20 Concepts + iostream works!" << std::endl;
    }
}

 - (void)test {
    printf("ObjC method dispatch with class reflection work %s\n", class_getName([self class]));
    fflush(stdout);
}  
@end

int main(int argc, char **argv) {
    printf("Objective C Class Location!: %p", [Person class]);
    Person* person = class_createInstance(objc_getClass("Person"), 0);
    printf("Objective C Instance Location = %p", person);
    [person test];

    printf("C API works %s\n", class_getName(object_getClass(person)));

    printf("ObjC method again: %d\n", [person isEqual:person]);
    fflush(stdout);
    return 0;
}

That would be C17, C++20, and Objective-C all combined together into one .mm file as Objective-C++ building and running fine on a Dreamcast.

[Ian Robinson]

Updated GDB in the dc-chain script to 13.1 (released February 19) for the testing/12.2.0 config.