KOS - KOS-PORTS mismatch?

[Chilly Willy]

I've recently updated to Xubuntu 16.04 and slowly rebuilding my cross compilers, and I've spent the weekend on KOS. Using this page, I was able to get kos and kos-ports to compile, but none of the examples using kos-ports work because they can't find the includes and libs. The "new" design for kos-ports means every lib has a soft-linked subdirectory for its includes, but the KOS_INC_PATHS var doesn't handle that. Do I have to change every #include to add all the soft-linked subdir pathing now?

[BlueCrab]

If you're using an up-to-date checkout of KOS, the default environ script should be setting up the include and library paths for you (it's set in environ_base.sh), so long as the KOS_PORTS value in the environ.sh is pointing to the right place (if you have your kos directory right next to kos-ports, then it should be correct by default).

I can't say anything about the guide you've followed, as I've not personally looked at it. That's bogglez' guide, so perhaps he can help out? The canonical "official" guide to setting KOS up is over here, although I haven't actually publicly placed that link anywhere before now (mainly because it was a WIP and it's referring to a release of KOS that I haven't actually had the chance to finish up with yet ). That said, a person over in the #dreamcastdev channel was able to follow it and get things up and running relatively easily, so that's a good sign.

Note: I need to update the prerequisite packages list slightly still -- libpng-dev needs to be added to it. That one was apparently installed by default on my Raspberry Pi, so I missed it initially.

[Chilly Willy]

The sample environ.sh has pristine as the subarch instead of DC. Is that fine? I always enable 'navi' instead.

Both pages have equivalent instructions.

As to what I mean about the includes, let's look at SDL as an example. After running build-all.sh, you have

/opt/toolchains/dc/kos-ports/include/~SDL
which is a softlink to
/opt/toolchains/dc/kos-ports/SDL/inst/include
What is inside that directory? Another include directory. And inside THAT directory are the SDL includes.

So anything using KOS_INC_PATHS will only get as far as
/opt/toolchains/dc/kos-port/include
when looking for includes. That's from the environ-base.sh file where it includes
-I${KOS_PORTS}/include

There's no way the existing includes can reach the SDL includes because they haven't been copied to a place that's part of KOS_INC_PATHS, and the softlink to where they are doesn't reach deep enough. Any time I try to assemble an example that uses any library from kos-ports, it errors out saying the kos-port includes and libraries can't be found.

example

Code: Select all

jlfenton@Mach-5:/opt/toolchains/dc/kos/examples/dreamcast/sdl/sound$ make
rm -f sdl_sound.elf example.o
kos-cc  -c example.c -o example.o
example.c:18:21: fatal error: SDL/SDL.h: No such file or directory
compilation terminated.
/opt/toolchains/dc/kos/Makefile.rules:13: recipe for target 'example.o' failed
make: *** [example.o] Error 1

[BlueCrab]

That's definitely odd, considering I just built the SDL port myself and the directory structure is correct... $KOS_PORTS/include/SDL is a symlink to $KOS_PORTS/SDL/inst/include, which directly houses all of the SDL header files. There definitely shouldn't be an extra "include" directory inside there.

Is that happening on all ports for you, or just SDL? It'd be very odd either way, but moreso if it's only happening on one library in there...

[Chilly Willy]

Okay, I cleaned up, made sure I was up to date (I am), did a rebuild, then checked every lib. It's SDL and tsunami... which were coincidentally the two examples I tried to build. Both have inst/include/include as the include paths. I didn't see any others with that issue, so it's not nearly as bad as I thought.

[Chilly Willy]

Okay, found the problem. It's in build.mk. You have this for copying an an entire directory of includes

Code: Select all

	if [ -n "${HDR_DIRECTORY}" ] ; then \
		cp -r ${HDR_DIRECTORY}/ ../../inst/install ; \
	fi

But that's an improper usage of cp. "cp -r dir1 dir2" doesn't copy the CONTENTS of dir1 into dir2, it copies dir1 itself and all subdirectories into dir2. So you wind up with dir2/dir1/whatever.h. You need to alter it like this

Code: Select all

	if [ -n "${HDR_DIRECTORY}" ] ; then \
		cp -r ${HDR_DIRECTORY}/ ../../inst ; \
	fi

and then it works fine for SDL and libtsunami.

I'm not sure if that's the best way to fix it since if the source directory isn't named "include", you'll wind up with it not being soft-linked correctly.

[BlueCrab]

I'm... not sure why that wouldn't work... From the cp manpage on my machine:

Code: Select all

     -R    If source_file designates a directory, cp copies the directory and
           the entire subtree connected at that point.  If the source_file
           ends in a /, the contents of the directory are copied rather than
           the directory itself.  This option also causes symbolic links to be
           copied, rather than indirected through, and for cp to create spe-
           cial files rather than copying them as normal files.  Created
           directories have the same mode as the corresponding source direc-
           tory, unmodified by the process' umask.

That's why the first argument ends in a '/' character -- since that should do exactly what I want, regardless of what the directory name is.

I guess this could be a difference between *BSD/OS X and Linux here, but I know I've used the same trick on Linux machines before...

[Chilly Willy]

Yeah, it's a BSD vs linux problem. The man for cp for linux is not at all like that. Linux cp works like this:
http://www.gnu.org/software/coreutils/m ... ation.html

In particular, look at this:
http://www.gnu.org/software/coreutils/m ... ctory.html

The cp, install, ln, and mv commands normally treat the last operand specially when it is a directory or a symbolic link to a directory. For example, ‘cp source dest’ is equivalent to ‘cp source dest/source’ if dest is a directory. Sometimes this behavior is not exactly what is wanted, so these commands support the following options to allow more fine-grained control:

They talk about quite a bit on those pages, but they're trying to be more POSIX compliant. Anywho, the copy command may need to be made a macro that people can choose for BSD vs linux.

[BlueCrab]

I think I have a way that should fix it on either (or at least this works on OS X and some references indicate that it should work on Linux as well)...

Try changing

Code: Select all

	if [ -n "${HDR_DIRECTORY}" ] ; then \
		cp -r ${HDR_DIRECTORY}/ ../../inst/include ; \
	fi ; \

To this:

Code: Select all

	if [ -n "${HDR_DIRECTORY}" ] ; then \
		cp -r ${HDR_DIRECTORY}/. ../../inst/include ; \
	fi ; \

That is to say, add a single dot after the / for the source of the cp command. See if that works out OK on Linux, and if so, I'll go ahead and commit that change.

[bogglez]

Hey guys, sorry for the late response.

Regarding my install script:
I intentionally kept the script very simple to not introduce new problems.
It uses the standard configuration and just executes the steps described in previous install tutorials.
I only added some safeguards such as not cloning the repositories again, checking for some packages and asking to install them, if possible.
I also chmod the installation directory at the very end, so that you can't accidentally delete stuff or otherwise corrupt the installation during development.

I created this script for convenience and for threads like this one, so that we can be sure that the bug reporter performed the installation correctly. In the past we had a lot of posts where we found out that some step was forgotten or not performed correctly.

[Chilly Willy]

I think I have a way that should fix it on either (or at least this works on OS X and some references indicate that it should work on Linux as well)...

Try changing

Code: Select all

	if [ -n "${HDR_DIRECTORY}" ] ; then \
		cp -r ${HDR_DIRECTORY}/ ../../inst/include ; \
	fi ; \

To this:

Code: Select all

	if [ -n "${HDR_DIRECTORY}" ] ; then \
		cp -r ${HDR_DIRECTORY}/. ../../inst/include ; \
	fi ; \

That is to say, add a single dot after the / for the source of the cp command. See if that works out OK on Linux, and if so, I'll go ahead and commit that change.

That worked great, and is better than my "fix". Thanks for looking into this.

Oh, one more question - is gcc 5 still a problem? I tried this with 5.3.1 (which comes with Ubuntu 16.04) and everything worked other than the issues we've been dealing with. I installed 4.9 and rebuilt everything when I first ran into the problems, but that wasn't the problem as we've seen here. So it seems like maybe 5.3.x might be acceptable. I didn't try everything, of course, but my old Doom port worked just peachy compiled with a toolchain made by 5.3.1.

[BlueCrab]

The GCC version used to compile the toolchain shouldn't matter at this point. A patch was provided to make it work with GCC 5.x.

As for the GCC version used for the toolchain itself, I'd have to actually test out newer versions of GCC to make any comment on them. I haven't had the time nor energy to do it. Unfortunately, they still seem to break minor targets like SuperH on a regular basis, based upon various bug reports that I've seen... Also, I'm not sure there's any real compelling reason to work with a newer version of GCC at this point either.

[nymus]

Hah! Interesting... I developed the habit of deleting that trailing slash whenever I tab-completed a directory but I don't know where it came from.

Also the "-r" option is deprecated everywhere! We should be using "-R!" I've been doing it WRONG! all this time! LOL

The POSIX specification, does not mention anything about a trailing slash in the source and my tests on NetBSD show that the trailing slash is not special; the "source" is created at "target" NOT the contents of source. OpenBSD man page also doesn't mention this. As such, it doesn't appear to be a "BSD" convention.

The trailing slash in source definitely copies the contents on my Mac (Lion) and it is explicitly mentioned in the both Mac and FreeBSD manuals (naturally, because Mac uses FreeBSD's userland)

-R
If source_file designates a directory, cp copies the directory and
the entire subtree connected at that point. If the source_file
ends in a /, the contents of the directory are copied rather than
the directory itself. This option also causes symbolic links to be
copied, rather than indirected through, and for cp to create spe-
cial files rather than copying them as normal files. Created
directories have the same mode as the corresponding source direc-
tory, unmodified by the process' umask.

It also doesn't work on Linux, or rather, Linux also complies with the standard in this case... (surpisingly, [RANT] because GNU usualy adds "features" and changes many conventions and standards. I had a particularly frustrating battle with (GNU Make 3.81) on my Mac where it just would not do what the manual says it should when assigning != (a shell command to a variable). Nothing was happening! I couldn't figure out what I was doing wrong! Did I misread the manual? Should I try := instead? What??!! After so much agony, I thought, hmm let me try this on NetBSD whose manual regarding these assignments matches the GNU manual, and Lo and Behold! It works "as expected!" Seriously?! I mean, I was trying out a new thing so knew I would cause errors but I expect them to be intuitive and explicable, especially when the manual says something should work. ARRGH.

Anyway, assuming it's not just my system with this issue, I would advise using the following pattern to assign shell commands to variables on GNU Make: "VAR := $(shell cmd [options])"

Oh, is there a specific reason for avoiding *?:

cp -R ${HDR_DIRECTORY}/* ../../inst/include

[BlueCrab]

Oh, is there a specific reason for avoiding *?:

cp -R ${HDR_DIRECTORY}/* ../../inst/include

* will not catch any files/directories that might start with a . character. Not that we should have any of those to deal with, but it doesn't hurt to deal with them just in case.

Glad to hear the . plan does work though. I'll go ahead and commit that (and change the -r to a -R too, just for good measure).

For once my putting off a release has actually helped find a bug before I release something buggy.

[Chilly Willy]

The GCC version used to compile the toolchain shouldn't matter at this point. A patch was provided to make it work with GCC 5.x.

As for the GCC version used for the toolchain itself, I'd have to actually test out newer versions of GCC to make any comment on them. I haven't had the time nor energy to do it. Unfortunately, they still seem to break minor targets like SuperH on a regular basis, based upon various bug reports that I've seen... Also, I'm not sure there's any real compelling reason to work with a newer version of GCC at this point either.

Good to hear that folks don't NEED to install 4.9 to avoid 5.x any more. As to getting 5.x for the toolchain, I'd probably wait until they've got the major improvements going for all the targets rather than just x86/AMD64/ARM. Those tend to be the initial targets, and all others suffer for a while until things stabilize. Probably don't need to look at it until 5.7 or 5.8 or so.