pthread and fs_read slowdown

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Post Reply
User avatar
Orion_
DCEmu Crazy Poster
DCEmu Crazy Poster
Posts: 31
Joined: Fri Nov 21, 2014 3:31 pm
Location: France
Has liked: 0
Been liked: 0
Contact:

pthread and fs_read slowdown

Post by Orion_ » Wed Mar 15, 2017 9:38 am

I try to load data from CD with a thread running in background and it's damn slow ...

When I load file from CD to ram without any thread started, it's fast, as it used to be.
Now when I start a thread (which itself will initialize an ogg player which seems to start another thread), when I load data from CD in the main thread, it's 7 times slower !
here is my test log:

Code: Select all

fs_copy /cd/dreamcast/snd/01.ogg to /ram/01.ogg
Time: 2195ms
fs_copy /cd/dreamcast/snd/02.ogg to /ram/02.ogg
Time: 2957ms
fs_copy /cd/dreamcast/snd/03.ogg to /ram/03.ogg
Time: 522ms
fs_copy /cd/dreamcast/snd/04.ogg to /ram/04.ogg
Time: 2746ms

-> fs_ramdisk_detach 01->04.ogg

-> pthread_create
-> Thread talking: init ogg player
snd_init(): loading 3336 bytes into SPU RAM
sndserver: initializing sndoggvorbis 0.7 [OggVorbis 1.0 based]
sndserver: pid is 5
Creating semaphore with deprecated sem_create(). Please update your code!
snd_stream: alloc'd channels 0/1
oggthread: waiting on semaphore
sndserver: successfully created thread

-> Thread talking: waiting infinite loop

Main thread talking:
fs_copy /cd/dreamcast/snd/01.ogg to /ram/01.ogg
Time: 16543ms
fs_copy /cd/dreamcast/snd/02.ogg to /ram/02.ogg
Time: 22327ms
fs_copy /cd/dreamcast/snd/03.ogg to /ram/03.ogg
Time: 4023ms
fs_copy /cd/dreamcast/snd/04.ogg to /ram/04.ogg
Time: 20837ms
it's really bad because, my game need to run this second thread, and I can't cache all the ogg file in memory
and audio cd player is out of the question because I need precise music synchronization
Last edited by Orion_ on Thu Mar 16, 2017 5:39 am, edited 6 times in total.
Retro Game Programming -> http://onorisoft.free.fr/ <-
Dreamcast, PS1, PCe, MD, Atari, Jaguar, GP32, GBA, ...
User avatar
Orion_
DCEmu Crazy Poster
DCEmu Crazy Poster
Posts: 31
Joined: Fri Nov 21, 2014 3:31 pm
Location: France
Has liked: 0
Been liked: 0
Contact:

Re: pthread and fs_read slowness

Post by Orion_ » Thu Mar 16, 2017 4:58 am

see update of the first post
Retro Game Programming -> http://onorisoft.free.fr/ <-
Dreamcast, PS1, PCe, MD, Atari, Jaguar, GP32, GBA, ...
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5384
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has liked: 1 time
Been liked: 11 times
Contact:

Re: pthread and fs_read slowdown

Post by BlueCrab » Thu Mar 16, 2017 6:45 am

Are you attempting to read from the CD in two different threads at the same time? That will cause some pretty hefty slowdowns as you'll effectively be ping-ponging back and forth across different parts of the disc, causing long seek latency delays. Unfortunately, there's nothing that can be done about that, other than trying to move all your reading from the CD into one thread.
User avatar
Orion_
DCEmu Crazy Poster
DCEmu Crazy Poster
Posts: 31
Joined: Fri Nov 21, 2014 3:31 pm
Location: France
Has liked: 0
Been liked: 0
Contact:

Re: pthread and fs_read slowdown

Post by Orion_ » Thu Mar 16, 2017 6:51 am

no, only the main thread read from CD.
meanwhile the second thread loop in an infinite while(flag);
the flag is set to 0 when the CD loading from the main thread is finished, so the second thread can continue doing stuff, but never access the CD.

btw, I'm using KOS 2.0.1, never had the success of compiling the latest version.
with sh-elf-gcc (GCC) 4.7.3
Retro Game Programming -> http://onorisoft.free.fr/ <-
Dreamcast, PS1, PCe, MD, Atari, Jaguar, GP32, GBA, ...
Chilly Willy
DC Developer
DC Developer
Posts: 414
Joined: Thu Aug 20, 2009 11:00 am
Has liked: 0
Been liked: 2 times

Re: pthread and fs_read slowdown

Post by Chilly Willy » Thu Mar 16, 2017 9:57 am

Are you running KOS in cooperative mode? If so, you need to sleep in the threads (or at least bump the scheduler) periodically to allow time to the other threads. Especially if one thread is just busy-looping as you mention in your last post - that will kill other threads unless you have some kind of wait to allow the OS to run the other threads.

I'm not sure how often KOS calls the scheduler in pre-emptive mode... it may be not often enough to make lots of small reads run fast. I ran into that on a number of OSes in the past. It might still be in your best interest to manually sleep in threads to allow IO enough time to go fast.
User avatar
Orion_
DCEmu Crazy Poster
DCEmu Crazy Poster
Posts: 31
Joined: Fri Nov 21, 2014 3:31 pm
Location: France
Has liked: 0
Been liked: 0
Contact:

Re: pthread and fs_read slowdown

Post by Orion_ » Thu Mar 16, 2017 11:06 am

what is the function to "sleep" the thread in KOS ?
I tried pthread_yield(); and shed_yield();, but both functions are not recognized in KOS sdk :/
Retro Game Programming -> http://onorisoft.free.fr/ <-
Dreamcast, PS1, PCe, MD, Atari, Jaguar, GP32, GBA, ...
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5384
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has liked: 1 time
Been liked: 11 times
Contact:

Re: pthread and fs_read slowdown

Post by BlueCrab » Thu Mar 16, 2017 1:50 pm

Chilly Willy wrote:I'm not sure how often KOS calls the scheduler in pre-emptive mode... it may be not often enough to make lots of small reads run fast. I ran into that on a number of OSes in the past. It might still be in your best interest to manually sleep in threads to allow IO enough time to go fast.
KOS calls the scheduler HZ times per second, which is defined by default as 100.
Orion_ wrote:what is the function to "sleep" the thread in KOS ?
I tried pthread_yield(); and shed_yield();, but both functions are not recognized in KOS sdk :/
thd_pass() is what you're looking for. Or, perhaps you want thd_sleep(), if you actually want the thread to sleep and not just pass away it's timeslice...

For what you're doing, it might be a good idea to look into condvars and other synchronization primitives, so that you don't have that second thread spinning and doing nothing as it is. A loop calling thd_pass() will still take time, whereas if you put the thread to sleep on a sync primitive (or using genwait), then it will actually be asleep and never cause the scheduler to schedule it.
Orion_ wrote:no, only the main thread read from CD.
meanwhile the second thread loop in an infinite while(flag);
the flag is set to 0 when the CD loading from the main thread is finished, so the second thread can continue doing stuff, but never access the CD.

btw, I'm using KOS 2.0.1, never had the success of compiling the latest version.
with sh-elf-gcc (GCC) 4.7.3
There is no 2.0.1 release of KOS, do you mean 2.0.0?
User avatar
Orion_
DCEmu Crazy Poster
DCEmu Crazy Poster
Posts: 31
Joined: Fri Nov 21, 2014 3:31 pm
Location: France
Has liked: 0
Been liked: 0
Contact:

Re: pthread and fs_read slowdown

Post by Orion_ » Thu Mar 16, 2017 3:33 pm

BlueCrab wrote:thd_pass() is what you're looking for. Or, perhaps you want thd_sleep(), if you actually want the thread to sleep and not just pass away it's timeslice...

For what you're doing, it might be a good idea to look into condvars and other synchronization primitives, so that you don't have that second thread spinning and doing nothing as it is. A loop calling thd_pass() will still take time, whereas if you put the thread to sleep on a sync primitive (or using genwait), then it will actually be asleep and never cause the scheduler to schedule it.
I tried

Code: Select all

	while (flag)
		thd_sleep(15);
and no change ... the loading still takes lots of time (around 7 times slower)
BlueCrab wrote:
Orion_ wrote: btw, I'm using KOS 2.0.1, never had the success of compiling the latest version.
There is no 2.0.1 release of KOS, do you mean 2.0.0?
That's what is written in the "RELNOTES" file, but yes I think I use the 2.0.0 version

Anyway,
I'm using a work around for now, I reduced the OGG files bitrate, and loaded them all in RAM FS before starting the second thread.

Can I call "pthread_mutex_lock" in the main "thread" (which was not created with pthread of course) to try to prioritize the loading ?
Retro Game Programming -> http://onorisoft.free.fr/ <-
Dreamcast, PS1, PCe, MD, Atari, Jaguar, GP32, GBA, ...
User avatar
BlueCrab
The Crabby Overlord
The Crabby Overlord
Posts: 5384
Joined: Mon May 27, 2002 11:31 am
Location: Sailing the Skies of Arcadia
Has liked: 1 time
Been liked: 11 times
Contact:

Re: pthread and fs_read slowdown

Post by BlueCrab » Thu Mar 16, 2017 9:40 pm

If this program is specific to the Dreamcast, I'd recommend not using the pthread_* functions, and instead just using the native ones instead. The pthread_* stuff is all just wrappers around KOS' native threads and sync primitives.

But to answer the question, yes, you can use pthread_mutex_lock() in the main thread, since it is just a wrapper around KOS' built-in mutex_lock().
Post Reply