Announcing the Sylverant PSO Server Project

Post by **BlueCrab** » Wed Sep 09, 2009 8:18 pm

Even though the day is almost over, and its not exactly what I was planning for the occasion of 9/9/09, I'd like to announce the Sylverant PSO Server Project.

Basically, Sylverant is an open-source Phantasy Star Online server written for Unix-like operating systems (Linux and Mac OS X are where it has been tested so far). My original plan was to have a fully-functional server running today, however a rather nasty bug exists when running the server under Linux, and the machine I'm running the server on runs Linux (thanks to GrizzlyAdams for use of the machine); so no running server today, sorry.

Anyway, if you care to know more and read about the project, visit http://sylverant.twihb.net/ . The current source code is available from the Google Code project linked to from there.

Nico0020 · Post by **Nico0020** » Wed Sep 09, 2009 8:57 pm

only have one second to be on here, but very interesting. 1st in probably a series of a lot of questions from me. But will this be a completely independent server from schthack, or will your servers be linked and appear on the shipgate of schthack?

Post by **BlueCrab** » Wed Sep 09, 2009 9:51 pm

Completely independent project, with its own shipgate and the like.

Neoblast · Post by **Neoblast** » Thu Sep 10, 2009 5:31 am

Hummm, interesting, I would like to test it with someone, anyone up for it?

Post by **BlueCrab** » Thu Sep 10, 2009 7:27 am

Before anyone tries to run the server, and finds the major bug that prevented me from putting up an actual public server, I guess I should warn people about it...

For some (as of yet unknown) reason, when running the ship server on Linux, there's a bug where you cannot join an existing game. If you do, for some reason, the leader of the game will never send out all the synchronization data, thus you will end up staring at the pillar in the middle of the hunter's guild, unable to move. Also, the leader will end up frozen on the player is joining screen. Note that creating games and joining regular lobbies works just fine.

The bug doesn't happen when running the ship server on Mac OS X, which makes it a bit harder for me to debug (since I don't have direct, physical access to the Linux machine I'm running everything on).

Neoblast · Post by **Neoblast** » Thu Sep 10, 2009 9:05 am

Oddly enough that bug happened in ther fuzziqer's server too, but only happened in the dc/pc version. it might be that the server HAS ( but doesnt, maybe thatpart is different form other versions of the game )to send the "PLAYER ALREADY HAS JOINED" command to everyone,then the player joining will spawn its character, and the oher won't be freeze no more with that "XXXXX player is joining the team please wait".

Can it be compiled for windows as well or only unix like systems?

Getting some game logs from this part, in both players, and both servers ( this and schthack for example ) would give us the idea of what is wrong or what data isnt sent.

Post by **BlueCrab** » Thu Sep 10, 2009 9:49 am

Neoblast wrote:Oddly enough that bug happened in ther fuzziqer's server too, but only happened in the dc/pc version. it might be that the server HAS ( but doesnt, maybe thatpart is different form other versions of the game )to send the "PLAYER ALREADY HAS JOINED" command to everyone,then the player joining will spawn its character, and the oher won't be freeze no more with that "XXXXX player is joining the team please wait".

What seems to be happening on Linux is that the leader never sends the second packet of game synchronization data. Other than that, the same things happens on Mac OS X as on Linux. On Mac OS X, all the synchronization data sends just fine (the code is the same between the two).

Can it be compiled for windows as well or only unix like systems?

The server is multi-threaded, and uses POSIX threads, so as long as there's a BSD-compatible network stack (Winsock should work fine here), and a working POSIX threading library, it should just work. The second part is what would prohibit it from working on Windows. In theory, it should work when compiled against Cygwin, but it hasn't been tested.

Getting some game logs from this part, in both players, and both servers ( this and schthack for example ) would give us the idea of what is wrong or what data isnt sent.

I've done plenty of packet logging, and like I said, my server behaves the same on Linux as it does on Mac OS X, but the game doesn't for whatever reason. The only obvious difference I can see is that the TCP receive window is different on Linux than it is on Mac OS X. I've tried at least two approaches to changing this but neither worked properly.

Ex-Cyber · Post by **Ex-Cyber** » Thu Sep 10, 2009 10:14 am

BlueCrab wrote:The only obvious difference I can see is that the TCP receive window is different on Linux than it is on Mac OS X. I've tried at least two approaches to changing this but neither worked properly.

Could it be a race condition, or is that part strictly sequential?

Neoblast · Post by **Neoblast** » Thu Sep 10, 2009 10:19 am

Hmmm it could be sent manually using the server.
As I said earlier this very same error occured in the pc part of kohle's server ( running on win ).
Do the clients keep sending the idle command every X seconds to the server when this occurs?

it was something like 04 00 00 00 and so on.

Try another TCP module, newer version or so.

Post by **BlueCrab** » Thu Sep 10, 2009 10:24 am

Neoblast wrote:Hmmm it could be sent manually using the server.
As I said earlier this very same error occured in the pc part of kohle's server ( running on win ).
Do the clients keep sending the idle command every X seconds to the server when this occurs?

it was something like 04 00 00 00 and so on.

Not that I've seen, no.

Try another TCP module, newer version or so.

I'm using version 2.6.30 of the Linux Kernel, which was the newest version up until today.

Neoblast · Post by **Neoblast** » Thu Sep 10, 2009 10:31 am

The TCP module in MACOSX is different or does it use the same as linux?

Hmmm is command 65 sent properly?

Code: Select all


DC/GC: 67 NN SS SS CC LL 01 TT BB BB EE EE 00 00 00 00 <player data sets, 0x43C bytes each>

(S = size, N - number of players, C = client's ID, L = leader's ID, T = lobby number, B = block number, E = event)

Always one player in the 65 command.

Make sure the command 6F is sent after the player is done loading:

Code: Select all

6F - set game status (client->server) (all versions)
DC/GC: 6F 00 04 00
PC: 04 00 6F 00
BB: 08 00 6F JJ 00 00 00 00
(J = 0 if game is open, 1 if a quest is in progress and the game should not be joined by anyone else)
Note: this command tells when the client is done loading into a game, so other players may join

Also in the dc join game command something might be different because it has not been documented at least on all the docs I have, it seems to be the same as the pcv2 version but with some header differences...

Code: Select all

64 - join game (server->client) (all versions)

PC: 68 01 64 NN LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL WW WW WW WW WW WW WW WW WW WW WW WW
WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW
WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW
WW WW WW WW XX XX XX XX XX XX XX XX XX XX XX XX
XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
XX XX XX XX YY YY YY YY YY YY YY YY YY YY YY YY
YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY
YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY
YY YY YY YY ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ
ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ
ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ
ZZ ZZ ZZ ZZ II MM 00 DD BB EE SS CC RR RR RR RR
00 01 00 00

DC FORMAT UNKNOWN

(N = number of players in game, L = map layout, W/X/Y/Z = player lobby data for players 1/2/3/4 respectively, I = client ID of joining player, M = leader ID, D = difficulty, B = battle mode enabled, E = holiday ID, S = section ID of original creator of game, C = challenge mode enabled, R = random number unique to this game, T = episode (0-2))

Post by **BlueCrab** » Thu Sep 10, 2009 10:41 am

Neoblast wrote:The TCP module in MACOSX is different or does it use the same as linux?

Most certainly different. Mac OS X is based on FreeBSD, not Linux.

Hmmm is command 65 sent properly?

Code: Select all


DC/GC: 67 NN SS SS CC LL 01 TT BB BB EE EE 00 00 00 00 <player data sets, 0x43C bytes each>

(S = size, N - number of players, C = client's ID, L = leader's ID, T = lobby number, B = block number, E = event)

Always one player in the 65 command.

Make sure the command 6F is sent after the player is done loading:

Code: Select all

6F - set game status (client->server) (all versions)
DC/GC: 6F 00 04 00
PC: 04 00 6F 00
BB: 08 00 6F JJ 00 00 00 00
(J = 0 if game is open, 1 if a quest is in progress and the game should not be joined by anyone else)
Note: this command tells when the client is done loading into a game, so other players may join

Also in the dc join game command something might be different because it has not been documented at least on all the docs I have, it seems to be the same as the pcv2 version but with some header differences...

Code: Select all

64 - join game (server->client) (all versions)

PC: 68 01 64 NN LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL LL
LL LL LL LL WW WW WW WW WW WW WW WW WW WW WW WW
WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW
WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW WW
WW WW WW WW XX XX XX XX XX XX XX XX XX XX XX XX
XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
XX XX XX XX YY YY YY YY YY YY YY YY YY YY YY YY
YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY
YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY YY
YY YY YY YY ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ
ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ
ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ ZZ
ZZ ZZ ZZ ZZ II MM 00 DD BB EE SS CC RR RR RR RR
00 01 00 00

DC FORMAT UNKNOWN

(N = number of players in game, L = map layout, W/X/Y/Z = player lobby data for players 1/2/3/4 respectively, I = client ID of joining player, M = leader ID, D = difficulty, B = battle mode enabled, E = holiday ID, S = section ID of original creator of game, C = challenge mode enabled, R = random number unique to this game, T = episode (0-2))

I don't know off the top of my head what happens where, but it should work right. Anything I've seen seems to work right.

Neoblast · Post by **Neoblast** » Thu Sep 10, 2009 10:44 am

Hmmm try to make the server a little more verbosy, and compare the packets, there are enough differences with versions for one or two things to not work properly sometimes, and take a special look in these packets, maybe linux won't send them correctly or with a bad format thus why it hangs on that part...
I will test it tonight see if we can isolate this problem.

Post by **Quzar** » Thu Sep 10, 2009 10:58 am

Ex-Cyber wrote:
BlueCrab wrote:The only obvious difference I can see is that the TCP receive window is different on Linux than it is on Mac OS X. I've tried at least two approaches to changing this but neither worked properly.
Could it be a race condition, or is that part strictly sequential?

That's the first thing I thought when I saw 'multi-threaded'. It's almost assured that the threads would go in a different order between OSX and any Linux distro.

Post by **BlueCrab** » Thu Sep 10, 2009 1:39 pm

Quzar wrote:
Ex-Cyber wrote:
BlueCrab wrote:The only obvious difference I can see is that the TCP receive window is different on Linux than it is on Mac OS X. I've tried at least two approaches to changing this but neither worked properly.
Could it be a race condition, or is that part strictly sequential?
That's the first thing I thought when I saw 'multi-threaded'. It's almost assured that the threads would go in a different order between OSX and any Linux distro.

Games are guaranteed to be on the same thread (since currently, there's only one thread per block). I doubt its related to the threading...

Guaripolo · Post by **Guaripolo** » Thu Sep 10, 2009 6:10 pm

wow! Congratulations! this is a good surprise for the DC users! thanks!

Maybe a net auditing tool can be useful? (like wireshark). If the mac version runs fine and the linux don't, a comparision between the two sequences can isolate the problem.

Neoblast · Post by **Neoblast** » Fri Sep 11, 2009 4:49 am

Yeah, probably some packet isnt sent properly to the client, then it still waits for something and stands still.

Impulse921 · Post by **Impulse921** » Thu Sep 17, 2009 3:30 am

BlueCrab, congrats on the server! This is very impressive.

This bug happens occasionally on the schthack server and is very common on the PPSOS v2. I wonder if it is a similar problem. I always thought it had to do with connection, but I guess it's in the code.

I grabbed the code from the Google site. How should I go about compiling it. I'd like to help in testing. I could run it on linux or Mac os x. What compiler would you recommend to get it running?

Post by **BlueCrab** » Thu Sep 17, 2009 6:55 am

Impulse921 wrote:BlueCrab, congrats on the server! This is very impressive.

This bug happens occasionally on the schthack server and is very common on the PPSOS v2. I wonder if it is a similar problem. I always thought it had to do with connection, but I guess it's in the code.

I grabbed the code from the Google site. How should I go about compiling it. I'd like to help in testing. I could run it on linux or Mac os x. What compiler would you recommend to get it running?

Well... If you're on Mac OS X, most of what you need is included with Xcode (at least on Leopard and Snow Leopard). You'll need to install MySQL (at least the development libraries) to build any part of it, and if you intend to run a shipgate/login server, you'll actually need to have a MySQL database running. Most of that can easily be transferred over to Linux. Note, a ship server is pointless without a shipgate to log into.

The build process... well... is a bit hairy at the moment. Only libsylverant (a dependency of all the other parts) has an autotools script setup at the moment. This is something I'd rather rectify before all kinds of people try to compile it and start reporting odd/random errors. But, if you really want to compile it ship server requires the following libraries: libsylverant, libcrypto (from OpenSSL, should be included just about everywhere), libiconv (only on Mac OS X), and libexpat. The other two parts require the MySQL libraries as well as what was listed there for shipgate. libsylverant itself requires libexpat and the MySQL libraries.

Hopefully this weekend I can work on the build process a bit and put up the stuff to set up a MySQL database for use with shipgate and login server.

Announcing the Sylverant PSO Server Project

Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project

Re: Announcing the Sylverant PSO Server Project