Changes in Networking (with patch)

Neddie · Post by **Neddie** » 26 Jun 2007, 23:33

Fair enough.

For networking, I'm the guy stringing the cables and checking for one to one relationships in the side room. I'm not yet the person actually monitoring the router traffic.

lazorwolf · Post by **lazorwolf** » 27 Jun 2007, 01:27

neddiedrow wrote:Could we devise a new method of TCP holepunching?

No. Reread Licho's posts. Then read Wikipedia's articles on TCP, UDP, and IP.

Holepunching only works with UDP because routers leave ports used by UDP connections open since there's no FIN or ACK packets to track connection lifetimes like TCP has.

Auswaschbar: Holepunching is a pretty popular feature of Spring, and implementing lots of TCP features with UDP is the price we have to pay.

Perhaps if Spring support UPnP and/or auto-hosts were more popular we could switch to TCP, but I'm afraid right now we'd just lose a lot of users.

Thanks for your hard work! 28% overhead seems high, but it may be necessary.

Wish I had more time to relearn C++ and tinker with Spring's code!

Tobi · Post by **Tobi** » 27 Jun 2007, 10:40

If this code works good enough I may drop my new netcode, I'm not really motivated to fix it up anyway. I'll try to look at it as soon as possible.

Post by **Tim Blokdijk** » 27 Jun 2007, 13:11

Well that leaves the job more then open for Auswaschbar.. I guess.
Would you like to work some more on the netcode Auswaschbar?

(I'm presuming here that your code is good enough.)
Maybe Tobi can give you some pointers about his work on the netcode branch if you need it?

lazorwolf · Post by **lazorwolf** » 27 Jun 2007, 15:58

On an indirectly related note, what if I created a new dev site for Spring with an active mailing list and Trac integration with SVN.

I know we had a mailing list, but I'm not even sure where it is anymore. I also know there's a bug tracker, but how much is it even used?

A new Trac site would also let us separate the Dev. Wiki from the main Wiki.

Let me know.

Thanks to the devs for all of your hard work! I'd love to help in non-coding ways.

AF · Post by AF » 27 Jun 2007, 16:14

You are aware that in the general discussion forum there'e 6 or 7 threads discussing a new website, including numerous site proposals?

You'll find the new site in the svn at /trunk/site/ email tim blodjik for details.

Post by **Auswaschbar** » 27 Jun 2007, 19:03

I will continue working on the netcode because there are still lots of things to do...

(I'm presuming here that your code is good enough.)

If it isn't considered good enough, I can still work on the netcode to make it better

Post by **Auswaschbar** » 27 Jun 2007, 19:07

Tim Blokdijk wrote:Maybe Tobi can give you some pointers about his work on the netcode branch if you need it?

I just checked it out and its very well documented, so there shouldn't be any problems

Post by **Tim Blokdijk** » 27 Jun 2007, 20:24

Great!

and yes there is still lots of things to do.. I'm glad you are willing to take a stab at it, hope you can get this on to a higher level.

Lippy · Post by **Lippy** » 27 Jun 2007, 20:37

Go Auswaschbar!

FLOZi · Post by **FLOZi** » 27 Jun 2007, 20:46

What improvements are there to be made with netcode?

Post by **Auswaschbar** » 27 Jun 2007, 20:54

Demo stuff should be removed from netcode
further cleanups needed
better packet handling with decreased overhead
maybe fix bugs
add fancy features i can't imagine (any ideas?)

Tobi · Post by **Tobi** » 27 Jun 2007, 21:03

Some ideas I had to implement on top of my netcode:
- On timeout, attempt to reconnect (& send over missing part of stream) instead of just dropping.
- On game start, have each client attemp to connect to each other client and collect stats on success/failure rate of this. Pick the client to which most other clients could connect succesfully and mark this as 'backup host'. Now if the host quits, all clients can connect to the 'backup host' and continue playing.
- Factor out CGameServer from the engine itself, in a way so it can be compiled in and run as a thread, but can also be run standalone (for autohosts etc.).

lazorwolf · Post by **lazorwolf** » 27 Jun 2007, 21:39

Tobi wrote:- Factor out CGameServer from the engine itself, in a way so it can be compiled in and run as a thread, but can also be run standalone (for autohosts etc.).

I think this is by far the best enhancement idea because it eliminates the need for "backup hosts."

You could also do clever things like write a lobby as a web application which could spawn Spring CGameServer processes for people. Writing a web lobby is an idea I've been toying with and wishing I had the time to implement!

Licho · Post by **Licho** » 28 Jun 2007, 11:43

Indeed! This would allow us to vastly improve hosting. Hosting could be later integrated into lobby program (you exit spring and hosting continues), autohost will be much less resource hungry, and autohost/lobby full integration with game will be possible. (Reading and writing into chat, exiting game, forcing start etc - those tasks will be much easier with sever implemented directly in autohost or lobby client)

LOrDo · Post by **LOrDo** » 29 Jun 2007, 08:26

That attempt to recconnect feature sounds pretty awesome to.

Post by **Auswaschbar** » 30 Jun 2007, 20:30

update:
new CDemoRecorder and CDemoReader
all the demo stuff is done in CNetProtocol now, not on CNet anymore
updated to svn 3870 (which breaks almost the whole new CNet, but patches now are integrated (in CDemoReader) )

Tobi · Post by **Tobi** » 30 Jun 2007, 23:14

The design changes in this patch are excellent, good job. At least somewhat less monolithic network class

But it causes some serious bugs currently:

Start positions aren't sent over the net anymore so players start in left-top corner with choose start positions.
In local demo playback mode you can not change speed etc. anymore. (maybe this is because net->isDemoServer() always returns false because you use a trick with actually hosting the demo for local demo playback; so serverNet->isDemoServer(), would be true, not net->isDemoServer())

I got a bunch of

Code: Select all

Overflow when sending to remote connection 39997 5 40000
Overflow when sending to remote connection 39997 10 40000

when testing local demo playback.

GlobalAIs are initialized and loaded in demo playback mode, which is wrong since the commands they gave are stored in the demo already. (Previously this was prevented by giving the watching user the maximum possible player number)

Also I noticed some minor esteatical thing, which is that the methods you added are in lowerCamelCase, while they preferably should be UpperCamelCase (aka PascalCase).

Any chance you can fix these things (and preferably also test all demo combinations

):

local demo playback of non-gameSetup game
hosted demo playback of non-gameSetup game
local demo playback of gameSetup game
host demo playback of gameSetup game

If the hosted demo playback is problematic because you need more people for it you can skip it and we can then test it later in the test server.

Dragon45 · Post by **Dragon45** » 02 Jul 2007, 15:41

Firefox-extension host anyone?

Post by **Auswaschbar** » 02 Jul 2007, 19:01

update:
- start pos had been sent, but there was a type conversion error in CGameServer from uchar -> float, so they were wrong
- you can now change speed etc. in demo playback, isDemoServer() got fixed
- "Overflow when sending to remote connection 39997 5 40000" was a copy'n'past error, the error occured in CLocalConnection and it's because in local playback mode, serverNet don't need any data but client keeps sending... (nothing critical, I will fix it later)
- gu->spectating and gu->spectatingFullView are set to true because client was not able to see enemy units (don't know if it fixes the GloblAI)

edit:
- removed handleerror() and made a class network_error instead (copy'n'paste from content_error)

If i have time, I will join you on wednesday

Spring RTS Engine

Changes in Networking (with patch)