The general spring modularisation discussion

[Forboding Angel]

Kloot, you're blowing what Hoijui said out of proportion, and worse, you're putting words in his mouth and then taking them personally.

Chill dude. No one ever called you a low level peon and it was never implied that you were a low level peon. Take a breath and think about it.

[SinbadEV]

Kloot, you're blowing what Hoijui said out of proportion, and worse, you're putting words in his mouth and then taking them personally.

Chill dude. No one ever called you a low level peon and it was never implied that you were a low level peon. Take a breath and think about it.

When he said "Low Level" I think he meant it in the context of "low level language" as in Assembly VS C++, such that the "low level" guys work on the fine details, closer to the engines core.

[danil_kalina]

Kloot, you're blowing what Hoijui said out of proportion, and worse, you're putting words in his mouth and then taking them personally.

Chill dude. No one ever called you a low level peon and it was never implied that you were a low level peon. Take a breath and think about it.

True
Hoijui is very straight and a bit rough.

[Forboding Angel]

When he said "Low Level" I think he meant it in the context of "low level language" as in Assembly VS C++, such that the "low level" guys work on the fine details, closer to the engines core.

That's what I took Hoijui's post to mean, but Kloot is interpreting it as though he's being called some bottomfeeder, good-for-nothing, peon, which is ridiculous and I don't see how any rational person could even think that.

[hoijui]

Reducing code duplication and using interfaces IS modularization.
Cause you will have 1 place to change the implementation instead of doing so in 20 files. With your way of doing it you split it across such many files and reduce locality.

what do you mean with using interfaces?
in my solution there is nothing like one thing split over many files.
if some weapons use the same way to draw on the minimap, they share the same minimap renderer file.
though this does not have to be done in the first step, and it actually has minor advantages to do it in several steps, but that is unimportant.
having 3D and minimap rendering split, means 2 instead of 1 interface -> more modules -> more modularization, and i already gave examples how this separation can make sense.

@Kloot
we already discussed at least one way to remove virtual function calls, be it necessary, and we did that in the first meeting, and i refereed to that in my post in this thread.

you mentioned low level stuff i did see myself. we found solutions for them, and you keep saying "no" without new arguments.
i always accepted that you see low level stuff i do not see, and i accepted every time you mentioned it. you never even considered that i might really know more about high level stuff, probably cause you saw me fail for so long at low level stuff.
CEO that says, prove me wrong and i will act accordingly, and has actually done that many times... where do you live again, kloot?
you in turn do whatever you want, and are always right, and if everyone tells you you are not, then you may accept the other way, but only cause you are so noble, while still being right with your way. on a bad day, you threaten with leaving.

btw, i also kind of.. nearly left, after the meeting where "my way" was turned down. though .. strangely, i have never done that before, when i was actually shown to be wrong.

.. i summon the grand jury!

[Tim Blokdijk]

Just to split out the discussion.

.. i summon the grand jury!

Sure, let me resolve this for you guys.
First off, your think speed optimisation and readable code are nice things. And Kloot/jK think that abstraction is a good idea in itself. Most of the time it's perfectly possible to have the cake and eat it to. The problem lies with the grey area where speed/readable/abstraction is excluding another to some degree. My suggestion: If there is no obvious way to get it all, then let the developer that writes the code decide?

[hoijui]

the speed problem of my solution, if it ever was one, was solved already.
and even though it usually works as you say (the one that writes the code decides about the details), i think this should not be the case for really big/important stuff.
i did write the code, and then showed it to the other devs, and they said no.
i often did that, and they often said no, but this time, there was no acceptable reason. not even vaguely acceptable.
there is still none.

.. could a moderator maybe move the posts relevant to this discussion from the other thread in here, please?

Done, moderation.

[knorke]

.. could a moderator maybe move the posts relevant to this discussion from the other thread in here, please?

sorry didnt read that thread until now.
i think i could copy over the posts from that other thread, but afaik they will be merged in order of posting time. so stuff would be in quite random order?

[SirMaverick]

The posts are already moved.

[hoijui]

thanks for moving the posts!

Modularization frameworks
modularization is far, far more then duplicate code reduction.
The most famous and most widely used software modularization framework is probably OSGi. the following page shows some of the features of it, of which most are applicable to modularization frameworks in general:
http://www.osgi.org/Technology/WhyOSGi

Native/Universal Modularization frameworks
I found an interesting article about (OSGi like) modularization frameworks for C and C++:
http://blog.cppmicroservices.org/2012/03/29/osgi-and-c++/

The two most interesting frameworks:

CTK Plugin Framework
http://www.commontk.org/index.php/Docum ... _Framework

• Uses the QT plugin system
• supports Bundle tracking

Celix (Apache)
http://incubator.apache.org/celix/

• C based
• OSGi interoperability

one big big disadvantage of these frameworks vs. OSGi, is that hardly anyone uses them. which i think, is to be explained by the nature of the C/C++ community.

Spring & Modularization frameworks
in a spring context, using some framework like this could improve the general structure of the engine. for example, it would technically drag towards using cleaner APIs between engine internal components, and could automate the initialization order of the components (which is quite a mess).

[Viod]

thanks for moving the posts!

Modularization frameworks
modularization is far, far more then duplicate code reduction.
The most famous and most widely used software modularization framework is probably OSGi. the following page shows some of the features of it, of which most are applicable to modularization frameworks in general:
http://www.osgi.org/Technology/WhyOSGi

Why do you need a framework to modularize the whole project? I mean, the project is really big, but I think you can do the same thing with a proper design of namespaces, classes and interfaces. With a modularization framework the risk is to create even more complexity. The only good thing is that you can work on your module and other devs can't tell you nothing, because that's YOUR module. So indipendence is really a good thing when there is no democracy within devs, but I really hope this is not the case.

I've one across plenty of skilled people in every aspect of development who have found spring and felt a desire to contribute to the engine or start a game but have been baffled because they have no idea how or where to begin.

If they were THAT skilled, they would figure it out, they would ask for help etc. Which we see. People who start a project/game/whatever in spring do not have to be that technically adept or even close to the level of person to work on the engine. Most of what hoi is talking about is above my head let alone some random guy who cannot figure out how to work GIT.

This is not true, trust me. I'm a git user and the problem is not with git, but with the code. The only way to understand how it works is to build and running it with a debugger. To attract new developers you need a good documentation, but the problem with it is that you guys don't care about documentation at all. You are devs, devs code, period. From outside, the way to do things here is like: "Fuck UML diagrams, fuck documentation, I need to remove this abstract class from here so the code can be 0,0001% faster. I don't care if it will be a monolithic mess, it's 0,00001% faster!! New devs? Fuck them. If they are THAT good, they will figure out how to do things here, I don't know what kind of things...just things."

or there would be more virtual function calls, which supposedly make the engine too slow (though that is very, very unlikely in my eyes...)

to stay strong and defend your beliefs at all costs is the CEO/big company way of doing things. it is probably their biggest weakness, and a source of great evil.

ORLY?

Here's a little anecdote: you mentioned QTPFS, which initially used a nice OO design with virtual tree nodes. Then I profiled it and found it was 20% slower per search execution than without the virtual structure. *That* is why I decided not to further modularize the pathfinders, because their code is already time-critical and they call into the engine many thousands of times per frame. So exactly who is defending their beliefs at all costs here and who has the empirical evidence to back up their points? Could it be us dumb low-level peons were right?

If we (the LL guys) supposedly cannot see anything of the big picture, then you (the HL guy) conversely cannot see the small details, and dismissing them as "laughable" and claiming "you can't conceivably explain why i am wrong" makes you the CEO and us the exploited employees whose opinion is never listened to or welcomed.

The problem with QTPFS is QTPFS. You can remove every virtual structure from it, you can even rewrite it in assembly, but it will never works and I tell you why:
1- I found this video ( http://www.youtube.com/watch?v=BQnh10suib0 ) and I can tell you this is ridiculously heavy to run. A pathfinder has to be simple, elegant and fast.
2- How it really works? Where is the algorithm? What is the computational cost of it? Can we improve or contribute to it? Again, no documentation at all. So why are you improving it if the computational cost of the algorithm is too high? Without a proper high-level view of the problem you risk to waste your time. It's like you are trying to solve a NP-complete problem in polynomial time.

[Kloot]

Thanks for your opinion. When I care about what you think you can tell me from watching a video or what you think I should do with my own free time, I'll let you know.

The fact that there is a general lack of documentation does not entitle you to anything.

[Viod]

Thanks for your opinion. When I care about what you think you can tell me from watching a video or what you think I should do with my own free time, I'll let you know.

The fact that there is a general lack of documentation does not entitle you to anything.

Are you developing QTPFS?
I did not want to offend you and I'm sorry for that. It's a bit hard for me to express myself correctly because English is not my native language. However, my point is: how do you know that QTPFS is usable in this context? Do you have the computational cost of the algorithm? I speak about documentation because it's really important. With a proper technical and theoretical documentation we can help you guys. I'm not here to insult your work, I think it's great, but sometimes I think you are going in the wrong direction. What's the point to improve QTPFS if it's non polynomial?

And QTPFS is just an example of how things work here. It's like there is no planning, no discussions about the future and no order in this project. How do you see Spring in 5 years? How many devs it will have? Do you think the current devs will be here in 5 years? The project is meant to be open source, but the open source idea is not only "free access to the code", but also a matter of openness, accessibility, clear designs and algorithms behind the code.

You are right, I can hardly evaluate the algorithm from a Youtube video, so can you tell me how it works? Can we discuss together about it? I may help you, or maybe not, but at least I (and others too) can try.

[Kloot]

You must hold us in pretty low esteem if you think we would ever knowingly (or worse, unknowingly) insert non-polynomial algorithms into Spring. On the other hand I'm also sure you are aware that even an exponential-time algorithm can be usable for small bases, and even polynomial-time algorithms can not be for large degrees. Theoretical analysis is not the whole story.

However, my point is: how do you know that QTPFS is usable in this context?

Testing and feedback.

If it turns out to be too slow at the end of the road, the only real loss is my time (with no guarantee that it would have been spent "better" in the alternate universe where QTPFS does not exist).

so can you tell me how it [qtpfs] works?

It's plain old A* on the leafs of a quadtree, no exptime code there. More specific questions can bring more specific answers.

It's like there is no planning, no discussions about the future and no order in this project. How do you see Spring in 5 years? How many devs it will have? Do you think the current devs will be here in 5 years? The project is meant to be open source, but the open source idea is not only "free access to the code", but also a matter of openness, accessibility, clear designs and algorithms behind the code.

In an ideal world, everyone would always get along, plans would always work out to the smallest detail, discussions about the future would always be perfect accurate predictions, documentation would always be in ample supply, complete, up to date and written by every developer, etc. We do not live in one here. Given that, planning five years ahead for a small niche project like Spring is (IMO) not a useful time investment. Developing Spring is a fun hobby, nothing more, nothing less. Direction and order would come as byproducts of a more serious approach, which for me (and I assume for the other devs) is where it would stop being fun and start becoming more like an unpaid obligation, and we have enough of those already.

[AF]

May I point out that I am highly proficient at eating cheesecake. I would very much like to eat cheesecake.

Presented with the ingredients to my favourite cheesecake, I could make one, but I have never tried, and it would take time, and research. It is unlikely that I would buy said ingredients and bake the cheesecake. I am perfectly capable of doing so and I have the skills to figure it out, but I very much doubt it would happen.

I'm sure a developer who arrives wanting to contribute could very well do so, but that doesn't mean that they will. Sometimes the cost and learning curve is too high, and just because you scaled it doesn't mean that all approaches should be interpreted as easy because you're at the easy shallow end of being in the know, or that the steepness of said curve is irrelevant because you've already scaled it.


Clearly we should make our site and games harder to use because we already know how to use it, who cares about new players, if they're that good at RTS games they can figure it out on their own.

[hoijui]

Why do you need a framework to modularize the whole project? I mean, the project is really big, but I think you can do the same thing with a proper design of namespaces, classes and interfaces. With a modularization framework the risk is to create even more complexity. The only good thing is that you can work on your module and other devs can't tell you nothing, because that's YOUR module. So indipendence is really a good thing when there is no democracy within devs, but I really hope this is not the case.

It's not that we need one, but it might help; as i wrote already, by dragging a bit more towards better design. every little piece of that is worth A LOT in spring. of course you don't need one in theory, and i agree that just using classes, interfaces and namespaces better would be the thing that we need, and of course we don't need a modularization framework for that. but in theory, you can also write nice OO projects in C++, and.. how often do we see that?
bear in mind, that spring devs don't just know more about the spring engine code then you do, but also about how this code is worked on, and about the spring engine dev community, and what is possible and what is not, due to the kind of people they are, and the reasons and motivations they work on spring and so on (see also what kloot already wrote). if you come with a replacement team of new devs, all with the "right" attitude... they would puke over the code and instead rewrite the engine, and make all games incompatible in turn (not that you could come up with such a team).
an other reason why i am talking about modularization frameworks, is to give a counter argument versus modularization == removing of code duplication.
modularization is mostly a high level design thing. code duplication is removed by using functions, which is low level. an OO software project should not be a set of functions spread over a reasonable number of source files. there should be classes, being subclasses of others, being parts of class clusters forming submodules that form bigger modules together with other submodules and.. possibly more levels.
to standardize such structures, one can use a modularization framework.
maybe, the ones in C++ are very bad.. i could even easily imagine them to be, as they will likely have to use dirty tricks to work around many special cases and what not. maybe.