P.O.P.S.- what it is, what it's good for, where to get it.

P.O.P.S.- what it is, what it's good for, where to get it.

Discuss game development here, from a distinct game project to an accessible third-party mutator, down to the interaction and design of individual units if you like.

Moderator: Moderators

Post Reply
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

So... I wrote a particle system.

Some pictures of tests with a GeForce 9800GTS:
Image
About 2400 events spawned via a building's death script + several hundred for the flaming bits. Framerate cost is about 10%, but most of that cost is actually the Piece projectiles hitting the CPU. It peaks at 15% when the smoke expands fast enough to create some depthtest hit, which makes it about 500% faster than the old CEGs were.

Image
About 1200 events spawned as cyclic rocket trails, 5 per projectile per gameframe. Note the spacing: it is accurate between updates and doesn't require any artificial acceleration of the particles, etc., and can be adjusted for more / fewer particles with one math step. No significant framerate cost at all, it's neither bound by CPU nor GPU for these sorts of shorter events with low amounts of depthtest.


What It Is

P.O.P.S. stands for "Powerful Open-source Particle System". It is a 100% free-as-in-beer, GPL'd particle system implementation, written with Lua and GLSL and tested for proper operation on nVidia and ATi hardware. It may even run on higher-end Intel chipsets that meet the base GLSL specification, although I doubt it, simply because Intel's drivers are so terrible. It should fail gracefully on hardware that does not meet specifications.

P.O.P.S. is a project I have been working on in between other things for many months now, specifically to give us a significantly faster general-purpose particle system. It is significantly faster than CEG event calls to CSimpleParticleSystem (the most commonly-used CEG event type) and, because it is coded with Lua, can be used for a very wide variety of tasks.

Why am I releasing it now?

Long story short, because enough basic functionality is done and it works well enough that I think people should take a look at it and start trying to learn how to use it,, and I think it's better to release it now while each example's fairly short than when each one is even more thousands of lines of code.

I don't think that the core code for POPS2_Smoke or POPS2_Fire are going to change between now and the release of P.U.R.E., and other than new refinements, new FX code, etc., the other sections are basically feature-complete and bug-free atm.

What It's Good For

P.O.P.S. is extremely powerful, in the traditional sense. It can be used to replace any standard CEG event with behaviors that are very similar to CEG's, and it can also be used for many things that are either difficult or impossible to do with CEG.

Examples of typical uses include:

Smoke, fire, dust, clouds, sparks, dirt, flying debris, explosions, weapon trails, weather-like effects... pretty much anything where we want to throw significant numbers of particles around in a scene, or create large numbers of particles per gameframe.

I have been developing different uses for P.O.P.S., such as complex building explosions, automated scalable Unit explosions, smoke effects upon damage, particle trails behind projectiles (this includes missiles with really nice smoke trails, no spaces between gameframes, and a very sexy flamethrower). These examples can be seen in the source-code project I am distributing this evening. I am still barely scratching the surface of what P.O.P.S. can do, though, and I am very excited about its potential to significantly speed up battle scenes.

Now, all that said, here are a few things people should understand, before you download it and start playing with it:

1. Are there things it can't do? Yes. It can't rotate particles, because I don't know how to do that with GLSL yet. If you have an effect where you're counting on that, use CEG, I probably won't have time to change that feature before I release P.U.R.E..

2. Will it play nicely with CEG? Absolutely.

3. Will it play nicely with custom UIs and other Lua OpenGL stuff? It should do so in practically all cases, most certainly including LUPS effects (in fact, it makes a great partner with LUPS for a lot of things).

4. Do I plan to write a detailed manual, or to make it very accessible to programming newbies, or build a UI for developing effects or give away magic ponies? Not at this time.

5. Will I answer intelligent questions about use-case stuff? Yes.

6. I tested the old version. What's new since that last tech demo? The short answer is, "a lot", but basically what happened is that I made a real breakthrough in terms of per-event speeds, and it's not only very flexible, but now it's very, very fast per event. There is one last optimization I could make, but atm it's more important (for me) to get it into production than to shave another small bit of performance.


Where To Get It

There is a link attached to this post.

Download it, stick the bitmaps into bitmaps/pops, the code into LuaRules/Gadgets, and you are done with installation.

Past that, you will need to figure out what code you want to modify and build test scenarios.

For a quick, easy test, take a tank or another moving vehicle and write this code into the script (I apologize that this is BOS, not Lua, but I am not yet using Lua):

Code: Select all

static-var isMoving;
lua_CallPOPS(arg) { return (0); }
EmitDust()
{
	var random;
	random = RAND(45,90);
	While(TRUE)
	{
		if(isMoving == 1)
		{
			call-script lua_CallPOPS(dust1,2,75);  //NOTE: "dust1" is a valid Piece name, use a valid Piece name here.
			call-script lua_CallPOPS(dust2,2,75);
			sleep random;
		} else
		{
			sleep random * 5;
		}
	}
}

Create()
{
start-script EmitDust();
}
StartMoving()
{ 
	isMoving = TRUE;	
}

StopMoving()
{ 
	isMoving = FALSE;
}
That code will give you a "motion trail" of P.O.P.S. events behind the tank as it moves. See POPSCall for details of how it works.



The following is a list of the files that are in the package attached to this post, and what they do (the short version):

POPS2_Fire, POPS2_Smoke: Core P.O.P.S. code. Very well-commented, not fun code to read. If you don't understand the basic ideas behind FBO'd textures being used like a Turing tape in a GPU-driven physics simulation, you probably want to stay the hell away from it.

POPSExplosion: Simple demonstration of CEG replacement. Nothing very complex here, I am so busy working on other things that I haven't had time to give you a new nuke sim yet, etc. ;)

POPSCall: Simple demonstration of how to call P.O.P.S. via COB or Lua unit animation scripts. Currently being used for a very basic generic explosion system for generating explosions from Piece locations, and the trail / dust-cloud effects for moving units.

Yes, you can certainly generate P.O.P.S. events that follow a Piece's dir- substitute the dir for the velX,Y,Z, adjust a particle system pre-built for an appropriate cone angle, voila, it will emit in the direction the Piece faces. I just don't have an example in here atm, but the calls to get the dir are already in that code and ready to use for stuff like "nano effects", "jet exhaust", etc., etc., etc.

POPSProjectileTrailEffects: This is the least newbie-friendly section in most ways. It contains complex behaviors for generating various types of particle trails and LUPS projectile trails. The easiest way to play with this is to set a weapon's customParams up with trail_type=0 and trail='yes' to see a projectile trail that doesn't require LUPS. Past that, you need to read the source and modify it to suit your needs- the math's very straightforward but it isn't trivial.

POPSBuildingDeath: This is that skyscraper's death animation atm, nothing more, nothing less. You can't really see this without the latest World Builder, and it's probably not a very good place to start poking at P.O.P.S. but I included it for completeness's sake.


That's all, good luck and have fun :-)
Attachments
POPS_Files.zip
Core files you will need to start working with P.O.P.S.
(199.35 KiB) Downloaded 57 times
screen00093.jpg
screen00093.jpg (59.31 KiB) Viewed 4848 times
screen00090.jpg
(62.42 KiB) Downloaded 2 times
User avatar
aegis
Posts: 2456
Joined: 11 Jul 2007, 17:47

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by aegis »

hey, zoomed out you can use it for puffy trees
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

Nah, they wouldn't adjust for camera angle / parallax, I couldn't really recommend that.

It could be used for simplistic trees if I built more quads and removed the billboarding, but that's massive overkill; really, for something like that, a LUPS-like design would be better.
User avatar
Das Bruce
Posts: 3544
Joined: 23 Nov 2005, 06:16

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Das Bruce »

What are its limits?
User avatar
AlcariTheMad
Posts: 28
Joined: 05 Feb 2007, 02:54

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by AlcariTheMad »

I don't mean to insult, but have you ever read a code style guide?

Also, Lua has object support for a good reason.
User avatar
Das Bruce
Posts: 3544
Joined: 23 Nov 2005, 06:16

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Das Bruce »

AlcariTheMad wrote:I don't mean to insult, but have you ever read a code style guide?
What in particular?

Also does that huge smoke plume appear to be casting square shadows or is it something else?
User avatar
AlcariTheMad
Posts: 28
Joined: 05 Feb 2007, 02:54

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by AlcariTheMad »

My entire post was directed at Argh's coding.
User avatar
Das Bruce
Posts: 3544
Joined: 23 Nov 2005, 06:16

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Das Bruce »

AlcariTheMad wrote:My entire post was directed at Argh's coding.
I know, but I thought their might be one or two obvious and easily explained things to point out, I'm an amateur and am probably making the same mistakes.

I once was given an award for 'wrost code evar'.
User avatar
1v0ry_k1ng
Posts: 4656
Joined: 10 Mar 2006, 10:24

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by 1v0ry_k1ng »

someone going to test this?

also, square shadows ?
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

What are its limits?
The version I've released can have 16384 particles active per system (pops_event, pops_fire_event) for a total of 32768 particles, total. This is an arbitrary limit, set mainly for performance reasons, as there is a small CPU cost for the physics shader's operation. That's almost 10X more than the default number of particles for Spring, and far higher than the number of CEG particles you'd normally want to operate.
shadows
None of the particles cast or receive shadows- neither do CEG events.
User avatar
Masse
Damned Developer
Posts: 979
Joined: 15 Sep 2004, 18:56

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Masse »

I'm interested :) When I got time I'm going to take a look.
Master-Athmos
Posts: 916
Joined: 27 Jun 2009, 01:32

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Master-Athmos »

Well thanks for sharing but my guess is nobody will really use this unless there's any documentation. With that I don't mean fancy UIs or huge effect libraries but just simple things about how this actually works and what settings/options one has...

From your initial post all one knows is how calling an effect has to look like where the first argument is a piece of the model while the 2nd one is a number whatever it may be good for (EDIT: I was wrong - there are 3 arguments). The rest is about actually reading the Lua code (which probably is the main reason next to nobody will use this in its current form).

After giving it a short glimpse it also appears to me that this isn't what I expected in the first place: It's what I would call a "fixed pipeline" for premade effects. There simply is no dropping some values in a definition file that get handled over to the particle system but rather each Lua file is about running a defined effect and all you really could do about is exchange some bitmaps or even set some values which just let that single effect run slightly differently...

That doesn't make POPS into a worthless piece of crap but I wouldn't expect high rates of people using this or big cheers. There's a lot of work in this but
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

There simply is no dropping some values in a definition file that get handled over to the particle system
Actually that's completely wrong, although I can see how you might see it that way at a first glance- the word "pre-built" may be a bit confusing without understanding the context.

P.O.P.S. events are passed to the P.O.P.S. core code via the following synced / unsynced communications:

Code: Select all

SendToUnsynced("pops_event", x, y, z,gravity,velX,velY,velZ,size,drag,sizeGrowth,myVelocity,startTime,finishTime,myRand,myType,myCone,color1R, color1G, color1B, color1A, color2R, color2G, color2B, color2A, color3R, color3G, color3B, color3A, color4R, color4G, color4B, color4A)
For each value in this communication, you can set many valid values to produce totally dynamic effects on the fly. IOW, there are no limits, other than the limits on what variables the physics shader and final shader can handle, and those are, of course, modifiable.

IOW, the "pre-built" code is just pre-built single events optimized for maximum speed, since we're essentially just talking about table-->table transfers to P.O.P.S. core.

But it isn't necessary at all- you can hand-code or procedurally generate all of those values during a program's operation, just like any other Lua code.

When a pre-built is called in my examples, it randomly picks from 1000 (or more) pre-built events, and you could use 100000 if you wanted to spend RAM on that, just to ensure that every individual event was very unique, so far as the end-user is concerned.

For most things, I've found that 1000 events picked randomly is more than enough randomness, but that will depend on the scale and desired behavior, and there's no absolute limit, nor is it necessary to pre-build at all (although this saves so much computation that it is advisable to pre-build as much as possible per event).

And lastly, if you look at the sections where the pre-builts are being constructed... they could just as easily be in a config file or files. The only reason I haven't constructed such a thing is that I don't think it's all that useful- as each example shows, there are a lot of uses for the system, and each one needs its own control logic and specific types of events. This isn't a lot different than the situation with CEG, where we tend to build specific CEGs for specific events. The code's longer, but only because it's being pre-built. Could I make it somewhat shorter, for slightly easier reading? Yes. Are there good reasons not to? Yes, I'd rather have the power of the full equations in each one- since it's getting run during Initialize, it's not a big deal- even building hundreds of thousands of events only takes a second or two.





As for documentation you're probably right, I should at least explain a few things that aren't entirely intuitive:

1. Most values are restricted to values between 0.0 and 1.0. This is because of how they are written to the P.O.P.S. physics simulation. So, you see values all getting divided by distScale a lot- this is a constant throughout P.O.P.S. code, and you want to use it to reduce values to valid floating-point values.

If you need a negative number, in most cases, negative numbers are between 0.5 and 1.0. This is especially important for position.

2. There are some important exceptions, however:

myVelocity may be any value, positive or negative.

velX,velY and velZ are vector values, with ranges between -1.0 and 1.0. You should think of this as being a combination of CEG's emitVector and emitRot. You can, for example, generate a "ring" of particles on the XZ axis by using the following vel values:

Code: Select all

velX = math.random(-1000,1000)/1000
velY = 1.0
velZ = math.random(-1000,1000)/1000
myCone = math.random(-1000,1000) / 1000
myRand should be left alone unless you want to do something fancy in terms of pseudo-random number generation.

myCone behaves like CEG's emitRotSpread; it is the cone value for the particles.

Color values need a range between 1.0 and 0.0.

One of the cool things about P.O.P.S. is that because it's programmable, it can deal with map light values (the references to sunR, sunG, sunB), which makes an incredible difference with smoke / dust / etc. in terms of feel, on maps with non-standard lighting.

Like CEG, the alpha value determines how much additive saturation occurs. Lower alpha values == higher saturation, so if you make it really low, then it will hit the top values of the RGB range. Obviously this needs to be managed appropriately to get subtle ranges of "light" from your particles over time, so that you don't just white-out, but I am sure that most people can figure that out pretty easily.

Anyhow, hope all that helps clear up some of the confusion :-)
Master-Athmos
Posts: 916
Joined: 27 Jun 2009, 01:32

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Master-Athmos »

Well I might be wrong but as it currently looks for me there are two "problems" - let's start with a quote:
Argh wrote:P.O.P.S. events are passed to the P.O.P.S. core code via the following synced / unsynced communications:

Code: Select all

SendToUnsynced("pops_event", x, y, z,gravity,velX,velY,velZ,size,drag,yadda yadda yadda...)
For each value in this communication, you can set many valid values to produce totally dynamic effects on the fly. IOW, there are no limits, other than the limits on what variables the physics shader and final shader can handle, and those are, of course, modifiable.
Yes I saw that but problem number one gets obvious here. You have lots of fancy variables but one of the most important is missing - the texture. There is no option to actually include a texture of your own which is quite important and ... well I guess I don't need to explain as to why that is. This leads us to my argument of the "premade" stuff. The textures actually get assigned in the Luas defining the particle generator like e.g. for the POPS2_Smoke one you did the definitions pretty much at the beginning...

Code: Select all

local tex1 = "bitmaps/pops/smoke1.tga"
local tex2 = "bitmaps/pops/smoke2.tga"
local tex3 = "bitmaps/pops/smoke3.tga"
local tex4 = "bitmaps/pops/smoke4.tga"

local tex5 = "bitmaps/pops/randdots.tga"
local tex6 = "bitmaps/pops/randdots2.tga"
local tex7 = "bitmaps/pops/circularthingy.tga"
local tex8 = "bitmaps/pops/genericexplosion.tga"
At first I thought there still might be the option to just clone that file with different bitmaps and function names and get the desired textures this way (while of course being an ugly way). But maybe due to the lack of descriptions or me missing something I didn't find any "names" for the effects that would be in the POPSCall.lua. All you have there are what I again would call "premade" functions. Actually I even don't understand how they're called from there as I don't see any references from that file to the actual particle generator definitions just as I don't understand that piece of BOS code. It's obvious that the piecename is there for the spawnpoint but I have no understanding what those two following numbers have to do with defining which effect actually should be spawned...

Maybe all of this is possible but to my mind it's far from being obvious leading to my point of next to no one using this because as it seems to me you'd have to read and understand every single line of your code in order to understand how things are interconnected. That's also why I said that it's not what I had expected because I would have thought that you'd just need to write down some definitions pretty much like in the POPSCall.lua give it a name and then just drop a line in a script or a tag in a weapon definition to make the effect spawn...

EDIT:
I might just have found the right line:

Code: Select all

function CallPOPS(unitID, ud, team, pieceID, popsType, popsPercent)
So the two numbers from the BOS script are popsType and popsPercent. You just defined the popsType numbers afterwards tying it to one of the effects right? For example:

Code: Select all

	if popsType == 1 then
		for i = 1,5 do
			index = math_random(1,1000)

			SendToUnsynced("pops_event", pX, pY, pZ,PB.frothParticles[index].gravity,PB.frothParticles[index].velX,PB.frothParticles[index].velY, and so on
popsPercent,PB.frothParticles[index].startTime,PB.frothParticles[index].finishTime,and so on
PB.frothParticles[index].color1B, PB.frothParticles[index].color1A, PB.frothParticles[index].color2R, PB.frothParticles[index].color2G, PB.frothParticles[index].color2B, PB.frothParticles[index].color2A, PB.frothParticles[index].color3R, PB.frothParticles[index].color3G, PB.frothParticles[index].color3B, PB.frothParticles[index].color3A, PB.frothParticles[index].color4R, PB.frothParticles[index].color4G, PB.frothParticles[index].color4B, PB.frothParticles[index].color4A)
		end

	end
So it would be possible to clone one of the generator definitions with own textures and add them there? But how are the respective effects connected to the generator's lua definition file?
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

I'll see what I can do about the texture situation. I think I may be able to build something suitable in terms of an atlas that will run quickly enough to use. That said, just like with CEG, there's not going to be just random "drop a texture in" for this- unlike LUPS, that's just not practical, due to differences in the way they're designed.
Master-Athmos
Posts: 916
Joined: 27 Jun 2009, 01:32

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Master-Athmos »

My suggestions to make it way more useable:
  • Make the textures a variable
  • Outsource the actual effects definitions you now would put into into the POPSCall.lua into an external definition file that gets included
  • Rather than having to enumerate every single effect in the POPSCall.lua automate this and let people call stuff with names they give their effects as it's no good concept to call the effects with numbers. You'd have to look up the number of the effect you want to use every time once you get a decent amount of different effects.
That would make all of this way more comfortable even for yourself and PURE...
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

Make the textures a variable
That's not possible, at least not "bitmaps/sometexturename.tga" for example. You could have a texture "fireTex" which is at reserved spot 8, and use that variable in place of 8 (which is exactly what CEG / Resources.lua does). That said, when I get a chance to play with a texture atlas, it should get considerably easier (if the strategy I'm thinking about works, of course).
Outsource the actual effects definitions you now would put into into the POPSCall.lua into an external definition file that gets included
That can already be done, pretty much. I think only a minor change to the structure in terms of handling things during Initialize is needed.

Rather than having to enumerate every single effect in the POPSCall.lua automate this and let people call stuff with names they give their effects as it's no good concept to call the effects with numbers. You'd have to look up the number of the effect you want to use every time once you get a decent amount of different effects.
1. COB (not Lua) animation code is forbidden from sending strings to Lua, so far as I know. I'd vastly prefer sending strings, like we do with sounds, but that would require an engine-level change and probably a change to Scriptor (which is very unlikely).

2. You can, at the Lua level, substitute numbers for easier-to-read strings. So effect #2 could be represented by "vehicleDust" or whatever. I just didn't bother doing so because I'm reading the numbers raw from COB and it's easier for me to track "2" through that code, frankly.
User avatar
Argh
Posts: 10920
Joined: 21 Feb 2005, 03:38

Re: P.O.P.S.- what it is, what it's good for, where to get it.

Post by Argh »

Good news!

I got annoyed and built a simple texture atlas.

It's not automatic yet, but it's certainly better than nothing, and allows up to 64 textures to be used without any impact on performance.

I'll see if I can make it automatic with a FBO write operation... later. Kinda exhausted atm, had a long day and this wasn't fun.
Post Reply

Return to “Game Development”