optimization suggestions for springrts.com
Moderator: Moderators
Re: optimization suggestions for springrts.com
Is the windows vm is needed to run something besides the ZeroK site / planetwars and the springies?
Is renting two (smaller) boxes, one of which could then run windows as host OS, a possibility?
Is renting two (smaller) boxes, one of which could then run windows as host OS, a possibility?
Re: optimization suggestions for springrts.com
It is a possibility but:
* its not as cost efficient. Hardware like CPU is mostly underused and its better to share it on same machine
* there is windows licensing issue. With OVH if you want windows on dedicated server you have to use one of their editions which is then leased at relatively high price. With VM you can use your own edition (in my case old but working win2k3)
* its not as cost efficient. Hardware like CPU is mostly underused and its better to share it on same machine
* there is windows licensing issue. With OVH if you want windows on dedicated server you have to use one of their editions which is then leased at relatively high price. With VM you can use your own edition (in my case old but working win2k3)
Re: optimization suggestions for springrts.com
Never realized the additional per month cost cause it's not immediately visible on their site. The rate for the "smallest" win2008 server edition (whateverthefuck that exactly is) is 20E/month which looks like retail price/24.Licho wrote: there is windows licensing issue. With OVH if you want windows on dedicated server you have to use one of their editions which is then leased at relatively high price. With VM you can use your own edition (in my case old but working win2k3)
And in broader terms, does anyone actually have a complete picture of what services/sites currently run?koshi wrote:Is the windows vm is needed to run something besides the ZeroK site / planetwars and the springies?
Re: optimization suggestions for springrts.com
I know what you can do with virtualization as I know we have to host windows apps but I asked for performance concerns.dansan wrote:There can be various reasons for that:
* apps require incompatible library versions
* you need different OS
* you need same OS but different kernel
* you need 32 bit and 64 bit software
* it enables high(er) availability setups
-- you can migrate a running VM from one hardware to another (to fix/upgrade/modify the previous hardware or to distribute load)
-- if a hardware fails hard, the VM can _quickly_ be restarted on another host
* it is "easy" to scale horizontally and vertically (more hosts for a cluster or more CPU/mem/HDD for a host)
-- this scaling can often be done without rebooting
* kernel upgrade requires reboot - you want to stop only as few service as possible - thus rebooting one VM at a time
* migrating a VM from one hardware to another only requires to setup the host system, as the VM does not (need to) know about the real hardware
* security:
-- restrict admins to work only in the VM they need to
-- easy and fast backups with snapshots
-- cracking/infecting 1 VM does not affect the other systems (web servers are more or less open doors - don't put your mail server on them, maybe not even the database)
-- possibly stricter firewall configurations
* distributing hardware differently for different work loads: web servers need lots but slow CPUs, databases fast CPUs and small, low latency disks, file servers need few and slow CPU and big, high throughput disks
In the end the overhead of a extra Linux-VM is just around 16-64 MB and additional admin time - that's not relevant when you have 8192 MB. It pays off in terms of manageability, uptime and security.
All your points are about compatibility, manageability, security, scalability. There is nothing about pure performance benefits from virtualized architecture no matter what the topic title says.
When I read 1 app per VM, I understood your point was not performance cause fully splitted ressources can never achieve better performances for a given hardware. You tell about application needs in very simple way and yet makes mistakes (databases require memory before disk).
Learn to tune an OS and its apps before playing with VMs because it's the last thing in vogue.
Re: optimization suggestions for springrts.com
scalability is about performance.
you act as if the goal is, to maximize performance on a single set of HW, but that is not the case. it would be per cost, not per single set of HW, plus there are many other concerns then performance, namely.. everything that dansan listed.
you act as if the goal is, to maximize performance on a single set of HW, but that is not the case. it would be per cost, not per single set of HW, plus there are many other concerns then performance, namely.. everything that dansan listed.
Re: optimization suggestions for springrts.com
Scalability is one way to improve performance.
What I say is paying for huge misused hardware is bad. I have the per cost/performance in mind and it doesn't seem to be the goal when I read one app per VM.
Am I the only one against virtualization for virtualization ?
What I say is paying for huge misused hardware is bad. I have the per cost/performance in mind and it doesn't seem to be the goal when I read one app per VM.
Am I the only one against virtualization for virtualization ?
Re: optimization suggestions for springrts.com
Well.. first of all work time (management, administration) and security is much more expensive than hardware. So the rest of this post is just to please you (no offense intended), not because I think single-host-performance is an issue. Performance must be good ofc, but other things are as or more important (manageability, availability and security).
But back to why performance can even be better with more VMs:
* a hung process using all CPU/memory/disk on one VM cannot kill another VM
How important is this? --> The lobby server on Lichos server would not get killed sometimes by apache/backup overload --> higher performance and availability for the lobby server!
* Sometimes you have to lock entire databases to make a consistent backup. That really hurts performance as no write access is possible at that moment and reading is slow. If you replicate the DB to another VM and make the backup there --> no locking on the main DB --> higher performance and availability for the primary DB.
Masure wrote:What I say is paying for huge misused hardware is bad. I have the per cost/performance in mind and it doesn't seem to be the goal when I read one app per VM.
BTW: 1 app / VM is in the case of limited resources a little bit extreme. As Linux and DB performance benefit a lot from free RAM, a compromise is the normal case: put some stuff together in a VM to save RAM to boost performance by having less disk-IO.dansan wrote:In the end the overhead of a extra Linux-VM is just around 16-64 MB [RAM]
But back to why performance can even be better with more VMs:
* a hung process using all CPU/memory/disk on one VM cannot kill another VM
How important is this? --> The lobby server on Lichos server would not get killed sometimes by apache/backup overload --> higher performance and availability for the lobby server!
* Sometimes you have to lock entire databases to make a consistent backup. That really hurts performance as no write access is possible at that moment and reading is slow. If you replicate the DB to another VM and make the backup there --> no locking on the main DB --> higher performance and availability for the primary DB.
I made very clear, that anything that is not needed should be avoided - that includes OS virtualization.Masure wrote:Am I the only one against virtualization for virtualization ?
Re: optimization suggestions for springrts.com
I understand 100% CPU usage by a process penalizes the others. I'm not a linux user but I assume Linux has some fine process prioritization configuration. That should avoid 1 single process killing the others.
Sorry for obviousness but why don't you focus on the hanging process ? An hanging process is not something that should happen and I would first dig hard into this instead of buying hardware and using virtualization to bypass the problem.
No offense (really) but do you really know what you're talking about with backup consistency & locking concerns ? Giving virtual solutions doesn't solve real issues... Some database providers implement consistent and non blocking backups.
Sorry for obviousness but why don't you focus on the hanging process ? An hanging process is not something that should happen and I would first dig hard into this instead of buying hardware and using virtualization to bypass the problem.
I don't wanna get into details but you won't achieve better backup consistency with VM replication. The problem is the same with a different backup target. If you're able to make a snapshot without lock to replicate your VM, you can do the same with database data.* Sometimes you have to lock entire databases to make a consistent backup. That really hurts performance as no write access is possible at that moment and reading is slow. If you replicate the DB to another VM and make the backup there --> no locking on the main DB --> higher performance and availability for the primary DB.
No offense (really) but do you really know what you're talking about with backup consistency & locking concerns ? Giving virtual solutions doesn't solve real issues... Some database providers implement consistent and non blocking backups.
Re: optimization suggestions for springrts.com
Most databases allow non blocking backup or replication.
If not you can use shadow volume copy (in windows at least) to preserve current consistent disk state and copy it while allowing continuous operation.
If not you can use shadow volume copy (in windows at least) to preserve current consistent disk state and copy it while allowing continuous operation.
Re: optimization suggestions for springrts.com
That's what I saidLicho wrote:Most databases allow non blocking backup or replication.
If not you can use shadow volume copy (in windows at least) to preserve current consistent disk state and copy it while allowing continuous operation.
Re: optimization suggestions for springrts.com
A thing that happens when you have 1 process @100% is, that the latency climbs. Ofc the other processes will not get killed by that, but your desired quality level will maybe not be met anymore.Masure wrote:I understand 100% CPU usage by a process penalizes the others. I'm not a linux user but I assume Linux has some fine process prioritization configuration. That should avoid 1 single process killing the others.
But that is still not scary... the biggest thread comes from processes eating your memory. First your server begins to swap - everything becomes slow, especially remotely logging in and administering. Then at some point the OOM-killer (Out Of Memory) becomes active and it's not a smart piece of software...
Another thing crazy software does is write GBs of log files - until the partition is full. Then suddenly lots of services begin to fail... some on purpose (for security considerations) others for being lazily programmed... in this situation data loss is common.
Ofc you should dig into this, but: It's already to late when it happens. And there is simply no software without bugs.Masure wrote:Sorry for obviousness but why don't you focus on the hanging process ? An hanging process is not something that should happen and I would first dig hard into this instead of buying hardware and using virtualization to bypass the problem.
Administrators must be paranoiac, because everything has bugs, and everything that can go wrong will go wrong at some point.
Please read carefully - I don't want to snapshot or replicate the VM. I want to contiguously replicate the DB to another VM (cluster style). There the locking/backup can take place.Masure wrote::lol: I don't wanna get into details but you won't achieve better backup consistency with VM replication. The problem is the same with a different backup target. If you're able to make a snapshot without lock to replicate your VM, you can do the same with database data.dansan wrote: * Sometimes you have to lock entire databases to make a consistent backup. That really hurts performance as no write access is possible at that moment and reading is slow. If you replicate the DB to another VM and make the backup there --> no locking on the main DB --> higher performance and availability for the primary DB.
Exactly: some. If you come across a DB that does not (some storage engines in MySQL, OpenLDAP), then you need locking.Masure wrote:Some database providers implement consistent and non blocking backups.
Also: What if your DB has a size of 50 GB? You think you can make a backup of this on your main server without heavy performance loss?
Given you have no idea what you're talking about as you obviously have no real world admin experience, and then write a sentence like this, I guess you're a troll and so I stop talking to you.Masure wrote:Giving virtual solutions doesn't solve real issues...
LVM can do that on Linux, but unfortunately it doesn't help with non-blocking, consistent DB backup.Licho wrote:shadow volume copy
I use it a lot to make backups of all running VMs: snapshot the VM-disk in the host, mount the snapshot, make backup.
Re: optimization suggestions for springrts.com
Server is still getting apache memleak!
Its not as severe as with SF, but it still means 3-4 apaches with 200mb ram each..
Why is it happening? What exactly was changed?
Its not as severe as with SF, but it still means 3-4 apaches with 200mb ram each..
Why is it happening? What exactly was changed?
Re: optimization suggestions for springrts.com
i guess there is some mem-leak in php.
so lowering MaxRequestsPerProcess could help in /etc/apache2/mods-enabled/fcgi.conf
imo also disabling/removing some php-modules could fix this too.
( /etc/php5/cgi/conf.d )
so lowering MaxRequestsPerProcess could help in /etc/apache2/mods-enabled/fcgi.conf
imo also disabling/removing some php-modules could fix this too.
( /etc/php5/cgi/conf.d )
Re: optimization suggestions for springrts.com
I found this page: http://serverfault.com/questions/88997/ ... hp-based-w - maybe it helps...
They advise to lower MaxRequestsPerChild (relation to MaxRequestsPerProcess?) and to find the problem do strace - have fun with that :P
They advise to lower MaxRequestsPerChild (relation to MaxRequestsPerProcess?) and to find the problem do strace - have fun with that :P
Re: optimization suggestions for springrts.com
found a solution that allows replacement of suPHP:
apache2-mpm-itk
seems to work at springfiles.com + its easy to setup. it also allows to user mod_php... this way you also can see, which user is using what resources. so exactly what you want!
apache2-mpm-itk
seems to work at springfiles.com + its easy to setup. it also allows to user mod_php... this way you also can see, which user is using what resources. so exactly what you want!
Re: optimization suggestions for springrts.com
Ok thats cool, i will try looking into it in a few months when i get more time (unless someone else will).
I also reduced max requests to 50. Does not seem to be helping much
I also reduced max requests to 50. Does not seem to be helping much