Hi,
I am using VMWare on my server & have a very powerful machine for this, dual processor QUAD CORE, 32GBram & 4TBhd RHEL 5.3 x64.
I am using the host only config, now the problem I am facing is all the VMs are interdependent on each other, a user hit web server & data is fetched from mysql server hosted on another vm the request is sent to a routable IP on the mysql VM & also returned to the web server on routable IP, in this transaction I am loosing bandwidth utilisation, because I have a quota of bandwidth from my Hosting Company, & as soon as I am going above my allocation I am being charged for additional usage.
Now, my question is, how to route traffic between VMs hosted on that one physical machine on non routable IPs or some how internally with in the physical machine so I can save my bandwidth for traffic out only. In vmware server 1.0.9 I can add multiple NICs to a VM at the moment host only interface is added with routable IP to each VM.
Following experiments have done for such purpose.
I have created a dummy interface to primary interface with non routable IP & created 2nd bridge interface on Vmware server & then I use IPtable on physical machine to route data from one source to another source on 2ndry virtual interface on each VM using 2nd dummy bridge interface, I have done that successfully to save bandwidth, but their is one down side of IPtables, I have 8 VMs running, & when traffic increases in a complex situation where mysql, DNS queries, qmail processes with Spam Assassin & file storage, process overhead of IPTables start hammering machine process utilisation up to 90%, to cater down this process, I have another machine with my hosting provider with squid configured as reverse proxy cashing for static contents I have throttle regex filters installed on SQUID so that even squid can not hammer my machine, all traffic is going in / out via SQUID, non of my VM will respond directly accept each other to a request as to protect them from flooding & other attacks because throttle control & other filters from squid are in place, but IPTables no solution, If I use nice between 13 to 17, then again responce is slow down, but still IPtables can hammer down my machine.
So to get rid of IPTable I am forced to route traffic via routable IPs on host only interfaces on VMware Server, so If I increase traffic to a maximum, machine is very quick responsive with 8 VM installed & all of them are interdependent on each other.
I have configured all swap & OS partitions of all VMs to a separate virtual HD & then in return these swap & OS Virtual Hard drives are located to a different physical hard disk from each other so two different physical hard disk drives heads are doing the read & write functions to maximise the response of VM, all the machine are divided to a memory limit & are not exceeding physical limit to RAM & they areworking perfectly fine.
Only issue is internal data routing is required, any one have experience of doing that.