I am running VMware Server 1.0.3 build 44356 host on a Windows Server 2003 Std with SP2, with two guests: (1) a Windows Server 2003 that serves as our primary domain controller and (2) a Linux based server with Red Hat as our web server for our in-house web page and email server. I recently set up a second Windows Server 2003 R-2, SP2 as a second domain controller for redundency sake. This second server is not a virtual server that also serves as our anti-virus server (Kaspersky) and Ghost imaging server. Until recently it was a stand alone member server in our domain, and no issues occurred until I promoted it to a DC with Active Directory shared from our virtualized primary DC.
Now, on occasion, we have had two seemingly random outages which have slowed the Linux based web server to a halt causing our client PCs to not access the in-house web page and email program. Upon viewing the VMware logs for that server I find the following:
.....
Nov 19 16:28:11: vmx| SCSI0:0: Command READ(10) took 50.973 seconds (ok)
Nov 19 16:29:21: vmx| SCSI0:0: Command WRITE(10) took 34.574 seconds (ok)
Nov 19 16:29:21: vmx| SCSI0:0: Command WRITE(10) took 34.574 seconds (ok)
Nov 19 16:29:21: vmx| SCSI0:0: Command WRITE(10) took 34.574 seconds (ok)
Nov 19 16:29:55: vmx| SCSI0:0: Command READ(10) took 34.183 seconds (ok)
Nov 19 16:29:58: vmx| SCSI0:0: Command READ(10) took 2.195 seconds (ok)
Nov 19 16:30:01: vmx| SCSI0:0: Command READ(10) took 2.342 seconds (ok)
Nov 19 16:31:05: vmx| SCSI0:0: Command WRITE(10) took 65.327 seconds (ok)
Nov 19 16:31:05: vmx| SCSI0:0: Command WRITE(10) took 65.327 seconds (ok)
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 101.861 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37944340 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 101.974 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37941820 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 101.974 seconds (ok)
....
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37941460 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37944280 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37942000 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37944100 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37943c80 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37942420 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.998 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37943ce0 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.999 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37942cc0 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.999 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x379421e0 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.999 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37943320 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 90.999 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37942120 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 36.646 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37941880 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0:0: Command WRITE(10) took 36.646 seconds (ok)
Nov 19 16:31:42: vcpu-0| LSI: CCB 0x37943920 cmd 0x2a btstat 0 sdstat 0 resid 0 completed during target or adapter reset
Nov 19 16:31:42: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:42: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:43: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:45: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:45: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:46: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:57: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:58: vcpu-0| SCSI0: RESET BUS
Nov 19 16:31:58: vcpu-0| SCSI0: RESET BUS
Nov 19 16:32:02: vcpu-0| SCSI0: RESET BUS
Nov 19 16:32:03: vcpu-0| SCSI0: RESET BUS
Nov 19 16:32:03: vcpu-0| SCSI0: RESET BUS
Nov 19 16:32:42: vcpu-0| SCSI0: RESET BUS
Nov 19 16:32:42: vcpu-0| SCSI0: RESET BUS
Nov 19 16:32:43: vcpu-0| SCSI0: RESET BUS
Nov 19 16:39:57: mks| SOCKET 1 recv error 10054: An existing connection was forcibly closed by the remote host
Nov 19 16:39:57: mks| SOCKET 1 destroying VNC backend on socket error: 10054
At this point the only way to correct this issue was to physically reboot the Host server which corrected the problem each time.
Is this an issue related to promoting the stand alone server to a second domain controller? Does this cause the Linux Virtual guest to use too many resources on the host or is the combination of the two virtual guests and second DC using too many host resources? Should I demote the second DC back to a member server? Or, preferably, can anything else be done to correct this issue within VMware or anywhere for that matter? Any help would be greatly appreciated.
Also, I have attached the latest VMware log for the Linux server in its entirity.
Thank you!