HA Virtual Machine - Not moving to other cluster node
hi,
i have 2 node 2008r2 sp1 datacenter cluster hyper-v role, both nodes pass cluster validity tests etc.
i built cluster no problems encountered, added virtual machine cluster, worked no errors.
as test tried move vm 2nd node. saved ok, failed restore on 2nd node. restored on node1.
i generated cluster.log, event logs visible are
cluster resource 'virtual machine cacti' in clustered service or application 'cacti' failed. event id 1069 (got 1 twice)
the cluster service failed bring clustered service or application 'cacti' online or offline. 1 or more resources may in failed state. may impact availability of clustered service or application. event id 1205
000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] rcm::rcmapi::movegroup: (cacti, 2) 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] rcm::rcmgroup::move: (cacti, 2) 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] rcm::rcmgroup::move: bringing group 'cacti' offline first... 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] transitiontostate(virtual machine cacti) online-->offlinecallissued. 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] rcm::rcmgroup::updatestateifchanged: (cacti, online --> pending) 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] transitiontostate(cacti) online-->waitingtogooffline. 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] bringing dependent resource 'virtual machine configuration cacti' offline before provider resource 'cacti'. 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] 'virtual machine configuration cacti' cannot go offline yet; 'virtual machine cacti' in state offlinecallissued. 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] transitiontostate(virtual machine configuration cacti) online-->waitingtogooffline. 000004d4.00000f08::2011/05/03-13:00:47.130 info [rcm] 'cacti' cannot go offline yet; 'virtual machine cacti' in state offlinecallissued. 00000200.00000eac::2011/05/03-13:00:47.145 info [res] virtual machine <virtual machine cacti>: current state 'online', event 'offline' 000004d4.000011a4::2011/05/03-13:00:47.145 info [rcm] handlemonitorreply: offlineresource 'virtual machine cacti', gen(0) result 997. 000004d4.000011a4::2011/05/03-13:00:47.145 info [rcm] transitiontostate(virtual machine cacti) offlinecallissued-->offlinepending. 00000200.00000804::2011/05/03-13:00:47.535 info [res] virtual machine <virtual machine cacti>: state change 'online' -> 'offlinepending' 00000200.00000460::2011/05/03-13:02:12.680 info [res] virtual machine <virtual machine cacti>: current state 'offlinepending', event 'shutdownresult', result 0x00000000 00000200.00000460::2011/05/03-13:02:12.743 info [res] virtual machine <virtual machine cacti>: current state 'offlinepending', event 'shutdown' 00000200.00000460::2011/05/03-13:02:12.743 info [res] virtual machine <virtual machine cacti>: state change 'offlinepending' -> 'offline' 00000200.00000804::2011/05/03-13:02:12.743 info [rhs] resource virtual machine cacti has come offline. rhs report resource status rcm. 000004d4.000003fc::2011/05/03-13:02:12.743 info [rcm] handlemonitorreply: offlineresource 'virtual machine cacti', gen(0) result 0. 000004d4.000003fc::2011/05/03-13:02:12.743 info [rcm] transitiontostate(virtual machine cacti) offlinepending-->offlinesavingcheckpoints. 000004d4.000011a4::2011/05/03-13:02:12.743 info [rcm] transitiontostate(virtual machine cacti) offlinesavingcheckpoints-->offline. 000004d4.000003fc::2011/05/03-13:02:12.743 info [rcm] 'cacti' cannot go offline yet; 'virtual machine configuration cacti' in state waitingtogooffline. 000004d4.000003fc::2011/05/03-13:02:12.743 info [rcm] transitiontostate(virtual machine configuration cacti) waitingtogooffline-->offlinecallissued. 00000200.00000eac::2011/05/03-13:02:12.743 info [res] virtual machine configuration <virtual machine configuration cacti>: current state 'online', event 'offline' 00000200.00000eac::2011/05/03-13:02:12.743 info [res] virtual machine configuration <virtual machine configuration cacti>: state change 'online' -> 'offlinepending' 000004d4.000011a4::2011/05/03-13:02:12.743 info [rcm] handlemonitorreply: offlineresource 'virtual machine configuration cacti', gen(0) result 997. 000004d4.000011a4::2011/05/03-13:02:12.743 info [rcm] transitiontostate(virtual machine configuration cacti) offlinecallissued-->offlinepending. 00000200.000003dc::2011/05/03-13:02:12.789 info [res] virtual machine configuration <virtual machine configuration cacti>: current state 'offlinepending', event 'configdeleted', result 0x00000000 00000200.000003dc::2011/05/03-13:02:12.789 info [res] virtual machine configuration <virtual machine configuration cacti>: state change 'offlinepending' -> 'offline' 00000200.000007c0::2011/05/03-13:02:12.789 info [rhs] resource virtual machine configuration cacti has come offline. rhs report resource status rcm. 000004d4.00000f80::2011/05/03-13:02:12.789 info [rcm] handlemonitorreply: offlineresource 'virtual machine configuration cacti', gen(0) result 0. 000004d4.00000f80::2011/05/03-13:02:12.789 info [rcm] transitiontostate(virtual machine configuration cacti) offlinepending-->offlinesavingcheckpoints. 000004d4.000011a4::2011/05/03-13:02:12.789 info [rcm] transitiontostate(virtual machine configuration cacti) offlinesavingcheckpoints-->offline. 000004d4.000003fc::2011/05/03-13:02:12.789 info [rcm] transitiontostate(cacti) waitingtogooffline-->offlinecallissued. 0000062c.000010b8::2011/05/03-13:02:12.789 info [res] physical disk <cacti>: offline request. 0000062c.00000f20::2011/05/03-13:02:12.789 info [res] physical disk: driveletter mask: 0x1000 000004d4.00000f08::2011/05/03-13:02:12.789 info [rcm] handlemonitorreply: offlineresource 'cacti', gen(0) result 997. 000004d4.00000f08::2011/05/03-13:02:12.789 info [rcm] transitiontostate(cacti) offlinecallissued-->offlinepending. 0000062c.00000f20::2011/05/03-13:02:12.789 info [res] physical disk <cacti>: harddiskpclosesvihandles: exit 0000062c.00000f20::2011/05/03-13:02:12.867 warn [res] physical disk <cacti>: offlinethread: failed lock volume \device\harddisk3\partition2, error 5 0000062c.00000f20::2011/05/03-13:02:13.179 info [res] physical disk: releasedisk: stop reserve succeeded on device 3 (sig 632c6491) 0000062c.00000f20::2011/05/03-13:02:13.382 info [rhs] resource cacti has come offline. rhs report resource status rcm. 000004d4.00000f08::2011/05/03-13:02:13.382 info [rcm] handlemonitorreply: offlineresource 'cacti', gen(0) result 0. 000004d4.00000f08::2011/05/03-13:02:13.382 info [rcm] transitiontostate(cacti) offlinepending-->offlinesavingcheckpoints. 000004d4.00000f80::2011/05/03-13:02:13.382 info [rcm] transitiontostate(cacti) offlinesavingcheckpoints-->offline. 000004d4.00000f80::2011/05/03-13:02:13.382 info [rcm] rcm::rcmgroup::updatestateifchanged: (cacti, pending --> offline) 000004d4.000003fc::2011/05/03-13:02:13.382 info [rcm] rcm::rcmgum::groupmoveoperation(cacti,2) 000004d4.00000c40::2011/05/03-13:02:13.725 info [gum] node 1: processing requestlock 2:19 000004d4.00000c40::2011/05/03-13:02:13.725 info [gum] node 1: processing grantlock 2 (sent 1 gumid: 963) 000004d4.00000ffc::2011/05/03-13:04:54.093 info [rcm] rcm::rcmgum::groupmoveoperation(cacti,1) 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] rcm::rcmresource::online: bringing virtual machine cacti's provider resource 'virtual machine configuration cacti' online. 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] rcm::rcmresource::online: bringing virtual machine cacti's provider resource 'cacti' online. 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] rcm::rcmresource::online: bringing virtual machine cacti's provider resource 'cacti' online. 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] transitiontostate(cacti) offline-->onlinecallissued. 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] rcm::rcmgroup::updatestateifchanged: (cacti, bringingtopersistentstate --> pending) 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] transitiontostate(virtual machine configuration cacti) offline-->waitingtocomeonline. 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] transitiontostate(virtual machine cacti) offline-->waitingtocomeonline. 0000062c.000010bc::2011/05/03-13:04:54.093 info [res] physical disk <cacti>: online request. 0000062c.00000e78::2011/05/03-13:04:54.093 info [res] physical disk <cacti>: arbitrate request: fastpath 0 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] handlemonitorreply: onlineresource 'cacti', gen(0) result 997. 000004d4.00001130::2011/05/03-13:04:54.093 info [rcm] transitiontostate(cacti) onlinecallissued-->onlinepending. 0000062c.00000e78::2011/05/03-13:04:54.093 info [res] physical disk: enter enumeratedevices: enumdevice 0 0000062c.00000e78::2011/05/03-13:04:54.109 info [res] physical disk: exit enumeratedevices: status 0 0000062c.00000e78::2011/05/03-13:04:54.390 info [res] physical disk: successful reserve, disk unowned, key 5c17466734d 0000062c.00000e78::2011/05/03-13:04:54.390 info [res] physical disk: update disk props returns 0 0000062c.00000e78::2011/05/03-13:04:54.390 info [res] physical disk <cacti>: disk offline 000004d4.00001130::2011/05/03-13:04:54.390 info [gum] node 1: processing requestlock 1:136 000004d4.00000c40::2011/05/03-13:04:54.390 info [gum] node 1: processing grantlock 1 (sent 2 gumid: 980) 0000062c.000013f0::2011/05/03-13:04:54.390 info [res] physical disk <cacti>: completed harddiskpupdateregistryworker (start time 13:04:54.390) 0000062c.00000e78::2011/05/03-13:04:54.405 warn [res] physical disk <cacti>: onlinethread: failed volume guid device \\?\globalroot\device\harddisk3\partition2\. error 3 0000062c.000007f0::2011/05/03-13:04:54.686 info [res] physical disk: pnp: adding volume \\?\storage#volume#{94b07779-60eb-11e0-9bcf-00237dd75120}#0000000008100000#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} 0000062c.000007f0::2011/05/03-13:04:54.686 info [res] physical disk: pnp: add volume exit, status 0 0000062c.00000e78::2011/05/03-13:04:58.415 warn [res] physical disk <cacti>: onlinethread: failed set volguid \??\volume{73577868-322a-441a-940d-3b43a33a6673}. error: 183. 0000062c.00000e78::2011/05/03-13:04:58.415 info [res] physical disk <cacti>: found 2 mount points device \device\harddisk3\partition2 0000062c.00000e78::2011/05/03-13:04:58.415 info [res] physical disk <cacti>: volumeisntfs: volume \\?\globalroot\device\harddisk3\partition2\ has fs type ntfs 0000062c.00000e78::2011/05/03-13:04:58.415 info [res] physical disk: volume \\?\globalroot\device\harddisk3\partition2\ has fs type ntfs 0000062c.00000e78::2011/05/03-13:04:58.415 info [res] physical disk: mountpoint m:\ points volume \\?\volume{73577868-322a-441a-940d-3b43a33a6673}\ 0000062c.00000e78::2011/05/03-13:04:58.415 dbg [clrtl] captureshareinfo: share 0000000000000050, server 0000000000000000, path 0000000000000056 sd 0000000000000000 size 242 0000062c.00000e78::2011/05/03-13:04:58.415 warn [clrtl] sscoreshareadd(): status = 2118 share = m$ server = (null) 0000062c.00000e78::2011/05/03-13:04:58.415 dbg [clrtl] captureshareinfo: share 0000000000000050, server 0000000000000000, path 0000000000000056 sd 0000000000000000 size 242 0000062c.00000e78::2011/05/03-13:04:58.415 info [rhs] resource cacti has come online. rhs report status change rcm 0000062c.00000e78::2011/05/03-13:04:58.415 info [res] physical disk: driveletter mask: 0x1000 000004d4.00001314::2011/05/03-13:04:58.415 info [rcm] handlemonitorreply: onlineresource 'cacti', gen(0) result 0. 000004d4.00001314::2011/05/03-13:04:58.415 info [rcm] transitiontostate(cacti) onlinepending-->online. 000004d4.00001314::2011/05/03-13:04:58.415 info [rcm] transitiontostate(virtual machine configuration cacti) waitingtocomeonline-->onlinecallissued. 00000200.00001140::2011/05/03-13:04:58.415 info [res] virtual machine configuration <virtual machine configuration cacti>: current state 'offline', event 'online' 00000200.00001140::2011/05/03-13:04:58.415 info [res] virtual machine configuration <virtual machine configuration cacti>: state change 'offline' -> 'onlinepending' 000004d4.000003fc::2011/05/03-13:04:58.415 info [rcm] handlemonitorreply: onlineresource 'virtual machine configuration cacti', gen(0) result 997. 000004d4.000003fc::2011/05/03-13:04:58.415 info [rcm] transitiontostate(virtual machine configuration cacti) onlinecallissued-->onlinepending. 00000200.000003dc::2011/05/03-13:04:58.493 info [res] virtual machine configuration <virtual machine configuration cacti>: current state 'onlinepending', event 'configregistered', result 0x00000000 00000200.000003dc::2011/05/03-13:04:58.493 info [res] virtual machine configuration <virtual machine configuration cacti>: state change 'onlinepending' -> 'online' 00000200.0000121c::2011/05/03-13:04:58.555 info [rhs] resource virtual machine configuration cacti has come online. rhs report status change rcm 000004d4.00001314::2011/05/03-13:04:58.555 info [rcm] handlemonitorreply: onlineresource 'virtual machine configuration cacti', gen(0) result 0. 000004d4.00001314::2011/05/03-13:04:58.555 info [rcm] transitiontostate(virtual machine configuration cacti) onlinepending-->online. 000004d4.00001314::2011/05/03-13:04:58.555 info [rcm] transitiontostate(virtual machine cacti) waitingtocomeonline-->onlinecallissued. 00000200.00000ea8::2011/05/03-13:04:58.555 info [res] virtual machine <virtual machine cacti>: current state 'offline', event 'online' 000004d4.00001314::2011/05/03-13:04:58.555 info [rcm] handlemonitorreply: onlineresource 'virtual machine cacti', gen(0) result 997. 000004d4.00001314::2011/05/03-13:04:58.555 info [rcm] transitiontostate(virtual machine cacti) onlinecallissued-->onlinepending. 00000200.0000121c::2011/05/03-13:04:58.571 info [res] virtual machine configuration <virtual machine configuration cacti>: 'virtual machine configuration cacti' registered configuration virtual machine. 000004d4.00001130::2011/05/03-13:04:58.571 info [nm] received request client address 10.4.2.52. 00000200.00000478::2011/05/03-13:04:58.602 info [res] virtual machine <virtual machine cacti>: state change 'offline' -> 'onlinepending' 00000200.00000460::2011/05/03-13:04:58.711 info [res] virtual machine <virtual machine cacti>: current state 'onlinepending', event 'started' 00000200.00000460::2011/05/03-13:05:07.977 info [res] virtual machine <virtual machine cacti>: current state 'onlinepending', event 'startupresult', result 0x00000000 00000200.00000460::2011/05/03-13:05:07.977 info [res] virtual machine <virtual machine cacti>: state change 'onlinepending' -> 'online' 00000200.00000478::2011/05/03-13:05:07.977 info [res] virtual machine <virtual machine cacti>: 'virtual machine cacti' started virtual machine. 00000200.00000478::2011/05/03-13:05:07.977 info [rhs] resource virtual machine cacti has come online. rhs report status change rcm 000004d4.00000f80::2011/05/03-13:05:07.977 info [rcm] handlemonitorreply: onlineresource 'virtual machine cacti', gen(0) result 0. 000004d4.00000f80::2011/05/03-13:05:07.977 info [rcm] transitiontostate(virtual machine cacti) onlinepending-->online. 000004d4.00000f80::2011/05/03-13:05:07.977 info [rcm] rcm::rcmgroup::updatestateifchanged: (cacti, pending --> online) 000004d4.00000f80::2011/05/03-13:05:07.977 info [rcm] group move 'cacti' has completed. 000004d4.00000ce4::2011/05/03-13:05:08.211 info [nm] received request client address 10.4.2.52. 000004d4.00000ef0::2011/05/03-13:06:56.507 info [nm] received request client address clserver1. 000004d4.000005a8::2011/05/03-13:06:58.254 info [cs] cluster service logging level 5 000004d4.000005a8::2011/05/03-13:06:58.254 dbg [gem] node 1: processack node 1 (already acked ()) 000004d4.00000ef0::2011/05/03-13:06:58.254 dbg [causal] node 1: sending message /rcm/causal/eventbatch timestamp 0 704 466 000004d4.00000c8c::2011/05/03-13:06:58.254 dbg [node] node 1: send message of size 0 2 000004d4.00000ef0::2011/05/03-13:06:58.254 dbg [gem] node 1: message added send queue 000004d4.00000c8c::2011/05/03-13:06:58.254 dbg [node] node 1: message node 2 sent 000004d4.00000c8c::2011/05/03-13:06:58.254 dbg [node] node 1: send message of size 322 2 000004d4.00000c8c::2011/05/03-13:06:58.254 dbg [node] node 1: message node 2 sent 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: processack node 2 (already acked (1)) 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: sendnextmessage 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: delivering message 1:305 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [causal] node 1: delivering message timestamp 0 704 466 (my timestamp 0 704 466) 1 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: processack node 1 (already acked ()) 000004d4.00000fec::2011/05/03-13:06:58.269 dbg [em] mscs::eventmanager::eventbatchhandler() 000004d4.00000c8c::2011/05/03-13:06:58.269 dbg [node] node 1: send message of size 0 2 000004d4.00000c8c::2011/05/03-13:06:58.269 dbg [node] node 1: message node 2 sent 000004d4.00000c8c::2011/05/03-13:06:58.269 dbg [node] node 1: send message of size 469 2 000004d4.00000c8c::2011/05/03-13:06:58.269 dbg [node] node 1: message node 2 sent 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: processack node 2 (already acked (1)) 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: sendnextmessage 000004d4.00000c40::2011/05/03-13:06:58.269 dbg [gem] node 1: no more messages in queue 000004d4.000005a8::2011/05/03-13:06:58.269 dbg [ver] calculated cluster versions: highest [major 6 minor 7601 upgrade 7 clusterversion 0x00061db1], lowest [major 6 minor 7601 upgrade 7 clusterversion 0x00061db1] exclude node list: () 000004d4.00000ef0::2011/05/03-13:06:58.269 dbg [api] s_apiclustercontrol(get_common_properties) 000004d4.00000ef0::2011/05/03-13:06:58.269 dbg [api] s_apiclustercontrol(get_common_properties)
ive removed hyper-v role, restarted, reinstalled hyper-v role , error still same.
any ideas on wrong?
phil
ignore msmq errors in cluster log... benign , have no impact.
Windows Server > High Availability (Clustering)
Comments
Post a Comment