Jump to content


Photo

Force shutdown of one Linux guest xenserver 5.6

Started by Guest , 30 September 2010 - 06:25 PM
33 replies to this topic

Guest Members

Guest
  • 4 posts

Posted 30 September 2010 - 06:25 PM

Dear Support,
We are having some problems with Linux guest machine. The system sometimes crashes, and there is no chance for killing the machine from Xencenter or ssh.
Even though the console window doesnt work either(white screen).

Here is output from ssh:

+[root@localhost ~]# xe vm-list params=all power-state=running uuid=164314b0-660e-024a-3da2-ca11b6bdeb5d+
uuid ( RO) : 164314b0-660e-024a-3da2-ca11b6bdeb5d
name-label ( RW): tz_test
name-description ( RW):
user-version ( RW): 1
is-a-template ( RW): false
is-a-snapshot ( RO): false
snapshot-of ( RO): <not in database>
snapshots ( RO): dc9cb9dc-8b2a-7cf2-d499-831ccc85d2e2
snapshot-time ( RO): 19700101T00:00:00Z
snapshot-info ( RO):
parent ( RO): dc9cb9dc-8b2a-7cf2-d499-831ccc85d2e2
children ( RO):
is-control-domain ( RO): false
power-state ( RO): running
memory-actual ( RO): 4294881280
memory-target ( RO): <expensive field>
memory-overhead ( RO): 39845888
memory-static-max ( RW): 4294967296
memory-dynamic-max ( RW): 4294967296
memory-dynamic-min ( RW): 4294967296
memory-static-min ( RW): 134217728
suspend-VDI-uuid ( RO): <not in database>
VCPUs-params (MRW):
VCPUs-max ( RW): 4
VCPUs-at-startup ( RW): 4
actions-after-shutdown ( RW): Destroy
actions-after-reboot ( RW): Restart
actions-after-crash ( RW): Restart
console-uuids (SRO): ca63126a-ad59-6849-89d0-ef4ccec7bedb
platform (MRW): timeoffset: 0; nx: false; acpi: true; apic: true; pae: true; viridian: true
allowed-operations (SRO): changing_dynamic_range; hard_reboot; hard_shutdown; pause; snapshot
current-operations (SRO):
blocked-operations (MRW):
allowed-VBD-devices (SRO): <expensive field>
allowed-VIF-devices (SRO): <expensive field>
possible-hosts ( RO): <expensive field>
HVM-boot-policy ( RW): BIOS order
HVM-boot-params (MRW): order: dc
HVM-shadow-multiplier ( RW): 1.000
PV-kernel ( RW):
PV-ramdisk ( RW):
PV-args ( RW):
PV-legacy-args ( RW):
PV-bootloader ( RW):
PV-bootloader-args ( RW):
last-boot-CPU-flags ( RO):
last-boot-record ( RO): <expensive field>
resident-on ( RO): e259bb16-3427-4f37-8fbe-9d4924c9d0e2
affinity ( RW): e259bb16-3427-4f37-8fbe-9d4924c9d0e2
other-config (MRW): last_shutdown_time: 20100930T15:43:20Z; last_shutdown_action: Destroy; last_shutdown_initiator: external; last_shutdown_reason: halted; mac_seed: e64c021b-5479-3d46-b93f-a2e297659eeb; install-methods: cdrom
dom-id ( RO): 29
recommendations ( RO): <restrictions><restriction field="memory-static-max" max="34359738368" /><restriction field="vcpus-max" max="8" /><restriction property="number-of-vbds" max="7" /><restriction property="number-of-vifs" max="7" /></restrictions>
xenstore-data (MRW):
ha-always-run ( RW): false
ha-restart-priority ( RW):
blobs ( RO):
start-time ( RO): 20100930T11:58:59Z
install-time ( RO): 20100930T11:25:48Z
VCPUs-number ( RO): 4
VCPUs-utilisation (MRO): <expensive field>
os-version (MRO): <not in database>
PV-drivers-version (MRO): <not in database>
PV-drivers-up-to-date ( RO): <not in database>
memory (MRO): <not in database>
disks (MRO): <not in database>
networks (MRO): <not in database>
other (MRO): <not in database>
live ( RO): <not in database>
guest-metrics-last-updated ( RO): <not in database>
cooperative ( RO): <expensive field>

+[root@localhost ~]# xe task-list+

+[root@localhost ~]# xe vm-list+
uuid ( RO) : a20ec112-f77f-4cb4-81ae-d5188f48060c
name-label ( RW): Control domain on host: xenserver-crtjoirk
power-state ( RO): running

uuid ( RO) : 164314b0-660e-024a-3da2-ca11b6bdeb5d
name-label ( RW): tz_test
power-state ( RO): running

uuid ( RO) : e076df94-3c3b-a336-24ff-83ec13484783
name-label ( RW): test
power-state ( RO): running

+[root@localhost ~]# xe vm-shutdown vm=tz_test force=true+
Another operation involving the object is currently in progress
class: VM
object: 164314b0-660e-024a-3da2-ca11b6bdeb5d (tz_test)

XenServer is free edition for testing purposes. And we got about seven Enterprise licenses for further purposes, espescially linux based web hosting.

The answer which we are looking for is the way to restart one vm, while not rebooting other machines.

Regards



Tobias Kreidl CTP Member

Tobias Kreidl
  • 17,935 posts

Posted 30 September 2010 - 08:15 PM

Sounds like one of these should help:

xe vm-reboot vm=(name or UUID of VM) --force

xe vm-reset-powerstate vm=(name or UUID of VM) --force

Both are commands that should only be done in emergency (see the "xe help vm-reboot" and
"xe help vm-reset-powerstate" for warning s about possible consequences).

Good luck,
--Tobias



Guest Members

Guest
  • 4 posts

Posted 01 October 2010 - 06:17 AM

Thanky You for post,
Unfortunately the both commands doesnt work.

+root@localhost ~# xe vm-reset-powerstate vm=tz_test force=true+
The operation could not be performed because a domain still exists for the specified VM.
vm: 164314b0-660e-024a-3da2-ca11b6bdeb5d (tz_test)
domid: 29

*root@localhost ~# xe vm-reboot vm=tz_test force=true*



Tobias Kreidl CTP Member

Tobias Kreidl
  • 17,935 posts

Posted 01 October 2010 - 06:44 AM

Try running "xe-toolstack-restart" from the CLI, and then try these two commands again and see if that helps. If not, you may have a hung task -- to see how to kill it, use "xe task-list" to view any running tasks, and if you see an incompleted one you recognize, you can try to kill it using "xe task-cancel uuid=(UUID of running task)".
--Tobias



Guest Members

Guest
  • 4 posts

Posted 01 October 2010 - 11:22 AM

we have done everything You posted before, my first post here.

http://balld31.blogspot.com/2010/06/vm-stuck-in-pending-state-on-xenserver.html
http://lists.xensource.com/archives/html/xen-users/2010-02/msg00325.html
http://debaan.blogspot.com/2009/05/xenserver-vm-wont-shutdown.html

None of those tips did work, thats why we wrote here.

Funny thing is the console from other machine(created after freezing) was displayed in the window of the frozen one.

Under xentop there was the process of vm (tz_test) and i thought there must me a way to kill the process.



Roland Pope Members

Roland Pope
  • 2 posts

Posted 16 November 2010 - 09:53 PM

I have the same issue.This only seems to occur with Linux HVM guests. What triggered this for me was when I lost access to the iSCSI NAS that was running the system disks for my VM's. All my Linux PV guests dropped the file system into read-only mode and kept on working, but the fully virtualised Linux guests became totally unresponsive and unable to be used. The only solution I found at this point was to reboot the Xenserver host that was running the wedged guests, which was not ideal.



CRAIG BERG Members

CRAIG BERG
  • 7 posts

Posted 01 February 2011 - 07:58 PM

I also had this issue twice now. The first time i was able to shut down the running vm's and reboot the host. But now that a VM froze during business hours I can't reboot the host without taking down other VM's and i don't have the swing space to migrate all the VM's to another host.

After i run xe vm-shutdown force=true i get "Another operation involving the object is currently in progress"

I then run xe task-list and there are no tasks listed other than XenCenter connections.

Any help would be appreciated.

Thanks in advance

-Craig



Jason Fairburn Members

Jason Fairburn
  • 3 posts

Posted 13 April 2011 - 02:11 AM

I am in the same situation.

I would very much like some assistance with this. there are three Linux VM's in my 5.6FP1 environment, and they are the only ones which crash (40+ windows servers are all OK).

its becoming critical.

any assistance there Citrix? thanks

Jason



Quoc Hieu Tran Members

Quoc Hieu Tran
  • 1 posts

Posted 13 April 2011 - 04:29 AM

Hi all!
I have a problem with my VPS. I am using Xenserver. when I created a new VM, but it wasn't successfull. Now I want to shutdown it ,then would delete it.
I cound not shutdown it because my VPS has a task is pending.
[root@123 ~]# xe task-list
uuid ( R0) : 0fc5b8-0b-850a-ffb6-e2ab....
name-lable ( R0) : Async.VM.start
name-description ( R0) :
status ( R0) : pending
progress ( R0) : 0.000
then:
[root@123 ~]# xe task-cancel force=true uuid=0fc5b8-0b-850a-ffb6-e2ab....
[root@123 ~]# xe task-list
uuid ( R0) : 0fc5b8-0b-850a-ffb6-e2ab....
name-lable ( R0) : Async.VM.start
name-description ( R0) :
status ( R0) : pending
progress ( R0) : 0.000
Please help me kill pending status.
thanks all very much!



Hung Pham Members
  • #10

Hung Pham
  • 94 posts

Posted 20 April 2011 - 06:41 AM

It seems I am having the same issue. I could not kill pending tasks. Whenever I run a "xe task-cancel" command, it creates another pending task.

Is there any fix for this yet?



Chris Goffin Members
  • #11

Chris Goffin
  • 35 posts

Posted 28 April 2011 - 05:53 PM

Run a xe-toolstack-restart on all of your XenServer hosts doing the pool master last. Keep in mind that when you do this on the pool master, you'll lose XenCenter for a few moments but just reconnect and you're good to go. Your VMs will not be impacted by running this command.



Angel L. Mateo Martínez Members
  • #12

Angel L. Mateo Martínez
  • 30 posts

Posted 12 May 2011 - 08:50 AM

I have the same issue. My problem could be a connection lost with the iscsi server too (but I'm not 100% sure).

I have tried to xe-toolstack-restart in all pool members and last in the master, but the problem is still there. I have to mannually reboot the server.



Falko Haufe Members
  • #13

Falko Haufe
  • 406 posts

Posted 18 May 2011 - 11:54 AM

Hi,

i am currently sitting in a Toolwirelab during a XD5 course and i experience the same problem. I have 2 VM with Win7 (pooled, 2 out of 4) and i tryed to restart the vms, restarted xenservices but without any luck. I had to reboot the whole server to get this issue sorted out.

Regards,
Falko



Dave Hildebrand Members
  • #14

Dave Hildebrand
  • 17 posts

Posted 22 June 2011 - 05:22 PM

Did you have to hard reboot your server, i.e. press the power button until it turns off?



Mike Burk Members
  • #15

Mike Burk
  • 6 posts

Posted 08 July 2011 - 09:02 PM

Same problem here, went through every step recommended by this and other threads.
Created another thread about it: http://forums.citrix.com/thread.jspa?messageID=1567174�



Mike Burk Members
  • #16

Mike Burk
  • 6 posts

Posted 15 July 2011 - 10:04 PM

I've found the resolution through my other thread. Here's what you need to do.

1 - "xe vm-list" to get the uuid of the VM that is hung

2 - "list_domains" to list the domain uuid's so you can determine
the domain # of the VM above by matching the uuids from this output with
the uuid for your VM from the previous command.

3 - "/opt/xensource/debug/destroy_domain -domid XX" where XX is the domain number from the previous command

4 - "xe vm-reboot uuid=XXXX --force" where XXXX is the uuid from the first vm-list command for your VM. (name-label may work but didn't work this time for me so I used the uuid)

Your VM will be back up running now in most cases. (I use this for my HVMs that hang so I don't have to reboot the host server)



Craig Scott Members
  • #17

Craig Scott
  • 3 posts

Posted 25 July 2011 - 02:46 AM

Thanks Webmaster@dawnet.net

I can confirm that this worked and fixed the service issue.



Joel DAVIS Members
  • #18

Joel DAVIS
  • 202 posts

Posted 25 July 2011 - 02:37 PM

Had a similar problem with a Windows VM and webmaster@dawnet.net's advice worked perfectly.



Uwe Hentzschel Members
  • #19

Uwe Hentzschel
  • 2 posts

Posted 10 August 2011 - 08:34 AM

I had the same problem and the solution works perfect.



Jean-Pierre Bergamin Members
  • #20

Jean-Pierre Bergamin
  • 19 posts

Posted 01 September 2011 - 09:08 AM

Webmaster@dawnet.net's solution also worked here. Two VMs hung and could not be rebooted.