Jump to content


Photo

XenServer 6.2 SP1 Tools

Started by Ivan Hutson , 15 December 2013 - 01:34 AM
42 replies to this topic

Ivan Hutson Members
  • #21

Ivan Hutson
  • 19 posts

Posted 17 December 2013 - 05:00 PM

Alright, forget that VM. 

 

I jumped over to a different VM that has not been messed with after the XenSever SP1 upgrade, put in the new xs-tools.iso and ran installwizard and it blue screens with '0x0000dead (xenvif.sys). 

 

So I reverted to the snapshot I took before doing anything, and put in the XenServer 6.2.0 ISO that has the currently installed out of date tools. Went to \client_install right clicked CitrixXenDriversX64.msi and selected uninstall and it blue screed with the same message. 

 

In both instances if I let the VM come back up after the blue screen XenCenter says 'XenServer Tools not installed.'

 

A quick Google search finds two links.

 

http://www.raido.be/knowledge-center/blog/detail/stop-error-0x0000dead-on-xenserver-6

http://support.citrix.com/article/CTX135099



virtualchunk Members
  • #22

Marcus Schmelzle
  • 396 posts

Posted 17 December 2013 - 10:38 PM

I had the same problem but solved it by installing the 3 msi files one by one and not using the installwizard.msi.

 

Did you try that already? In my case it was a w2k8r2 vm.

 

BR,

-Marco



Marc Marais Members
  • #23

Marc Marais
  • 4 posts

Posted 18 December 2013 - 06:17 AM

My apologies for derailing your thread Ivan :)



Ben Chalmers Citrix Employees
  • #24

Ben Chalmers
  • 19 posts

Posted 18 December 2013 - 09:47 AM

(Possibly stating the obvious here) It sounds like you are hitting a problem uninstalling your 6.2 RTM drivers - however, it is quite possible they have successfully uninstalled in the process - which should then mean that installing the msi's manually (like Marco suggests) will get you back to a fully updated working system.

 

If you have the time, I would be interested in seeing a copy of /var/log/daemon.log from your DOM 0 so that I can identify what caused the crash, and possibly prevent it happening in the future (probably too long to post here - feel free to mail it to me at ben.chalmers@citrix.com )



Ivan Hutson Members
  • #25

Ivan Hutson
  • 19 posts

Posted 18 December 2013 - 02:30 PM

No problem Marc.

 

What ended up working:

  • Booted VM into Safe Mode
  • Uninstalled
    • Windows Driver Package - Citrix Systems Inc. (xenbus)
    • Windows Driver Package - Citrix Systems Inc. (xennet)
    • Windows Driver Package - Citrix Systems Inc. (xenvif)
    • Windows Driver Package - Citrix Systems Inc. (xeniface)
    • Windows Driver Package - Citrix Systems Inc. (xenvbd)
  • Rebooted
  • Uninstalled
    • Citrix Xen Windows x64 PV Drivers
    • Citrix XenServer Tools Installer
    • Citrix XenServer VSS Provider
    • Citrix XenServer Windows Guest Agent
  • Rebooted
  • Put in xs-tools.iso and ran installwizard and everything went fine, rebooted at the end of the installer. Now the current updated tools are installed. 

 

Ben if you would still like to see the /var/log/daemon.log I can send it over to you. Thank you for all of your help. 



Christopher Cabuzzi Members
  • #26

Christopher Cabuzzi
  • 149 posts

Posted 26 December 2013 - 06:17 PM

I'm hoping you don't see this as a thread hijack, but I thought my problem was very similar to yours, and since I found this thread via google search, others may have as well.

 

I've had nothing but issues since installing 6.2 SP1:

 

- The first issue I encountered was that the 2nd server in my pool bombed out after the patch install (yes, I did the master first) and wouldn't connect to the pool because of the "no interface configured" error.  I followed this guide to fix it without having to reinstall: http://www.e-rave.nl/xenserver-6-how-to-fix-no-interface-configured-after-update-or-hotfix

 

- The next issue is that all my servers (Debian, Win2k8, Win2k12, etc...) are horrendously slow.  Just to start the process of updating the XenTools install on my Win2k8 domain controllers took nearly 30 minutes. 

 

- Which leads to the tools install.  The install on the Win2k8 DC stopped after an hour, with a hang on the driver install portion.  I tried forcing a reboot, but it took 30-40 minutes to get to the desktop.  Instead of saying the tools were outdated, it said they were not installed.  I am now in the process of restoring to the snapshot I took right before starting the tool install (thank God I did that), but it's been nearly 30 minutes since I selected "restore", and it still hasn't booted up yet.

 

- The last problem is that I keep losing the connection to the pool master every 10 minutes or so... sometimes, every 1-2 minutes.  In XenCenter, I've just had to right-click and hit connect again, but I've never had this issue before. 

 

If I could get both DCs up and running, I would evacuate them to the 2nd server (I can only imagine how long that is going to take) and reboot the pool master.  That is where I'm at right now.  If anyone can help, I would really appreciate it. 

 

If I should start a new thread, I can do that as well.  It just seemed very related to the OP.  :)

 

** EDIT **

At this point, I'm considering doing a hard reboot of both hosts.  The VMs are so slow right now, they're essentially unusable.  Not sure what else to do.  It has been over 45 minutes, and the restored-from-snap (pre-SP1 tools install) still has not fully booted up.

 

** EDIT 2 **

After a hard reboot of both XenServers an hour ago, things seem to be better.  The VMs are still slow, but they are at least usable now.  I am still dealing with an Exchange server that didn't like the hard shutdown very well, so I'm restoring it from a snap. 

 

I am still very hesitant to install the updated XenTools.  I'm hoping it goes better the next try, which will probably be tonight when everyone is sleeping.  :)



Dean Tyler Members
  • #27

Dean Tyler
  • 1 posts

Posted 28 December 2013 - 10:01 PM

We're having the same/similar problems installing the SP1 tools on Server 2008 R2 VMs. 

 

The XenServer SP1 upgrade went smoothly enough, upgrading from 6.2 and XenCenter reported that the XenServer Tools were out of date.  I tried doing an in-place upgrade using xs-tools.iso and immediately the VM crashed with the 0x0000dead (xenvif.sys) BSOD and after rebooting XenCenter reported that the tools weren't installed at all.  I tried to reinstall them again, and the installer runs for several minutes then reports error 1603 - too many retries.

 

I tried the installation on a second 2008 R2 server (after taking a snapshot this time - always learn from your mistakes! :) ) with exactly the same result.  Once the VMs are in this state, it's extremely difficult to get the tools working again as the tools installation is inconsistent, with different versions of the Windows Driver Package files installed which seems to prevent removal/reinstallation of the tools. 

 

On the snapshot'd server I was able to get it running again by uninstalling the PV drivers, then removing the Windows Driver Package installs in safe mode, deleting the ones that it won't uninstall from the registry, then reinstalling the 6.2 tools, removing them and installing 6.2 SP1.

 

On the first VM I tried I can't get the tools installed no matter what I try, they cannot be repaired or reinstalled with either version of the installer, and if I uninstall the PV drivers and Driver Packages manually then reinstall either 6.2 or 6.2 SP1 the installer runs in a constant cycle of getting as far as "Drivers: Initializing" then rebooting the VM, starting the install again after the reboot, then rebooting at the "Drivers: Initializing" stage - a dozen reboots on it still does the same thing.

 

On a third (and subsequent) VM I completely uninstalled the 6.2 tools first, then installed 6.2 SP1 and everything installed fine.

 

It seems to me the SP1 tool installer is broken, or at least the ability to do an in-place upgrade is.  I'm now stuck with a VM which I can't install the tools on, no matter what I try.  I've tried removing the tools in safe mode/manually, deleting the Citrix devices and drivers (NIC, SCSI card etc), reinstalling 6.2 instead of SP1 etc.

 

Any ideas (or a fixed tools installer!) would be gratefully received!

 

Deano



Christopher Cabuzzi Members
  • #28

Christopher Cabuzzi
  • 149 posts

Posted 29 December 2013 - 06:08 PM

Dean,

 

Sounds very similar to what I experienced.  Unlike you and others though, the installer never actually bombed out with the 1603 error.  It just hung forever until I rebooted the VM, which led to the "no tools installed" state that could not be fixed (as you had with your last VM).  I had a snapshot of that machine, so I reverted it and installed the tools again, which worked this time.

 

The tools install definitely seems inconsistent.  The only difference between the installed that completely jacked up the VM, and the one that went ok, was that I had rebooted the two updated Xen hosts in the pool.  That definitely seemed to help.



Christopher Cabuzzi Members
  • #29

Christopher Cabuzzi
  • 149 posts

Posted 29 December 2013 - 07:10 PM

I might also note, that all of the Win2k8 servers seem to be much slower, and less responsive since the SP1 update.  I'm not sure if that has to do specifically with the tools, or with the hosts themselves.  CPU/RAM utilization is exactly the same as it was before... they are just sluggish... especially our Exchange server (Outlook repeatedly freezes for a few seconds at a time).



Hubertus Feindler Members
  • #30

Hubertus Feindler
  • 62 posts

Posted 30 December 2013 - 10:20 AM

Problems also here: SBS 2011: 

Installation Part 2 hangs. Opening the device manager installs missing vif and vbd drivers. 

Tried uninstalling as described above, also removed directories and registry entries. 

Installation still not completed. XenCenter still shows "XenServer Tools not installed".



konvikkt1 Members
  • #31

kon vikkt
  • 782 posts

Posted 30 December 2013 - 11:13 AM

i had a hard time for my one of the windows 2k8  updating xentools. what worked was :

 

> unsintalled the Av and the backup agent.

> followed the article http://support.citrix.com/article/CTX139031

 

hope this helps.



Christopher Cabuzzi Members
  • #32

Christopher Cabuzzi
  • 149 posts

Posted 03 January 2014 - 05:41 AM

i had a hard time for my one of the windows 2k8  updating xentools. what worked was :

 

> unsintalled the Av and the backup agent.

> followed the article http://support.citrix.com/article/CTX139031

 

hope this helps.

 

Holy moly.  Just read that article.  I cannot stress enough the importance of taking snapshots of VMs prior to upgrading XenTools.  Even though it seemed to take forever to recover, it was still quicker than having to follow that guide.  :)



Will Sours Members
  • #33

Will Sours
  • 13 posts

Posted 23 January 2014 - 02:56 AM

So no one from Citrix chiming in on this? I'm running into very similar / same problems and this should not happen. 



Christopher Cabuzzi Members
  • #34

Christopher Cabuzzi
  • 149 posts

Posted 23 January 2014 - 03:22 AM

@Will
I wish I could provide more insight. At this point, I think of this tool update as one of the various other traumatic points in my life.... like losing my pet cat, wrecking my motorcycle, or my brother dropping my guitar and cracking the neck.

From my post-6.2 SP1 install readings, the only thing I've learned is that the XenCenter disconnects were likely the result of a lack of domain connection, as the DCs were busily chugging away, post-tools install, trying to figure out some unsolvable problem. Once everything was restored from snapshot and rebooted, the install went much better.

The big takeaway for me was that if you want to/need to install these now, it is best to do so after a fresh restart of all your Xen hosts (as well as the VMs). In all the environments I've started the install from fresh reboots, things seemed to go much better.

Will Sours Members
  • #35

Will Sours
  • 13 posts

Posted 23 January 2014 - 06:43 AM

@Will
I wish I could provide more insight. At this point, I think of this tool update as one of the various other traumatic points in my life.... like losing my pet cat, wrecking my motorcycle, or my brother dropping my guitar and cracking the neck.

From my post-6.2 SP1 install readings, the only thing I've learned is that the XenCenter disconnects were likely the result of a lack of domain connection, as the DCs were busily chugging away, post-tools install, trying to figure out some unsolvable problem. Once everything was restored from snapshot and rebooted, the install went much better.

The big takeaway for me was that if you want to/need to install these now, it is best to do so after a fresh restart of all your Xen hosts (as well as the VMs). In all the environments I've started the install from fresh reboots, things seemed to go much better.

 

Its turning me into a nervous wreck. I've now got a SQL server stuck in a boot loop after I thought my new process of uninstalling the old tools, rebooting twice, and installing the new ones was working. Well this install is more unpredictable than a 19 year old girlfriend. Luckily I took a snapshot of this machine first. However I still have the problem of 6.2 SP1 hosted vm's randomly locking up that I was HOPING updating the xentools would fix, but attempting that can result in an hour+ of downtime (if I remembered to take a snapshot), plus resetting my network settings, some of which are rather complex.

 

After the SP1 install I did reboot all my hosts one by one. And that was after I did a rolling pool upgrade from 6.0.0 to 6.2. After SP1 installed and hosts were rebooted I found about 15% of my vm's hung. I could live with that, but this boot looping to update tools and random hangs is killin me. 



Christopher Cabuzzi Members
  • #36

Christopher Cabuzzi
  • 149 posts

Posted 23 January 2014 - 07:40 AM

Yes, I agree it's terrible. This is a huge fiasco, so I'm surprised Citrix isn't getting involved.

The worst is how long it takes to do ANYTHING. As you said, even with a snap to go off, restoring can take upwards of an hour if things go south.

What I meant by upgrade is rebooting the hosts BEFORE you upgrade then, then once again for the rolling upgrade. Next, reboot the VMs before you update their tools. That approach helpede immensely.

CHARLES STEVENSON Members
  • #37

CHARLES STEVENSON
  • 3 posts

Posted 15 August 2014 - 04:00 PM

I also hosed up a domain controller in a new forest I was creating and planning to migrate everything into.  I was remiss in not performing a backup since the upgrade had worked on other DCs, file servers, and Windows 7 vms.  Mine would BSOD and reboot so quickly I couldn't read it.  Eventually it prompted me to boot into Startup Repair.  I entered my administrator password (directory services restore password) and opened a command prompt.  After lots of trial and error and Google searches this is what worked for me (based almost entirely on a post in Experts Exchange http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/Windows_Server_2008/Q_27433521.html#accepted-solution).

 

  1. Boot Domain Controller into Startup Repair
  2. Enter Administrator password (directory services repair)
  3. Open the command prompt and type regedit
  4. Select HKLM node in regedit and click File -> Load Hive
  5. Browse to your OS drive, mine was G:\, into the \Windows\System32\config folder
  6. First backup "SYSTEM" hive (right-click "SYSTEM" & copy & paste)
  7. Open SYSTEM hive
  8. For the Key Name enter Test
  9. Expand HKLM\Test
  10. Use the Find and seach & destroy all Citrix and Xen keys
  11. Highlight HKLM\Test -> File -> Unload Hive
  12. exit command prompt
  13. Restart
  14. System should boot normally and XenTools shouldn't be installed.
  15. Actually backup the VM this time via snapshot or export or copy
  16. Re-attempt XenTools install


James Cannon Citrix Employees
  • #38

James Cannon
  • 4,402 posts

Posted 15 August 2014 - 04:20 PM

Hi Charles,.

 

Thank you for sharing the tip. This will help other forum users.



Michael Martis Members
  • #39

Michael Martis
  • 10 posts

Posted 31 October 2014 - 02:36 PM

BUMP

 

 Bringing the issue back to OP. I recently upgraded from XS 6.02 to 6.2 SP1 and I am no receiving xentools are not installed.

 

I did a complete rip and replace of every Citrix component as well as the updated Citrix tools on the master image (Win 7 SP1) and updated the catalog and VMs

 

No effect.

 

I added the vm-param  device-id  and this made the xentools show installed and proper in xencenter. Xendesktop could once again manage the VMs. However, now the VMs that I add this parameter to prompt to reboot for changes to take effect... EVERY time the machine reboots.  I checked the master to ensure it didn't have any pending reboots and rebooted 3 times for good measure before re-arming.

 

you can check  your VMs for the device ID parameter via:

  xe vm-param-get uuid=UUID   (look for device_id:0001 or 0002 [0001 denotes legacy xen tools])

 

or you can set parameter via:

xe vm-param-set uuid=UUID platform:device_id=0002    

 

Also after changing this , I noticed on a reboot, the VM might bluescreen. If I Power Off the VM and start it from xencenter , it would fix the blue screen magically and I further normal reboots would be fine. 

 

Why does xentools even query the device_id for the xentools? I have two other Pool that are still on 6.0.2 and device_id is not even present in the parameter.

 

 

We're coming up to a year and Citrix Support still hasn't provided any useful insight as to cause and resolution.

 

Come on Citrix, step up already!

 

 

 



James Cannon Citrix Employees
  • #40

James Cannon
  • 4,402 posts

Posted 31 October 2014 - 04:00 PM

Hi Michael,

 

To receive official support from Citrix, you should call toll free number for Citrix. The forums are provided for the benefit of the greater Citrix community. Any participation by Citrix employees is strictly voluntary.