Jump to content


Photo

Slow booting again: XenServer 7.0, PVS 7.9

Started by Stephen Borrill , 07 June 2016 - 01:19 PM
9 replies to this topic

Stephen Borrill Members

Stephen Borrill
  • 85 posts

Posted 07 June 2016 - 01:19 PM

I've upgraded from XenServer 6.2 to 7.0 and also upgraded PVS servers to 7.9 (via 7.8 for a day or two).

 

Existing vDisks (PVS 7.1 target device software, XS 6.2 tools) and updated vDisks (PVS 7.8 target software, XS 7.0 tools) both take around 18 mins to boot hanging on the "Starting Windows" screen. Some VMs get stuck and run at 100%, they never successfully boot. Once booted, the vDisk status shows a boot time of around 22 secs and a low uptime. Retries are negligible (0-10). I'm assured (though didn't test it myself) that prior to the XenServer and PVS upgrades, the boot times were nowhere near that high.

 

I've done netio speed tests between VMs, hypervisors and PVS servers; all were decent.

 

I've tried rolling back to 7.1 bootstrap.

 

I've tried creating a new vDisk with a clean Windows installation.

 

I've tried removing 'ghost' network devices in Device Manager.

 

Large Send Offload is disabled both client-side and server-side (and on the hypervisors).

 

I've tried booting from a virtual PVS server on the same hypervisor as a VM.

 

No logs in either client or PVS server event viewer.

 

PVS servers have 6 NICs (Broadcom) teamed with 2012R2 Teaming in switch-independent mode

 



Stephen Borrill Members

Stephen Borrill
  • 85 posts

Posted 07 June 2016 - 02:00 PM

Additional info:

 

Write caches are on local storage on the hypervisors.

 

Setting to device RAM with overflow onto device HDD makes no major difference



Romans Makarovs Members

Romans Makarovs
  • 28 posts

Posted 07 June 2016 - 05:11 PM

Hi Steve,

 

This is just to keep in mind, did you follow correct upgrade steps in vDisk:

 

1. Uninstall PVS Tools

2. Uninstall XS Tools

3. Delete ghost NICs

4. Install XS 7.0 Tools

5. Install PVS 7.9 Tools

 

Just to say, we're PVS7.9 and Hyper-V 2012R2 hypervisor and no issues there in this area

Win7/Win10 vDisks



Stephen Borrill Members

Stephen Borrill
  • 85 posts

Posted 08 June 2016 - 08:05 AM

Had same problem with 7.8 and not seen problems elsewhere with 7.8, just at this site.

 

Note that I tested the old non-upgraded vDisk as well as a clean new vDisk. They both showed the problem, not just the upgraded one, so I'm not convinced that the upgrade steps are the cause. I've not looked too closely at the ghost NICs before as I've never seen them bring a problem. In my original test, I tried deleting them after installing the PVS tools. However:

 

I went back to a re-import of the old vDisk into XenServer and did:

1) Uninstall PVS Device Software

2) Upgrade XS Tools

3) Delete any ghost NICs, leaving Realtek RTL8139C+. N.B. No ghost XenServer NICs at this point

4) Install PVS 7.9 Device Software

 

After installing the PVS software, 9 ghost XenServer NICs appeared. Removing PVS software removed those ghost NICs.

Check Enum\XENVIF shows the extra 9 appearing and disappearing as the tools are installed and reinstalled.



Kuno von Arx Members

Kuno von Arx
  • 2 posts

Posted 08 July 2016 - 11:56 AM

Same Problem:

 

Xenserver 6.5 / PVS 7.9

 

Normaly Clients Boot in 60 Sek.. / now -10 Minutes / reinstall help not /



John Gorena Members

John Gorena
  • 5 posts

Posted 06 August 2016 - 12:59 AM

JG

 

I have the same issue spinned some xendesktop 7.9 via PVS 7.9 and boot time is very slow



Stefan Wagner Members

Stefan Wagner
  • 2 posts

Posted 13 August 2016 - 08:37 AM

Same Problem:

Windows Server 2012 R2 fresh installation takes over 10 min to boot on R730x from Dell.



Luis Gutierrez Members

Luis Gutierrez
  • 2 posts

Posted 04 October 2016 - 03:21 PM

We have same problem with Hyper-V 2012 and PVS 7.6 boot 15 minutes

 

We have changed NIC's

Changed network

Upgrade to PVS 7.9 importing de vDISK

Upgrade windows security patches

 

Nothing working, now we are changing teh DataBase to other segment and also creating a new vDisk.

 

Anyone else have an idea? 



Mark Lininger Members

Mark Lininger
  • 40 posts

Posted 11 February 2017 - 11:05 PM

Has anyone found a resolution here?  We are on XS6.5 and PVS7.9 and 2 XA65 vDisks are taking 10+ minutes to boot.  There are a total of 5 vDisks on those servers, and only the two are having issues.  Manually booting previous revisions are OK, but the latest on the two disks are slow.  Did a packet capture on the PVS server and traffic stops for about 5 Minutes and then starts back up, and at the same time, CPU jumps on the target and eventually boots when in standard mode.  Private mode seems to hang at the black screen indefinitely, but I am going to leave that overnight and see if it eventually boots and gets back to normal.

 

Thanks!

Mark



Stanley Schwartz Members
  • #10

Stanley Schwartz
  • 11 posts

Posted 14 February 2017 - 12:53 PM

Verify you've disabled all of the "Offload" options on the NIC, on both the server side and in your image.  This will cause a LOT of re-transmissions.

 

Even though you have it set to cache locally, make sure it's not writing to a WriteCache directory on your PVS servers.  This could be caused by unformatted cache disks.  If it cant' write to the local cache disks, it defaults to writing it back to the PVS server, effectively taking up all (or most) of the bandwidth.