Tips for Setting Up NTP on XenServer Hosts

[Tobias Kreidl | 9,492]

Greetings:
There are many threads on this forum and information elsewhere dealing with NTP (Network Time Protocol) issues, so I thought I'd share a few things that I've learned over the years and summarize a number of these tips and experiences. These practices pertain, of course, to Linux OS installations, in general.

Part 1

NTP synchronization is important for a number of reasons. For one, XenServer depends internally on having members of pools close in time as they are intercommunicating important information over the primary management network and Citrix mandates that for proper functionality that this be done.

There are two general ways of setting up NTP. One is to not run the NTP daemon (ntpd) and simply run a periodic "ntpdate ntp-server-host" to update the clock. This is not the standard way of doing things, since it doesn't take into account issues such as latency or internal clock drifts. It is, however, a good one-time means of synchronizing the server's clock if it's a new installation or is way off for some other reason. Adding a host to a pool that has a large time offset can in of itself lead to issues, so it's a good idea to make sure the time is set close. With the ability to storage Xenmotion running VMs even to different pools, keeping all XenServers (as well as other servers that support Citrix and other products and services) all in proper time sync is as important as ever. To check how far a host is off from the local NTP standard setting, run "ntpstat -s" from the CLI.

The preferred way of implementing NTP is to run the NTP daemon (ntpd), which gets its settings from the /etc/ntp.conf file. This is pre-configured with some basic settings as delivered with a XenServer installation.
NTP depends on external machines to be used for synchronization -- using three is recommended, as (1) a single host may be unavailable at one time or another, (2) using more than one allows NTP to find the "best" source, and (3) it delivers peace of mind to see when all NTP servers are close to each other, providing confidence that all is working correctly,

As to the selection of NTP servers, one can of course choose any ones, as long as they are accessible (they use tcp/udp port 123 (or if SSL is used, 563) for communication. The typical setup involves establishing a secondary NTP server group within one's organization, that in turn receives its input from "official" external NTP sources. A comprehensive list of suggested servers can be found, for example, at http://pool.ntp.org/ or specifically for the US, the National Institute of Standards and Technology provides a suggested list by region at http://tf.nist.gov/tf-cgi/servers.cgi. The primary reason to synchronize to a set of local NTP servers within one's organization is that in the event of Internet connectivity issues, the local hosts can continue to internally synchronize to the same common source, as long as Intranet functionality is still present.

This is all well and good, but what happens if there is a partial outage within one's organization that prevents access to the local NTP servers? If short, the systems can of course continue to still keep reasonable time, but computer clocks are notoriously not that accurate (you can buy a $10 watch that keeps better time!). One very simple option is to make use of the "peer" feature within ntpd. This is as simple as adding a line for each of the hosts within, say a single XenServer pool, which all share the same management interface. The purpose of the peer option is to provide a means of at least keeping the server pool internally synchronized in such a case. The ntpd is smart enough to assign peers a high stratum value (basically, a hierarchy of dependencies off the primary source machines, which are at stratum 0; the greater the stratum value, the further down the chain lies the machine). Typical machines used to connect to directly have a stratum of 2 - 4.

When access to NTP servers within one's organization fail, the peer mechanism takes over. The host itself will negotiate with the other named peers and together they will all decide on common primary and secondary sources. These are combined in a weighted scheme to come up with a single time that's used for synchronizing all members that share this same group of sources (be they real NTP server or just peers). When the NTP server once again become available, ntpd will automatically switch back and recalibrate the polling times (which initially drop to much shorter time intervals) and reset the server clocks.

As an example of a basic setup, here is a sample /etc/ntp/conf file and note the in-line comments:

# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
#===# this restricts who can talk to this server to obtain NTP information
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
#===# self-explanatory
restrict 127.0.0.1
## restrict -6 ::1

# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use Xen's public servers.
#===# This is in case your organization does not have its own internal NTP servers.
#broadcast 192.168.1.255 key 42 # broadcast server
#broadcastclient # broadcast client
#broadcast 224.0.1.1 key 42 # multicast server
#multicastclient 224.0.1.1 # multicast client
#manycastserver 239.255.254.254 # manycast server
#manycastclient 239.255.254.254 key 42 # manycast client

# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available.
## server 127.127.1.0 # local clock
#===# I add this to force the local machine to have a low stratum compared to external NTP servers.
fudge 127.127.1.0 stratum 10

# Drift file. Put this in a directory which the daemon can write to.
# No symbolic links allowed, either, since the daemon updates the file
# by creating a temporary in the same directory and then rename()'ing
# it to the file.
#===# This is where your machine stores its drift file, which is the estimated clock frequency error
driftfile /var/lib/ntp/drift

# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography.
keys /etc/ntp/keys

# Specify the key identifiers which are trusted.
#trustedkey 4 8 42

# Specify the key identifier to use with the ntpdc utility.
#requestkey 8

# Specify the key identifier to use with the ntpq utility.
#controlkey 8

### our local specifics ###
#===# This deals with how self-modifying the local server can be to its own settings, as well
#===# as not permitting external queries.
restrict X.Y.0.0 mask 255.255.0.0 nomodify notrap
restrict 10.15.9.0 mask 255.255.255.0 nomodify notrap

#===# A list of your local (organizational) NTP servers
server ntp1.mydomain.org
server ntp2.mydomain.org
server ntp3.mydomain.org

#===# Here is the option to add local peers (within a subnet, server pool, etc.)
# peers in case the standard NTP servers cannot be reached. These are the three XenServers in the pool.
peer 10.15.9.17
peer 10.15.9.18
peer 10.15.9.19

 

That's basically it. Anytime a change is made, it's a good idea to restart ntpd: "service ntpd restart".

What does a typical running system deliver in terms of stability? To be continued in the next post, as space has run out...

[Tobias Kreidl | 9,492]

Part 2

A very useful command is "ntpq". To show peers, as well as our main NTP servers, we can issue the command "ntpq -p" and here is a sample output:

 

remote refid st t when poll reach delay offset jitter
==============================================================================
+ntp1.mydom.org XXX.YYY.4.101 2 u 315 1024 377 1.033 -1.177 0.715
+ntp2.mydom.org.XXX.YYY.5.210 2 u 78 1024 377 1.132 -0.887 0.647
*ntp3.mydom.org XXX.YYY.4.105 2 u 344 1024 377 1.195 -1.585 0.528
xs1.p1.mydom.o .INIT. 16 u - 1024 0 0.000 0.000 0.000
xs2.p1 mydom.o AAA.BBB.12.15 3 u 34h 512 0 0.977 0.694 0.000
xs3.p1.mydom.o.INIT. 16 u - 1024 0 0.000 0.000 0.000

 

Since this was run from the XenServer xs3.p1.mydom.org, note how it treats itself as a peer (and that the FQDN is often truncated), what the strata are and the offset and jitter. The offset is the deviation from what ntpd calculates as the difference between what it determines the proper time to be and that of the server, while the jitter is the fluctuation seen when averaging multiple reads The asterisk ("*") in front of ntp3 indicates it's being used as the primary source while the plus sign ("+") in front of ntp1 and ntp3 indicate they are acceptable sources. You will sometimes see the choice of the primary changing sometimes to point to different NTP sources.

The peers show up as the last three lines, with the current host being ntp2 in this example. Note how it assigns itself a slightly higher stratum than the other two nodes in the pool "p1" but still lower than the organization's NTP servers.

Should connectivity to the NTP servers fail, the peers take over and re-negotiate a new synchronization setup among the named peers. Once NTP server access is restored, the main NTP servers take on that role once more and the resynchronization process recommences.

The big advantage here is that all pool members remain synchronized to the extent that XenServer's stability demands are satisfied and log times remain internally consistent. This model can be extended such that different pools can all participate, as long as network access is permitted, so that multiple pools can all be kept in sync with each other.

[Tobias Kreidl | 9,492]

Part 3

 

Left open to discussion in the previous installments was how to deal with problems and troubleshooting. One fairly common issue, especially with new servers, is if a host being joined to a pool doesn't have a reasonably close clock time to the others, things can get ugly fast. This can be triggered by a time difference as small as a few hundred milliseconds. One of the easiest ways to mitigate this is to set the clock close to that of the pool before the join operation is performed. To do this, you must first ensure that the NTP daemon itself is not running or there will be a conflict. Check with:

 

service ntpd status

 

which will show you if it is running or not and if so, just stop it with:

 

service ntpd stop

 

to temporarily shut it down. Once you have identified the one or more NTP servers being used by the rest of the pool, you can either synchronize the date to one or more of those, or even simpler, just synchronize to the pool master (really, any of the pool hosts should be close enough) with:

 

ntpdate server-name <optional-second-server-name>

 

or the IP address can be used, as well. There are a number of additional parameters available in the ntpdate command, which might be fun to to investigate via "man ntpdate".

 

It would also be advisable to set up the NTP configurations ahead of time before joining the server to the pool so the new server will integrate more smoothly. An additional comment here is to watch for servers which may have different time zones set before joining a pool; they should be checked and all be the same. The file /etc/localtime will be a pointer to a real file that defines the time zone being used to calculate the real local time offset from Coordinated Universal Time or Greenwich Mean Time (UTC/GMT). For example, if it points to /usr/share/zoneinfo/Iceland and you want your server to reflect the true local time, the server had better be somewhere in Iceland. Changing the timezone pointer is as easy (and dangerous) as issuing the command:

 

ln -sf /usr/share/zoneinfo/... /etc/localtime

 

so using the example above, the command would be:

 

ln -sf /usr/share/zoneinfo/Iceland /etc/localtime

 

with the path of course pointing to a valid specific file entry (and clearly, not a directory entry) under the /usr/share/timezone/ subdirectory. If you are really in Vienna, Austria or in the same time zone as Vienna, it should point instead to /usr/share/zoneinfo/Europe/Vienna as should all the servers at that one location that share the same common time zone. Clearly, the time will need to be uniform within a pool. A caveat would be if your organization has multiple pools spread out over different time zones, in which case you will want to make sure each pool or standalone server has its own internal uniformity as to time zones and choices of reasonable NTP servers with which to synchronize.

 

Note that another, and perhaps preferred, way of changing the time zone is to use the utility embedded within the meu options of the xsconsole routine.

 

Now, on to the independent wallclock thing...

 

Note that to run ntpdate successfully on most Linux boxes, you have to make sure the setting of the so-called independent wallclock parameter is temporarily toggled from its default value on XenServer of 0 (zero) indicating it is not being used as the primary source, if I understand it correctly. You can check it simply via the command:

 

cat /proc/sys/xen/independent_wallclock

 

and if it's a 0, that standard. If, however, an ntpdate command fails to properly update the clock, make sure again first that the NTP daemon (ntpd) is not running and then disable the independent wallclock parameter temporarily with:

 

/bin/echo 1 > /proc/sys/xen/independent_wallclock

 

after which you would set and verify the change made via ntpdate, and immediately set the wall clock parameter back using:

 

/bin/echo 0 > /proc/sys/xen/independent_wallclock

 

After this process, you can proceed with the standard setup and configuration for the parameters needed for ntpd to run properly.

 

Yet another item to heed is making sure your servers are not blocked because of a firewall or other network filter. By default, XenServers should be open, but check your network to make sure that TCP and especially UDP port 123 can be accessed  (note that the SSL version of NTP uses port 563). You can see what NTP processes are running on your server with:

 

netstat -a | egrep "ntp|Act|Proto"

 

which should display one or more connections (note the mandatory quotation marks and additional string match tokens in the command above to preserve the header output).

 

Finally, sometimes log files really can be your friends when you are digging for information that may show up rarely. The NTP daemon generally sends its events to the file /var/log/daemon.log and there should be entries showing up there periodically indicating successes. Error messages or the lack thereof can be very revealing about how well something is working or if not working as it should, help in identifying the nature of the errors.

 

In conclusion, this series of articles should help not only in setting up and maintaining NTP services on your hosts, but also provide a way for pools (or standalone servers, even) to maintain reasonably consistent time synchronization in the event of network or local NTP server outages.

[Tobias Kreidl | 9,492]

Reserved for Part 4 (should it ever come into existence...)

[Jesse Benedict | 44]

Excellent!  Shared on the Twitternets :)

 

--jkbs

@Xenfomation

[Alan Lantz | 3,533]

Tobias, whats your thoughts on whether to NTP sync the hosts and let the VM's hopefully get time through XenTools, or should one setup NTP on the VM's and let them take care of time directly ?

 

Alan ---

[Tobias Kreidl | 9,492]

@Alan: That is a great point that you bring up. Citrix does state that VMs are supposed to automatically time-sync to the XenServer host. I have not been able to find any details on exactly how this is done, how the synchronization is checked and how often, whether XenTools have to be enabled for this to work, etc.  Above all, what happens if you move your VM to another pool -- it seems like it's a question then on whether you trust the pool itself more or the VM's setup that is independent.

 

My personal preference is to set them up independently, above all, because you can then check the time offsets and see if they are really staying within one's expectations. If, however, the VM moves to another pool, it is true that you will likely then have to make some changes to the configuration, and if this is something one does quite often, it may be a better choice to let the XenServer take over that task.

 

@Jesse: Thanks for the kudos! I do hope others can profit from these collected experiences.

 

-=Tobias

[James Cannon | 1,675]

Great job Toby!

 

One thing most people make mistake, is to use the default time servers, which rely on port open and access to internet. XenServer should use same time server as the other systems in environment and have the /etc/sysconfig/ntpd file modified (from default) to update the hardware clock.

[Tobias Kreidl | 9,492]

@James: Yes, those are excellent points. In so many cases the "out of the box" configuration is not how you want to leave your setup!

-=Tobias

[swald77 | 0]

Excellent tips.

 

But my Xenservers have problems between hardware and system clocks :

Mon Nov  3 10:38:10 CET 2014
Mon 03 Nov 2014 10:38:04 AM CET  -0.626216 secondes
 

I must restart ntpd service regularly. Can youy help me please to sync clocks ?

 

It must be have a second between hardware and system clock... No ?

 

Thanks for advance for your feedback.

[Tobias Kreidl | 9,492]

Great question.
To synchronize the hardware clock to also use the Network Time Protocol daemon, you need to check the following file:

/etc/sysconfig/ntpd

 

If necessary, change the line:

 

SYNC_HWCLOCK=no

 

to be instead:

 

SYNC_HWCLOCK=yes

 

to have it synchronized to match the software-synchronized NTP time.

 

A typical file looks then like this:

# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"

# Set to 'yes' to sync hw clock after successful ntpdate
SYNC_HWCLOCK=yes

# Additional options for ntpdate
NTPDATE_OPTIONS=""

 

-=Tobias

[swald77 | 0]

Thanks for your response.

 

But my file /etc/sysconfig/ntpd is correct (exactly as the sample). But there are five or eight seconds between hardware and system clocks on my XenServers...

 

It's a configuration problem ?

 

Sébastien WALD.

[James Cannon | 1,675]

Sebastien,

 

You are using a local time server to sync your XenServers to? We only need local time server to go to internet. In most cases, your local domain server (used by Windows) should be a good candidate as NTP source for XenServer.

 

In the xsconsole, where you can add/remove NTP servers, should have the local time servers first. If it can reach an internet server (if listed before local time server) it will use that. So, order is important.

[Alan Lantz | 3,533]

I use Internet time servers (us.pool.ntp.org specifically) for my XenServers. I run my domain controllers as virtuals that provide internal NTP timekeeping and things seem to get really weird with time when the XenServer is pointing to a VM that its hosting for time. Just wanted to throw that out there.

 

Alan Lantz

SysAdmin

City of Rogers, Ar

[James Cannon | 1,675]

Hi Alan,

 

Good point!

[Tobias Kreidl | 9,492]

We have three internal NTP servers which in turn synchronize to outside, standard "national" NTP servers. That way, there are means to at least still syncronize to local NTP servers if the Internet goes down. If there are internal network issues, then the next fallback is the peer mechanism on a pool level, as discussed in the postings above.

 

To be honest, on most of our VMs, we specifically set up NTP so that it is run and controlled independently of what XenServer provides. That way, you have direct control over all aspects of which and how NTP servers are leveraged.

 

Thanks, James and Alan, for your inputs!

 

-=Tobias

[swald77 | 0]

Thanks James, Alan and Tobias for your inputs.

 

We uses two local NTP servers on Ubuntu Lucid 10.04 (without Internet servers). This configuration should be the same for each server. But only XenServer have this anomaly.

 

Best regards. Sébastien WALD.

[swald77 | 0]

The only solution is to restart ntpd regularly with cron.

[Jesse Benedict | 44]

Reserved for Part 4 (should it ever come into existence...)

Tobias!

 

On the clock, but saw your reply back to me regarding Creedence, etc.  This very discussion/topic has been bookmarked a long time ago by myself.  No secret I, like you, try to spend any free time in and outside of work to share in the knowledge - sleep or no sleep.  Most importantly, I am wrapping up what I hope to be a definitive, no questions, end all to the NTP discussion so I can turn it into code, etc.  Naturally, any tidbits you have posted here will be properly cited, but as for part #4... here goes.

 

------

 

As I have been writing my final revision to "NTP: Time Waits for Nothing" (originally titled "NTP (or How I came to love time sharing and its modernized title of time division with regards to the multi-multi-multitasking virtual world that relies more heavily on precision for ticks", there is a common trend I have come to find in 90% of our client, partner, and Citrix Community's XenServer deployments (especially where other Citrix or Infrastructure products are also deployed):

 

Using a Guest VM for an NTP source... I shiver!

 

This is not just a "XenServer" thing and sparing the technical details I am almost done with in the article I am writing, the short of it is this:

 

The intention for Guest VMs is to keep the their concept of time (for this applies to CPU ticks) as driftless as possible.  In simple terms, the hypervisor uses time and its own ticks to wind Guest VM clocks back and forth, accordingly, to the best of its abilities.

 

NTP is a process and is subservient to the OS it runs on.  If the Guest VM, running your infrastructure's NTP source, is making calls through a busy hypervisor... guess what?  It's perspective of time is skewed and as the NTP process works to schedule drift compensation, etc, that means it is returning an authoritative, skewed rendition of "time" to every client.

 

It is very hard, if not impossible to change "math" -- you have to adjust or conform to it.  So, without beating this topic to death, "What do you suggest, oh Jesse?"

 

Glad I asked for you!  :D

 

1.  Hypervisor must have NTP sources that are reliable as Tobias has covered.

2.  NTP conf does not have a limit on time sources, though I would say no more than 4-5 should be needed

3.  Stand up your own NTP server as Tobias outlined, but on a physical machine if you HAVE to, but for this server, go back to item #2

4.  "But my VMs are synced to time.somevendor.com"  Okay, that's great - really!  The problem of accuracy of your VMs clocks is minimized, but if I were you... follow Tobias' suggestion, again, and point it to your stand-alone NTP server... or better yet!  Point it to your actual XenServer's (so long as they are properly configured to update time and distribute time"

5.  A Sebastien pointed out - make sure you add a cron job to run an NTP update at least 2 times a day

 

That's my rant.  Never use a Guest VM for a time source.  It gets complicated quick and always start with the hypervisor's NTP accuracy first.  The rest trickles accordingly....

 

Tobias, I eagerly look forward to your eloquent synopsis that I equate to a "Harsh beating with a pair of cotton socks: not traumatizing, necessary, and you'll never forget your lesson!"

 

--jkbs | @xenfomation

[Tobias Kreidl | 9,492]

Thanks for your added comments, Jesse.Your feedback is always valued and of value.

Addressing your comments:

 

Yes, using a guest VM as an NPT server is like putting the fox in the henhouse to guard the chickens: destined to fail in a big way. In part, it's also because by default, the VM will get it's time updates directly from its XenServer (and many people, including myself, will deliberately configure NTP to not use a XenServer hosts as the primary time source to avoid this).  In addition, a hardware clock source in an independent server will not be adversely influenced by other extraneous things, not to mention be more robust as an entity by being completely separate from your XenServer hosts and/or pools. I don't have  a problem, however, with virtual NTP servers within an organization as long as they only synchronize to true standard external NTP primary sources.

 

I fully agree that around 3 - 5 sources is more than plenty. Most organizations out there seem to have three internal NTP servers defined. If you don't, pick something reliable.

 

If you are going to synchronize, do so consistently! It makes little sense to set up NTP services to point to that are different on various hosts and pools. If you want uniformity, you need to start out that way.

 

The use of peers is IMO one of the best ways to prevent inter-pool time drifts and since doing this, we are never off more than a few tens of milliseconds off at worst. For organizations with network issues that make access to their centralized NTP servers unreliable, this is a slick way of maintaining internal consistency and ensuring a "clean" way to get back on track when connectivity to your real NTP servers is reestablished. If you use peers, you may as well include all members of a pool. If on the same or easily accessible subnet, you could even use peers from multiple pools.

 

One more comment for now, namely if you are uncertain how reliable your NTP daemon is running, use Nagios or as suggested above, a simple cron job to either monitor the service or just periodically restart it to make sure it is active. A dead NTP daemon serves no purpose and can lead to trouble when least expected.

 

Again, thanks much Jesse for your continued contributions! And not a harsh comment at all out of me, to boot...

 

-=Tobias

[Jesse Benedict | 44]

The cronjob is the magic solution!

 

@Alan: That is a great point that you bring up. Citrix does state that VMs are supposed to automatically time-sync to the XenServer host. I have not been able to find any details on exactly how this is done, how the synchronization is checked and how often, whether XenTools have to be enabled for this to work, etc.  Above all, what happens if you move your VM to another pool -- it seems like it's a question then on whether you trust the pool itself more or the VM's setup that is independent.

 

My personal preference is to set them up independently, above all, because you can then check the time offsets and see if they are really staying within one's expectations. If, however, the VM moves to another pool, it is true that you will likely then have to make some changes to the configuration, and if this is something one does quite often, it may be a better choice to let the XenServer take over that task.

 

@Jesse: Thanks for the kudos! I do hope others can profit from these collected experiences.

 

-=Tobias

 

Still stirring the pot as I am trying to wrap up a few articles (citations given, naturally).

 

You gents (Tobias and Allen) have brought up a remarkable question regarding Guest VMs and time accounting.  Outside of the office or even in the office, once in a very strange while someone has ONE VM that seems to have a flux capacitor... swaying this way and that.

 

I have seen this time dissonance with HVMs... as in Windows.  Time issues in PV/HVMs running Linux?  Nope -- ntp is standard, for the most part along with other advantages.

 

So what is going on here?

 

Any Hypervisor is supposed to feed time to Guest VMs - not just XenServer.  In the aspect of Windows - with or without tools - if the bare metal is under load (thus consuming its own time), then how great the delay for the Guest VM to become informed? 

 

Example:

 

1 Hypervisor with its CMOS/BIOS tuned, as humanly possible, to "real-time".

10 Windows Guest VMs with Tools Installed

 

In the Windows Guest VM metadata is a place holder for clock sway/updates, but here is the kicker: the Hypervisor has to handle such an update in round robin fashion.  So, dom1, dom2, and dom3, and dom4 get a clock update from the Hypervisor.  THEN IT HAPPENS...

 

dom6 is a VIF sucking, tapdisk/blockdisk smashing machine running, well, something substantial.  It makes a VM EXIT call to the Hypervisor and now, that poor Hypervisor is sending "blocking" responses to dom7, dom8, and dom9.  While it handles dom6's call, it sends a VM ENTRY call back to dom6 and also updates its clock.  However, it isn't accurate.  The bare metal was so busy multitasking that NTPD fell to the bottom of the process list.

 

Now, as things slowly even out for the Hypervisor, dom8 gets a slightly distant time and dom9 is now drifting past Pluto (in terms of time).  The user either has a w32m command to update time, but even still - the Hypervisor hasn't come back to NTPD yet.

 

Furthermore, if we recall, in the 5.x days - it didn't matter if you adjusted the Hypervisor's time... the Guest VM time had to be updated.  Ergo, viridian and all that jazz. 

 

In summary of this rant, I say to everyone:

 

There is a place in metadata/templates to hold time and I know with 6.2 that it is recommended to have the control domain and Guest VMs on NTP:

 

http://docs.vmd.citrix.com/XenServer/6.2.0/1.0/en_gb/guest.html#windows_time

 

and in 6.5, it gets even COOLER!

 

http://docs.vmd.citrix.com/XenServer/6.5.0/1.0/en_gb/guest.html#windows_time

 

 

Lastly... to everyone, thanks and I recommend Independent Wallclocks because, well, if you reference the Hypervisor's clock (as described)... you are kinda stuck with the results...

 

 

--jkbs | @xenfomation

[Tobias Kreidl | 9,492]

Jesse,

Thanks very much for these additional notes. Of particular interest was the change in philosophy over time as to whether or not to synchronize to the wall clock or not, as well as enhanced options under XenServer 6.5.

 

Indeed, in XenServer 6.5 the default in /etc/sysconfig/ntpd is "SYNC_HWCLOCK=no". Note also that as part of the general startup process of your XenServer that a call to ntpdate is part of the init process, which is good to get you to the approximately right time, otherwise ntpd will be very unhappy if the time isn't reasonable close to start with.

 

This goes back to what was stated earlier, namely that I really have always preferred running NTP independently on all VMs, regardless of OS, as a means of have tighter control over all aspects of the time synchronization.

 

Indeed, Linux gets the bonus options in XS 6.5 as far as flexibility goes:

 

To set individual Linux VMs to maintain independent times

1. From a root prompt on the VM, run the command: echo 1 > /proc/sys/xen/independent_wallclock

2. This can be persisted across reboots by changing the /etc/sysctl.conf configuration file and adding:

   # Set independent wall clock time
   xen.independent_wallclock=1

3. As a third alternative, independent_wallclock=1 may also be passed as a boot parameter to the VM.

 

Having that much more control could be really important, especially considering what happens once you start migrating VMs not only to different servers within a pool but to other pools altogether. There are clearly other good reasons where you may want to keep your different VMs running on different time zones and having fine control on the VM level.

 

As to hardware clock issues, I have seen times where clocks got way off because of the machine being so overwhelmed that even the hardware clock, which normally runs at one of the highest interrupt levels, was losing ticks! And, ironically, this is why I trust the clock on a GPS-fed smartphone a lot more than even a Rolex. ;)

 

-=Tobias