Jump to content


Photo

LUN allocation best practice

Started by Stephen Eaton , 17 May 2010 - 03:16 AM
7 replies to this topic

Stephen Eaton Members

Stephen Eaton
  • 119 posts

Posted 17 May 2010 - 03:16 AM

G'day one and all,

What is best practices for a SAN LUN allocation on Xenserver. Is there a VM to LUN ration we should be sticking too?

Regards,

Stephen...



Tobias Kreidl Members

Tobias Kreidl
  • 13,259 posts

Posted 17 May 2010 - 04:04 AM

Hi, Stephen:

A lot will depend of course on the nature and volume of I/O going on among the various VMs and how they contend with each other. The characteristics and capabilities, as well as the configuration of the SAN (type of RAID, disk type (e.g., SATA vs. SAS), number of spindles, cache, available bandwidth, disk RPMs, etc.) will all weight in.

The main thing, IMO, is to monitor performance with utilities such as top, iostat, vmstat, mpstat, etc. If things are noticeably sluggish, something's to blame, which could be one particular server dragging the rest down or simply all the VMs vying for disk I/O. The number of allocated CPUs and memory, as well as certain kernel tweaks can all weigh in, as well.

I have one real I/O hog on a VM, for example, that interferes surprisingly little with other VMs sharing the same pooled SR.

On the other hand, there are many things one can do to help. One is to isolate disk partitions with known high I/O rates (/var/log comes to mind, as well sometimes as swap space) to separate SRs than, say, the partitions for the OS or databases.

We've spent literally years in some cases adjusting things over time to attempt to optimize the current environment and -- the next thing you know -- you change your equipment and have to start the process all over again to some degree.

Is there a particular scenario that you've encountered or have concerns about?

That all having been said, we have as many as a dozen or so VMs all running off a single SAN (albeit with multiple virtual volumes), but as stated above, everyone's situation will be unique.

Cheers,
--Tobias



Stephen Eaton Members

Stephen Eaton
  • 119 posts

Posted 17 May 2010 - 06:00 AM

Thanks for the reply Tobias,

We've recently installed a couple of additional disk shelves on our MD3000i as we've started to suffer from "VM creep".

I was wanting to re-organise our existing Volumes and LUNS on the SAN, initially when things started out I had a single Volume and single LUN for an exchange SR, a single Volume and LUN for our main file server SR then a more generalise Volume and LUNs for everything else, however this is a bit inefficient in terms of our physical disk usage, and as our VM contingent has grown they have proliferated into all available space, including the Exchange and file server SR's, so I'm looking at moving things around to be more efficient.

After a bit of research online I see that VM:LUN ratios are often stated by vendors, and was wondering what others are using, or how they divi up the SAN for their VM and SR allocations.

Stephen...



Tobias Kreidl Members

Tobias Kreidl
  • 13,259 posts

Posted 17 May 2010 - 06:24 AM

We have one MD300i configured with 12 1-TB SATA disks in a RAID6 configuration, yielding around 8 TB of useful storage. They are all ganged together, then chopped into a couple of separate chunks, from about 1 TB up.
SATA isn't great for lots of I/O packets, so this is mainly being used as a storage repository.

On eof our SANs with a dozen or so VMs is an old Sun 6130, with 10k RPM 300 GB SAS drives and does a good job at this. RAID5 arrays tyoically consist of anywhere form 5 to 14 drives.

Do you have SATA or SAS drives in your MD3000i?

While a vendor can make a rough recommendation, the individual's situation will always be somewhat unique. They can give you guidelines, but a generic setup is stll just a starting point. The Linux kernel has a slew of tunable parameters and there;s a lot that can be gained in some cases just by altering a couple of parameters.



Stephen Eaton Members

Stephen Eaton
  • 119 posts

Posted 17 May 2010 - 06:48 AM

We have a total of 3 trays, two ~5TB of SAS (~10TB total) and a single 10TB SATA.

Our head unit is choc-a-block full so I was thinking of making a single volume (Raid 5) on the second SAS tray and split this up into LUNS for different SRs and migrate across from my first tray then clean up my 1st tray and do something similar. As the 2nd (newer) SAS tray has larger drives installed so I was going to just keep the two SAS trays as separate volumes and spread the LUNS between the two volumes.

I'm only going to be using using the SATA for backups and template images, currently I have this as a single Raid 6 volume and no LUNS yet.

Regards,

Stephen....



Tobias Kreidl Members

Tobias Kreidl
  • 13,259 posts

Posted 17 May 2010 - 07:08 AM

That sounds reasonable, unless you have a particular case with really heavy I/O. In that case, I'd split the one brick into two separate partitions and allocate separate physical drives to the unusual case. The bigger drives will be somewhat slower, but the MD3000i has a lot of cache, so unless things get really busy, that shouldn't be so much of an issue.

We're soon going to add a second, separate SR from a different LUN probably instead of expanding the existing one (configured with about 600 GB). Heavy-duty disk needs (such as for Samba mounts) are dealt with on separate SAN LUNs so they don't interfere with the basic OS I/O.

Running RAID6 on a SATA system makes sense -- I figured it could take 24 hours for a 1-TB drive to be reconstructed!

Sounds to me like you have a good plan, Stephen.



Stephen Eaton Members

Stephen Eaton
  • 119 posts

Posted 17 May 2010 - 07:44 AM

Thanks for that, it's nice to confirm that I'm heading on the right track.

I was think of doing something similar with putting my OS on separate Volume/LUNS to my Files Server Shares, and Exchange Data stores

Regards,

Stephen...



Stephen Eaton Members

Stephen Eaton
  • 119 posts

Posted 17 May 2010 - 07:51 AM

I forgot to ask in the previous reply, you mentioned splitting one of the trays up. One of the SAS trays? or the SATA?

Thanks again,

Stephen...