Asus P9D-I (LSI MegaRAID) and XenServer 7.2

[Yvo Koot]

Hi i like to run a small XenServer ( 3 VMs max ) with my ASUS P9D-I mainboard.

 

Created a RAID10 array using onboard LSI MegaRAID BIOS.

When booting the array is online, but since this is software RAID the array configured seems to be useless to Linux initially.

 

However it appears older drivers are available at ASUS ( https://www.asus.com/nl/Commercial-Servers-Workstations/P9DI/HelpDesk_Download/ ), though I am not sure whether I need the C22x LSI or LSI 2308 RAID drivers. As the Intel C222 controller supports the Broadcom (LSI) 2308.

​Anyone familiar with similar set up to get things going?

 

My idea after reviewing few sources would be making storcli available to dom0 ( once XenServer is installed ).

Thus after installing Xen following (similar) needs to be executed

 

cd /tmp
#wget https://docs.broadcom.com/docs/STORCLI_SAS3.5_P3.zip
unzip STORCLI_SAS3.5_P3.zip
cd STORCLI_SAS3.5_P3//unified_storcli_all_os/Linux
rpm -ivh storcli-007.0204.0000.0000-1.noarch.rpm

 

#rest of commands here

 

Oh please note XenServer will boot from an separate disk,

as the RAID10 array will be allocated to Virtual Machines only.

 

​Thanks

[Tobias Kreidl]

Yvo,

Software RAID is not supported in XenServer, so whether or not it works will be hit or miss.

-=Tobias

[Yvo Koot]

Hi - thanks for your feedback .

 

Reason I posted my question is it seems for this C22x controller there are older drivers available for Xen6 at Asus website ( LSI 2308 though - which confuses me): "00:1f.2 RAID bus controller: Intel Corporation 8 Series/C220 Series Chipset Family SATA Controller 1 [RAID mode] (rev 05)"

 

Also C22x LSI RAID drivers exist both at ASUS and newer ( though no longer for C22x as it seems ) at Intel. The idea is the same to blacklist the ahci driver ( Linux dd blacklist=ahci nodmraid ) and pick up the settings configured in the LSI Software RAID config utility.

 

​So I was kind of hoping this way the array could be made available to Xen, but you are right this is not a hardware RAID - however bear in mind this is far from an enterprise server as this is home usage for max 3VMs

 

thanks

[Tobias Kreidl]

Depending on how old the drivers are are, they may only work for the pre-64-bit version of XenServer. Worst case, I'd just disable the SW RAID and use it "as is" w/o any RAID options, at least. Sorry, like I said, this is a big unknown. Depending on your budget, you may perhaps be able to find a suitable compatible HW RAID controller.

 

-=Tobias

[Yvo Koot]

so took it from a little different angle and installed CentOS7 and upgraded to 7.3...

Also used Intel's file: MR_SWR_Driver_1.50-17.01.2016.1107.zip - which installs a new module to replace more or less (blacklist) ahci.


The module loaded for the RAID controller shows now:
00:1f.2 RAID bus controller: Intel Corporation 8 Series/C220 Series Chipset Family SATA Controller 1 [RAID mode] (rev 05)
Subsystem: ASUSTeK Computer Inc. Device 8552
Kernel driver in use: megasr
Kernel modules: ahci, megasr


Also output is getting satisfactory considering /dev/sdd:


[root@localhost test]# fdisk -l

Disk /dev/sda: 320.1 GB, 320072933376 bytes, 625142448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
Disk label type: dos
Disk identifier: 0x0008cd51

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 562642943 281320448 83 Linux
/dev/sda2 562642944 625141759 31249408 82 Linux swap / Solaris

Disk /dev/sdd: 998.0 GB, 997998985216 bytes, 1949216768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


So next step is most likely to compile this module for kernel 4.4 using Xen's DEV image or I could choose to install XEN packages on CentOS7 ... I will see how to proceed ....

[Tobias Kreidl]

Looks like good progress! Soldier on...  :) Note that the Xen distribution is not the same as XenServer and while much of the functionality will be there, not all the features and control will be, including XenCenter. If you're comfortable with thw CLI, you should be fine. If modifying the XenServer installation itself, any added on packages would likely void support, but this is a home lab and not a production environment.

 

-=Tobias

[Yvo Koot]

thanks for your response.

 

Therefore meanwhile I set up a DDK ( XenServer-7.2.0-DDK ) so I should be able to build a kernel module, but I failed ...

This was the plan

 

 

1/ Creating megasr.ko module for Citrix kernel and rhel7

 

On DDK:

---

put latest intel package (  ESRT2_Linux_DRV_v17.01.2016.0216_r3.tar.gz ) in /tmp
 

cd /tmp
tar -xvzf ESRT2_Linux_DRV_v17.01.2016.0216_r3.tar.gz

 

make -C /usr/src/kernels/4.4.0+10-x86_64 M=/tmp/megasr-raid5/shim/library/rhel7-64

 

 

 

2/ Install XenServer

 

 

Set BIOS to boot from RAID

 

boot: menu.c32
select Install and <tab> 
add blacklist=ahci # just before --- /boot/vmlinuz

 

 

- till <F9> disk drivers

[Alt] [F2]

 

mkdir /mnt/usb

mount -t vfat /dev/sdf1 /mnt/usb

 

cd /lib/modules/4.4.0+10/kernel/drivers/scsci
tar -xvzf /mnt/usb/megasr/megasr.tar.gz

 

insmod megasr.ko

 

 

BUT ALAS: "ERROR : could not inser module mesasr.ko: Unknown symbol in module"

 

 

When compiling on DDK following can be seen as attached in logs ( both warnings and compile process ), except for warnings modules got built. Therefore can one confirm I am heading towards the proper direction? Also please note currently I try to install to my RAID10 array and use it to boot XEN as well storage later for my VMs to be build.

 

 

meanwhile help is appreciated in an attempt to get this going,

thanks!

 

 

xen.zip

[Yvo Koot]

hmm did one more attempt but failed as well:

 

 

 

mkdir /usr/src/megasr-17.01.2016.0216
cd /usr/src/megasr-17.01.2016.0216
tar -xvzf /tmp/ESRT2_Linux_DRV_v17.01.2016.0216_r3.tar.gz

 

[root@localhost RPM]# systemctl status dkms
● dkms.service - Builds and install new kernel modules through DKMS
   Loaded: loaded (/usr/lib/systemd/system/dkms.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sun 2017-07-30 08:59:50 UTC; 3s ago
     Docs: man:dkms(8)
  Process: 13415 ExecStart=/bin/sh -c dkms autoinstall --verbose --kernelver $(uname -r) (code=exited, status=0/SUCCESS)
 Main PID: 13415 (code=exited, status=0/SUCCESS)

Jul 30 08:59:50 localhost systemd[1]: Starting Builds and install new kernel modules through DKMS...
Jul 30 08:59:50 localhost systemd[1]: Started Builds and install new kernel modules through DKMS.

 

[root@localhost RPM]# cd /usr/src/megasr-17.01.2016.0216/megasr-raid5/RPM
[root@localhost RPM]# rpm -ivh megasr-17.01.2016.0216-1.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:megasr-17.01.2016.0216-1         ################################# [100%]
Error! Could not locate dkms.conf file.
File: /usr/src/megasr-17.01.2016.0216/dkms.conf does not exist.
warning: %post(megasr-17.01.2016.0216-1.noarch) scriptlet failed, exit status 4

 

Thus created /usr/src/megasr-17.01.2016.0216/dkms.conf:

 

MAKE="make -C /usr/src/kernels/4.4.0+10-x86_64 M=/usr/src/megasr-17.01.2016.0216/megasr-raid5/shim/library/rhel7-64"
PACKAGE_NAME="megasr"
PACKAGE_VERSION="17.01.2016.0216"
BUILT_MODULE_NAME[0]="megasr"
DEST_MODULE_LOCATION[0]="/kernel/drivers/scsi/"
AUTOINSTALL="yes"

 

 

But also failed to build:

 

[root@localhost RPM]# rpm -ivh megasr-17.01.2016.0216-1.noarch.rpm --force

Preparing...                          ################################# [100%]
Updating / installing...
   1:megasr-17.01.2016.0216-1         ################################# [100%]

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...(bad exit status: 2)
make -j8 KERNELRELEASE=4.4.0+10 -C /usr/src/kernels/4.4.0+10-x86_64 M=/usr/src/megasr-17.01.2016.0216/megasr-raid5/shim/library/rhel7-64...
Error!  Build of megasr.ko failed for: 4.4.0+10 (x86_64)
Consult the make.log in the build directory
/var/lib/dkms/megasr/17.01.2016.0216/build/ for more information.
warning: %post(megasr-17.01.2016.0216-1.noarch) scriptlet failed, exit status 7

 

[root@localhost RPM]# more /var/lib/dkms/megasr/17.01.2016.0216/build/make.log
DKMS make.log for megasr-17.01.2016.0216 for kernel 4.4.0+10 (x86_64)
Sun Jul 30 09:35:51 UTC 2017
make: Entering directory `/usr/src/kernels/4.4.0+10-x86_64'
  Building modules, stage 2.
  MODPOST 1 modules
WARNING: "__stack_chk_fail" [/usr/src/megasr-17.01.2016.0216/megasr-raid5/shim/library/rhel7-64/megasr.ko] undefined!
make: Leaving directory `/usr/src/kernels/4.4.0+10-x86_64'

 

 

Also came to my attention dkms installs kernel-dev version 3.10.0-514.26.2.el7.centos.plus.x86_64 while 4.4.0 is being used ...

[Yvo Koot]

So now I said goodbye to 7.x version and reverted to 6.5 with little more satisfatory result, but still failed to install with error

" FAILED TO GENERATE FALLBACK INITRD ... "

 

I need to dive in to that ( dmesg ) why this shows up on the very end but driver is compiled OK and it seems to be " working " as the array gets identified now ( /dev/sdd ) ...

 

Maybe somehow I can create a separate boot image in ddk so I can load these drivers with <F9> instead of following procedure ..

 

 

In a nutshell:

 

boot: menu.c32
select Install and <tab> # immediately!
add blacklist=ahci ( just after --- /boot/vmlinuz ) # small glitch in previous post

 

- till <F9> disk drivers

[Alt] [F2]

mkdir -p /mnt/usb

fdisk -l #|grep W95  should show only your USB disk as ahci is blacklisted, thus  /dev/sda1

 

mount -t vfat /dev/sda1 /mnt/usb

cd /lib/modules/3.10.0+2/kernel/drivers/scsci
tar -xvzf /mnt/usb/megasr/megasr.tar.gz

 

umount /mnt/usb
rm -rf /mnt/usb

 

insmod /lib/modules/3.10.0+2/kernel/drivers/scsi/megasr.ko

 

 

now RAID10 gets identified and can be installed on to

 

Hope one can give me a hand with this

cheers

[Yvo Koot]

​OK - stayed with Xen6.5 and dug in to things a little further with satisfactory result.

I have both a RAID10 ( megasr.ko ) and single disk ( to boot XenServer and for DDK ), did not succeed ( yet ) with only RAID10

 

Steps taken have been attached for the enthousiasts that are interested, would be awesome to have something similar for 7.x ( 8?) coming year as it will be EOL in Q3 next year ...

 

Actually it is interesting if this could be a "supported" installation as official drivers from Intel have been used to make it work ...

XenCenter65_AsusP9D-I.zip

[Roman Varlamov]

Hello, sorry for necroposting. XenServer 7.6, intel Server Board S2600GL, raid controller in ESRT2 mode.

For compilling without this warning

Quote

WARNING: "__stack_chk_fail" [/usr/src/megasr-17.01.2016.0216/megasr-raid5/shim/library/rhel7-64/megasr.ko] undefined!

I added an empty function to the end of the file

Quote

../shim/open_source/osl/lin/linux_osl.c

//.........

int megasr_printk(const char *fmt, ...)
{
	va_list		ap;
	char		buf[768];
        char *fmt1 = (char *)fmt;
	va_start(ap, fmt);
	//vsprintf(buf, fmt, ap);
        vsnprintf(buf, sizeof(buf), fmt1, ap);
	va_end(ap);
	//return printk(buf);
         //return printk(KERN_ERR":%s\n",buf);
         return printk(KERN_ERR":%s",buf);
}


void __stack_chk_fail(void){};			//i added this string

module_init(lsraid_init);
module_exit(lsraid_exit);


//.........

after this insmod works without errors