Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > Linux Forums > Installation/Configuration

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Thread Tools Display Modes
Old 29th July 2013, 13:42
IanH IanH is offline
Junior Member
Join Date: Jul 2013
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
Default What shoud be the correct settings for Xen Dom0 based on my resources ?

Hi All,
I'd like some advice with selecting the best settings for Xen.

Currently I have 2 servers, each with 8 CPU Xeon cores, 12Gb memory and 1TB of Raid 5 storage. I had installed CentOS6 on both of these servers, and they were only used for a very trivial task. Having closely monitored the CPUs, I find that CPU0 is used on average around 40~60% and CPUs1~7 are barely used at all at less than 5%.
I decided its time to make better use of my resourses, and I installed Xen.

I followed the instructions from http://www.howtoforge.com/virtualiza...virtualization
And it seems that I have a successful installation.
My boot/grub parameters are set up with
kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin
I installed the first PV DomU (snl3) it seemed fine. I gave it 3 CPUs and 4Gb of memory, and CentOS6.4
I installed the 2nd PVDomU (snl4) it also seemed fine, I gave that 2CPUs and 3Gb of memory, CentOS6.4
In both cases I created and started these DomUs with xl create (not using xm or virsh)
I then installed the applications on each DomU, and this is where the problems began.

Originally I ran a script on the dedicated server, it took just over 4 seconds. On the DomU snl3, its now taking over a minute to run, which is not good, because the script is executed by cron every minute.
The script that I ran on DomU snl4, well, it just didn't finish, where as previously on dedicated server it took just over 2 mins.
Upon checking xl vcpu-list, I see that Dom0 is using VCPU0=CPU0, and DomU(snl3) is using VCPU0=CPU0 , VCPU1=CPU2, VCPU2=6 and DomU(snl4) VCPU0=0 and VCPU1=4, there appeared to be no consistency, IE not choosing the next available unused CPU, however, refreshing xl cpu-list then showed it all changed again.
I checked the CPU utilization of Dom0 and the DomUs, and in all cases, CPU0=100% utilized. (I was also very surprised that Dom0 had only 1 CPU, as I expected to see 8 CPUS from the SNMP stats, but I think I misunderstand how the hardware interacts with CentOS and Xen and the Dom0)

I decided to make a couple of new CPU pools, "Pool-snl3", and "Pool-snl4". I removed CPUS 3~7 from Pool-0 and added 3,4,5 to Pool-snl3 and 6,7 to Pool-snl4.
I assigned DomU (snl3) to pool-snl3 and then pinned the CPUs to 3,4,5
I assigned DomU (snl4) to pool-snl4 and then pinned the CPUs to 6,7
I ran the script on snl3 again, and this time it completed in under a minute, but only just, but still there were many times when it took longer than a minute.
I checked the CPU utilization of the Dom0 and DomUs again and still they were at 100%.
I tried to use taskset to get the scripts to run on a specific VCPU, but the results I got led me to believe that taskset was trying to run the task on the actual CPU specified rather than the VCPU.

At this point, I decided to abandon what I had done and restart from new. Reading further made me believe that Dom0 controls how the DomUs are given access to the hardware CPU and memory resources, etc.
I re-installed CentOS6.4 on my server, and re-installed Xen again.

I changed the boot/grub
kernel /xen.gz dom0_mem=12G cpufreq=xen dom0_max_vcpus=8 dom0_vcpus_pin
It failed to start
I changed it to
kernel /xen.gz dom0_mem=11G cpufreq=xen dom0_max_vcpus=8 dom0_vcpus_pin
it started, but then I could not create and DomUs
I changed it to
kernel /xen.gz dom0_mem=8G cpufreq=xen dom0_max_vcpus=6 dom0_vcpus_pin

I have now created DomU(snl3), installed CentOS6.4, with 2 VCPUs (from Pool-0) and 2G memory, and the script how takes 10 seconds to execute. But, SNMP shows that VCPU0 is averaging at 97% while VCPU1 is at 9%.
I've also checked Dom0 with SNMP and all CPU0 is under 4%, while CPUs1~5 are all under 1%
xl vcpu-list shows that Dom0 VCPU0=0, V1=1, V2=3, V3=3, V4=4, V5=5, and DomU(snl3) is VCPU0=7, V1=5 (but again, refreshing xl vcpu-list shows that this changes, except for Dom0 that doesnt change)

So now my head is quite messed up. I cant see the relationship between how the DomU is allocated a VCPU, sometimes it comes from a free CPU not allocated to the Dom0, and sometimes its allocated a CPU that is pinned to the Dom0.

The bottom line is this, I have 8 CPU Cores, and 12G memory.
Should I be allocating all my CPUs and memory to Dom0, thus to allow Dom0 to allocate its known resources to my DomUs ? Or should I stick with letting the Dom0 have 1G memory and 1 CPU, and allocate resources from what is left over to each of my DomUs ?
Also, If I wish to pin a CPU to a DomU and give it exclusive, unscheduled and unshared access, how can I do this ?

Please can you advise me, as I'm starting to lose sleep over this now.

Reply With Quote
Sponsored Links


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ISPConfig fresh instalation - email tab is empty artur_gib Installation/Configuration 14 28th January 2014 20:02

All times are GMT +2. The time now is 19:22.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.