How To Make Your Xen-PAE Kernel Work With More Than 4GB RAM (Debian Etch With GRUB)

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Fri, 2007-12-14 13:30. :: Debian | Xen | Virtualization

How To Make Your Xen-PAE Kernel Work With More Than 4GB RAM (Debian Etch With GRUB)

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 12/13/2007

If you have a server with more than 4GB RAM and want to install a 32bit Debian Etch on it (following this tutorial: Debian Etch And Xen From The Debian Repository), you'd expect the Xen-PAE kernel to see all your RAM because the Xen-PAE kernel supports up to 64GB RAM. In fact, it recognizes only about 3.3GB RAM due to a bug in the GRUB bootloader. This article explains how you can fix GRUB so that all your RAM gets recognized.

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

This bug exists on 64bit Debian systems as well - the default Debian Xen kernel also recognizes only 3.3GB of RAM. It's possible that the following procedure works for 64bit Debian Etch systems as well, although I have tested it only on a 32bit Debian Etch system.

I assume that you have already set up Xen on your system according to this tutorial: Debian Etch And Xen From The Debian Repository

I have tested this on a system with 6GB RAM.

Don't follow this tutorial if you're using another bootloader than GRUB (e.g. lilo)!

 

2 Check Your Memory

You can run

cat /proc/meminfo

to see how much memory your system recognizes:

Debian-40-etch-32-minimal:~# cat /proc/meminfo
MemTotal:      3468288 kB
MemFree:       3237948 kB
Buffers:         55884 kB
Cached:          46124 kB
SwapCached:          0 kB
Active:          68820 kB
Inactive:        41412 kB
HighTotal:     2731012 kB
HighFree:      2667548 kB
LowTotal:       737276 kB
LowFree:        570400 kB
SwapTotal:     2104440 kB
SwapFree:      2104440 kB
Dirty:            3524 kB
Writeback:           0 kB
AnonPages:        8196 kB
Mapped:           3932 kB
Slab:            10140 kB
PageTables:        304 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3838584 kB
Committed_AS:    50368 kB
VmallocTotal:   116728 kB
VmallocUsed:      3776 kB
VmallocChunk:   112612 kB
Debian-40-etch-32-minimal:~#

As you see, only about 3.3GB are recognized by the system...

 

3 Rebuild GRUB

Now let's rebuild the GRUB bootloader. We need the package dpkg-dev for it:

apt-get install dpkg-dev

Then we download the GRUB sources...

apt-get source grub

... and install all requirements to rebuild GRUB:

apt-get build-dep grub

We must modify the stage2/common.c file in the GRUB source directory. First we make a copy of that file:

cd grub-0.97/
cp -a stage2/common.c stage2/common.c.original

Then we open it:

vi stage2/common.c

Around line 143, you should find this section:

[...]
#ifndef STAGE1_5
  unsigned long cont, memtmp, addr;
  int drive;
#endif
[...]

Modify it as follows:

[...]
#ifndef STAGE1_5
  unsigned long memtmp, addr;
  volatile unsigned long cont;
  int drive;
#endif
[...]

Save the file, then build the new GRUB package:

debian/rules binary

Afterwards we can install the new GRUB package as follows:

dpkg -i ../grub_0.97-27_i386.deb

Now we must install GRUB on our hard drives. I'm using two hard drives, /dev/sda and /dev/sdb (for software RAID1), so I run

grub-install /dev/sda
grub-install /dev/sdb

If you only got one hard drive, then run grub-install only once. Make sure that you use the correct device name for your hard drive (for example, instead of /dev/sda, yours might be /dev/hda - you can find out about the hard drive names by running

fdisk -l

).

Finally, it's time to reboot:

reboot

 

4 Check Your Memory Again

After the reboot, it's time to check the memory again. If everything went well, the system should now recognize the full amount of RAM:

cat /proc/meminfo

Debian-40-etch-32-minimal:~# cat /proc/meminfo
MemTotal:      6067200 kB
MemFree:       5902700 kB
Buffers:          1596 kB
Cached:          13964 kB
SwapCached:          0 kB
Active:          19316 kB
Inactive:         4408 kB
HighTotal:     5329732 kB
HighFree:      5299600 kB
LowTotal:       737468 kB
LowFree:        603100 kB
SwapTotal:     2104440 kB
SwapFree:      2104440 kB
Dirty:             784 kB
Writeback:           0 kB
AnonPages:        8160 kB
Mapped:           3932 kB
Slab:             8616 kB
PageTables:        324 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   5138040 kB
Committed_AS:    50388 kB
VmallocTotal:   116728 kB
VmallocUsed:      3776 kB
VmallocChunk:   112612 kB
Debian-40-etch-32-minimal:~#

As you see, my system is now recognizing my full 6GB of RAM.

 

5 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by burke3gd (registered user) on Wed, 2008-03-12 07:53.

Thanks for the help. I had the same problem on my amd64 system and this fixed it for me. I still get different RAM totals if I use a non-xen amd64 kernel, but now the difference is only a couple of megabytes.