Building a XenServer Stack

xenstack1I have been building a lot of Virtual Machines for a variety of reasons over the last 12 months, and up until now have been using external vendors, such as Digital Ocean and Host Virtual to provide the virtualisation and hosting services (both of these companies have been excellent).

I wanted to try bringing some of these services in-house and thought it would be a good opportunity to experiment with the available technology and learn something new.

The original plan was to build a platform similar to that of the above on-line providers. A web-based portal where I could log in, view and manage my existing Virtual Machines, and create new ones (from templates or ISOs) as required.

I looked in to VMWare, VirtualBox and XenServer for the Virtualisation technology. OpenFiler and FreeNAS for shared storage.

I have in no way completed this project, but thought I would start to document things in case I forget something or any of this might be useful to others. I am also by no means an expert in this area, and welcome any comments about how I should improve things.

Current Hardware:

  • Existing Ubuntu 12.04 server providing DHCP & DNS (but this could come from your router etc)
  • Mac Mini Workstation for management and installation tasks (including Windows 7 VM for XenCenter).
  • 3 x HP Microserver G1610T Gen8 (Celeron, 16GB, 64GB SSD)
  • 1 x HP Microserver Gen7 (Celeron, 4GB, 4 x 2TB)
  • 1 x HP PS1810-8G Switch (no PSU!)
  • 1 x TP-LINK TL-POE150S Power over Ethernet injector
  • External USB DVD drive
  • 32 GB USB Key

Current Software:

  • XenServer 6.2
  • FreeNAS
  • Xen Orchestra (software for web-based administration)

The HP Gen 8’s will be the XenServer hosts. The Gen 7 will be a shared storage device. The HP switch connects everything up via GbE. The USB key will be used to host FreeNAS as I want to use all the internal drives for shared storage.

Issues Encountered:

I am currently running everything (management and data) on a single subnet. This is mainly because I wanted to get everything working as quickly as possible. The next stage of this project is to separate things, since the Gen8’s have two NICs where as the Gen7 only has one.

Annoyingly, the HP switch did not come with a power supply, and even though the instructions state that it does (even in the ‘please check contents’ section) – contacting eBuyer and even HP resulted in them politely telling me that I needed to get power to the unit in another way. I ended up going for the POE injector since HP could not tell me which PSU to buy!

I also purchased the incorrect memory for the Gen8’s initially – since the machines seem to be very strict about what they will accept. I ended up going for the “CT4487631 16GB kit (8GBx2), 240-pin DIMM Upgrade for a HP Compaq Proliant MicroServer Gen8 System” from Crucial, at £127 + VAT.

I also wanted to run XenServer from the Gen8’s internal Micro SD card slot – but no matter what I tried, I could not get it to install successfully. After many, many lost hours I ended up getting some cheap SanDisk SSD drives from Novatech that made the process a whole lot easier. The added bonus here is that I can use the SSDs for some fast local VM storage if I need it.

Installation Process:

XenServer Hosts

These will be where the VMs do all their computing. XenServer is a breeze to install providing your hardware is all set up correctly. I decided on XenServer rather than Xen (hosted on Ubuntu etc) since I do not need to use these boxes for anything other than virtualisation.

  1. Download a copy of the latest free version of XenServer ISO.
  2. Burn the ISO to disk (or USB etc) and prepare the machine to boot from it.
  3. Power up the machine and follow the simple installation steps.
  4. Choose the local SSD drive for the destination media.
  5. Chose the local DVD for installation media.
  6. Chose not to install any additional ‘packs’.
  7. Provide a root password when prompted.
  8. Change the network settings to fixed (assign IP, gateway etc).
  9. Rename the hostname to something memorable (I used “xenserver-monroe”).
  10. Choose the timezone details.
  11. Enter an NTP server.
  12. Continue to install the software and when completed, remove the installation media and reboot.
  13. Repeat as required for each host.

This YouTube video might be useful for those installing XenServer for the first time.

XenCenter

This is currently a Windows only app, so I had to use my VM. Very easy to set up and use, and definitely worth the hassle of setting up Windows. You can even download a version of Windows 7 that works without registration for 30 days if you don’t already have a license.

  1. Download the XenCenter installer to the Windows OS (in my case, a VM on the Mac).
  2. Install using the defaults.
  3. Load the software up.
  4. Click the “Add Server” icon.
  5. Add the IP, username (root) and password.
  6. Choose to save and restore the connection state on start up.
  7. Create a master-password for all your XenServers (this allows you to access all the servers you connect with a single-sign-on).
  8. Repeat steps 1-6 for all XenServer hosts.
  9. Right-click one of the hosts.
  10. Choose “Add to Pool” -> “New Pool”.
  11. Enter a Pool name (I used “hollywood”).
  12. Right click the the other XenServer hosts and add to the same Pool.

FreeNAS Server

After doing some light reading, I decided on FreeNAS as the shared storage platform. It is well documented, and free to use. It also supports iSCSI which seems to be the preferred method of connecting storage to XenServer.

  1. Disable the hardware RAID on the server. I want FreeNAS to handle all the disks itself.
  2. Download the latest IMG file for USB installation. This is not an installer, but rather a disk image that is a pre-installed version of FreeNAS. This means you put the USB in and it boots up a working version of FreeNAS.
  3. Copy the image to the USB stick. I used this guide.
  4. Install the USB in to a suitable slot (I used the internal), set any required BIOS settings to allow booting from USB, and boot up.
  5. Follow the simple on-screen instructions.
  6. When provided with the URL for web-based management, switch to your workstation, open your browser and navigate to that address.
  7. Log in (default credentials are “root” / “freenas”).
  8. Go to the Interfaces menu and Add new.
  9. Set up the interface with a fixed IP. After you change this, you will have to navigate to the new IP in your browser.
  10. Go to the Storage menu, open the Volumes option and open the ZFS volume manager.
  11. From here you can create a volume from the installed hard drives and choose from a variety of disk formats. I went with RAIDZ2 (after reading this) – but suggest you read in to what each of the different methods offer. From my 4 x 2TB disk, I now have a single 3.6TB volume called “nas01”.
  12. Next, open the Services menu and find “iSCSI”.
  13. Open “Extents” and “Add Extent”. I named mine “extent01”.
  14. Open “Initiators” and “Add Initiator”. I named mine “init01”.
  15. Open “Portals” and “Add Portal”. I named mine “portal01”.
  16. Open “Targets” and “Add Target”. I named mine “target01”. Select your Portal and Initiator and select “none” for both the “Auth method” and “Authorisation Group Number”.
  17. Open “Targets / Extents” and “Add Target / Extent”. Select your Target and Extent.
  18. Go to the main Services page and enable the iSCSI service.

Connecting FreeNAS to XenCenter

Now we have the XenServers and the FreeNAS box running, we need to tell the XenServers that the Pool can use the FreeNAS box as a shared storage device. This was actually very painless.

  1. Choose “Add SR” from the “Storage” menu in XenCenter.
  2. Select “Software iSCSI”.
  3. Give the repository a suitable name (eg. “primary-nas”).
  4. Enter the IP of the FreeNAS box and click the “Discover IQNs” button.
  5. Select (if not already selected) your IQN.
  6. Click the “Discover LUNs” button.
  7. Select (if not already selected) your LUN.
  8. Click Finish.

You may want to add some more storage for ISO’s and images – but this is also simple to do with FreeNAS. I ended up creating a CIFS share since it is then easy to access from Windows boxes too.

Xen Orchestra

You can download a virtual machine of the Xen Orchestra software from their website. This is a pre-built Debian-based VM with the Xen Orchestra software installed. It can be added to the Pool by simply using the Import menu item (for some reason, this took several hours on my set up).

At this stage, the Xen Orchestra software seems like it needs quite a lot of work to be able to use it in production. It is very easy to connect to the existing Pool (you enter the web-interface, add the IP, username and password of one of the VM’s are you’re done) and you can see that the project has great potential. Its unfortunate that at this early stage, many of the useful features seem to just be a button that leads to a “to do” page. It definitely looks very promising and I hope to be able to contribute to the project.

I’m also going to look in to the XenServer API – since it seems quite straight forward and hope to be able to take a look at both OpenStack and CloudStack soon.

7 thoughts on “Building a XenServer Stack

  1. Nice article, thanks for sharing. The grey font on a white background is very hard to read, I had to change colour via web dev toolbar to be able to read.

    Thanks

  2. where are you install the xenserver? the n54l or the gen8>?

    when i install the xenserver in the gen8, it cannot boot up in raid mode.

    What is your way?

    1. Xenserver is install on the Gen8’s. I have a single small SSD in there since I could not get the SD-Card to boot (it does work fine for VMWare though). However, I dont really use the SSD space for the VM’s – which are hosted on a FreeNas share.

  3. Hello,

    this is a really great tutorial and exactly what I’d like to build. I’l like to install Xenserver 6.5.0.
    I use the same HP Microserver G1610T Gen8 like you, but when trying the install from usb-stick it fails with ”
    Problem With Location

    Setup was unable to acess the location you specified – please check and try again”.

    After reading the following posts I enabled AHCI support in BIOS

    http://h30499.www3.hp.com/t5/ProLiant-Servers-Netservers/HP-Microserver-G8-and-XenServer/td-p/6654248
    http://h20564.www2.hp.com/hpsc/doc/public/display?docId=emr_na-c03264177
    http://homeservershow.com/forums/index.php?/topic/5980-has-anyone-loaded-run-whs-2011-on-a-gen8/page-4#entry79759

    But this had no effect!

    I would be very grateful, if you could tell me your BIOS-Settings (especially changes from default), so I could proceed.
    What kind of DVD-drive you are using? What are the neccessary standards to comply with in order to get it working an the Proliant-Server.

    Thank you in advance! You are my hope!

    Stefan

    1. Sorry for the late reply Stefan!

      I had all sorts of problems trying to get things working from USB drive or SD Card – and in the end installed small SSD drives in each of the Gen8’s on to which Xenserver was installed. I think we were able to install a slightly older version of Xenserver from USB – but it was a lot of hassle. I have an external USB-based DVD drive that I think we have since used to rebuild. I would recommend going this route if you could – as Xenserver seems to approve.

      I dont have a DVD drive in any of the machines… though I did buy a nice half-height Blu-ray – which turned out not to fit. 🙁

      Anyway – hope this helps a little – even though its not really the solution!

      1. Hi Samuel

        thank you very much for your feedback! I bought an USB-DVD-Drive and installed the latest version of XenServer with enabled AHCI. I think, I’ll try the setup with an SSD for the XenServer-OS. At the moment I use an old HD for testing. I agree with you, that this is the only working solution. USB-stick is no option.

        It would be great to hear the latest development in you xenserver-setup!

        Thank you,

        Stefan

Leave a Reply

Your email address will not be published. Required fields are marked *