UEC

This tutorial covers UEC installation from the Ubuntu 12.04 LTS Server Edition CD, and assumes a basic network topology, with a single system serving as the «all-in-one controller», and one or more nodes attached.

From this Tutorial you will learn how to install, configure, register and perform several operations on a basic UEC setup that results in a cloud with a one controller «front-end» and one or several node(s) for running Virtual Machine (VM) instances. You will also use examples to help get you started using your own private compute cloud.

To deploy a minimal cloud infrastructure, you’ll need at least two dedicated systems:

  A front end.

  One or more node(s).

The following are recommendations, rather than fixed requirements. However, our experience in developing this documentation indicated the following suggestions.

Use the following table for a system that will run one or more of:

  Cloud Controller (CLC)

  Cluster Controller (CC)

  Walrus (the S3-like storage service)

  Storage Controller (SC)

The other system(s) are nodes, which will run:

  the Node Controller (NC)

  Download the Ubuntu 12.04 LTS Server ISO file, and burn it to a CD.

  When you boot, select “Install Ubuntu Enterprise Cloud”. The installer will detect if any other Eucalyptus components are present.

  You can then choose which components to install, based on your chosen topology.

  When asked whether you want a “Cluster” or a “Node” install, select “Cluster”.

  It will ask two other cloud-specific questions during the course of the install:

      Name of your cluster.

          e.g. cluster1.

      A range of public IP addresses on the LAN that the cloud can allocate to instances.

          e.g. 192.168.1.200-192.168.1.249.

The node controller install is even simpler. Just make sure that you are connected to the network on which the cloud/cluster controller is already running.

  Boot from the same ISO on the node(s).

  When you boot, select “Install Ubuntu Enterprise Cloud”.

  Select “Install Ubuntu Enterprise Cloud”.

  It should detect the Cluster and preselect “Node” install for you.

  Confirm the partitioning scheme.

  The rest of the installation should proceed uninterrupted; complete the installation and reboot the node.

  Nodes are the physical systems within UEC that actually run the virtual machine instances of the cloud.

  All component registration should be automatic, assuming:

      Public SSH keys have been exchanged properly.

      The services are configured properly.

      The appropriate uec-component-listener is running.

      Verify Registration.

  Steps a to e should only be required if you're using the UEC/PackageInstall method. Otherwise, if you are following this guide, these steps should already be completed automatically for you, and therefore you can skip "a" to "e".

  Exchange Public Keys

  The Cloud Controller's eucalyptus user needs to have SSH access to the Walrus Controller, Cluster Controller, and Storage Controller as the eucalyptus user.

  Install the Cloud Controller's eucalyptus user's public ssh key by:

      On the target controller, temporarily set a password for the eucalyptus user:

      sudo passwd eucalyptus

      Then, on the Cloud Controller:

      sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub \
      eucalyptus@<IP_OF_NODE>

      You can now remove the password of the eucalyptus account on the target controller, if you wish:

      sudo passwd -d eucalyptus

  Configuring the Services

  On the Cloud Controller:

      For the Cluster Controller Registration:

          Define the shell variable CC_NAME in /etc/eucalyptus/eucalyptus-cc.conf

          Define the shell variable CC_IP_ADDR in /etc/eucalyptus/eucalyptus-ipaddr.conf, as a space separated list of one or more IP addresses.

      For the Walrus Controller Registration:

          Define the shell variable WALRUS_IP_ADDR in /etc/eucalyptus/eucalyptus-ipaddr.conf, as a single IP address.

  On the Cluster Controller:

      For Storage Controller Registration:

          Define the shell variable CC_NAME in /etc/eucalyptus/eucalyptus-cc.conf

          Define the shell variable SC_IP_ADDR in /etc/eucalyptus/eucalyptus-ipaddr.conf, as a space separated list of one or more IP addresses.

  Publish

  Now start the publication services.

      Walrus Controller:

      sudo start eucalyptus-walrus-publication

      Cluster Controller:

      sudo start eucalyptus-cc-publication

      Storage Controller:

      sudo start eucalyptus-sc-publication

      Node Controller:

      sudo start eucalyptus-nc-publication

  Start the Listener

  On the Cloud Controller and the Cluster Controller(s), run:

  sudo start uec-component-listener

  Verify Registration

  cat /var/log/eucalyptus/registration.log
  2010-04-08 15:46:36-05:00 | 24243 -> Calling node cluster1 node 10.1.1.75
  2010-04-08 15:46:36-05:00 | 24243 -> euca_conf --register-nodes returned 0
  2010-04-08 15:48:47-05:00 | 25858 -> Calling walrus Walrus 10.1.1.71
  2010-04-08 15:48:51-05:00 | 25858 -> euca_conf --register-walrus returned 0
  2010-04-08 15:49:04-05:00 | 26237 -> Calling cluster cluster1 10.1.1.71
  2010-04-08 15:49:08-05:00 | 26237 -> euca_conf --register-cluster returned 0
  2010-04-08 15:49:17-05:00 | 26644 -> Calling storage cluster1 storage 10.1.1.71
  2010-04-08 15:49:18-05:00 | 26644 -> euca_conf --register-sc returned 0

  The output on your machine will vary from the example above.

After installing and booting the Cloud Controller, users of the cloud will need to retrieve their credentials. This can be done either through a web browser, or at the command line.

  From your web browser (either remotely or on your Ubuntu server) access the following URL:

  https://<cloud-controller-ip-address>:8443/

  You must use a secure connection, so make sure you use "https" not "http" in your URL. You will get a security certificate warning. You will have to add an exception to view the page. If you do not accept it you will not be able to view the Eucalyptus configuration page.

  Use username 'admin' and password 'admin' for the first time login (you will be prompted to change your password).

  Then follow the on-screen instructions to update the admin password and email address.

  Once the first time configuration process is completed, click the 'credentials' tab located in the top-left portion of the screen.

  Click the 'Download Credentials' button to get your certificates.

  Save them to ~/.euca.

  Unzip the downloaded zip file into a safe location (~/.euca).

  unzip -d ~/.euca mycreds.zip

  Alternatively, if you are on the command line of the Cloud Controller, you can run:

  mkdir -p ~/.euca
  chmod 700 ~/.euca
  cd ~/.euca
  sudo euca_conf --get-credentials mycreds.zip
  unzip mycreds.zip
  ln -s ~/.euca/eucarc ~/.eucarc
  cd -

Now you will need to setup EC2 API and AMI tools on your server using X.509 certificates.

  Install the required cloud user tools:

  sudo apt-get install euca2ools

  To validate that everything is working correctly, get the local cluster availability details:

  . ~/.euca/eucarc
  euca-describe-availability-zones verbose
  AVAILABILITYZONE   myowncloud                 192.168.1.1
  AVAILABILITYZONE   |- vm types                free / max   cpu   ram  disk
  AVAILABILITYZONE   |- m1.small                0004 / 0004   1    128     2
  AVAILABILITYZONE   |- c1.medium               0004 / 0004   1    256     5
  AVAILABILITYZONE   |- m1.large                0002 / 0002   2    512    10
  AVAILABILITYZONE   |- m1.xlarge               0002 / 0002   2   1024    20
  AVAILABILITYZONE   |- c1.xlarge               0001 / 0001   4   2048    20

  Your output from the above command will vary.

The following is by far the simplest way to install an image. However, advanced users may be interested in learning how to Bundle their own image.

The simplest way to add an image to UEC is to install it from the Image Store on the UEC web interface.

  Access the web interface at the following URL (Make sure you specify https):

  https://<cloud-controller-ip-address>:8443/

  Enter your login and password (if requested, as you may still be logged in from earlier).

  Click on the Store tab.

  Browse available images.

  Click on install for the image you want.

Once the image has been downloaded and installed, you can click on «How to run?» that will be displayed below the image button to view the command to execute to instantiate (start) this image. The image will also appear on the list given on the Image tab.

There are multiple ways to instantiate an image in UEC:

  Use the command line.

  Use one of the UEC compatible management tools such as Landscape.

  Use the ElasticFox extension to Firefox.

Here we will describe the process from the command line:

  Before running an instance of your image, you should first create a keypair (ssh key) that you can use to log into your instance as root, once it boots. The key is stored, so you will only have to do this once.

  Run the following command:

  if [ ! -e ~/.euca/mykey.priv ]; then
      mkdir -p -m 700 ~/.euca
      touch ~/.euca/mykey.priv
      chmod 0600 ~/.euca/mykey.priv
      euca-add-keypair mykey > ~/.euca/mykey.priv
  fi

  You can call your key whatever you like (in this example, the key is called 'mykey'), but remember what it is called. If you forget, you can always run euca-describe-keypairs to get a list of created keys stored in the system.

  You must also allow access to port 22 in your instances:

  euca-authorize default -P tcp -p 22 -s 0.0.0.0/0

  Next, you can create instances of your registered image:

  euca-run-instances $EMI -k mykey -t m1.small

  If you receive an error regarding image_id, you may find it by viewing Images page or click "How to Run" on the Store page to see the sample command.

  The first time you run an instance, the system will be setting up caches for the image from which it will be created. This can often take some time the first time an instance is run given that VM images are usually quite large.

  To monitor the state of your instance, run:

  watch -n5 euca-describe-instances

  In the output, you should see information about the instance, including its state. While first-time caching is being performed, the instance's state will be 'pending'.

  When the instance is fully started, the above state will become 'running'. Look at the IP address assigned to your instance in the output, then connect to it:

  IPADDR=$(euca-describe-instances | grep $EMI | grep running | \
  tail -n1 | awk '{print $4}')
  ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR

  And when you are done with this instance, exit your SSH connection, then terminate your instance:

  INSTANCEID=$(euca-describe-instances | grep $EMI | grep running | \
  tail -n1 | awk '{print $2}')
  euca-terminate-instances $INSTANCEID

The cloud-init package provides «first boot» functionality for the Ubuntu UEC images. It is in charge of taking the generic filesystem image that is booting and customizing it for this particular instance. That includes things like:

  Setting the hostname.

  Putting the provided ssh public keys into ~ubuntu/.ssh/authorized_keys.

  Running a user provided script, or otherwise modifying the image.

Setting hostname and configuring a system so the person who launched it can actually log into it are not terribly interesting. The interesting things that can be done with cloud-init are made possible by data provided at launch time called user-data.

First, install the cloud-init package:

sudo apt-get install cloud-init

If the user-data starts with '#!', then it will be stored and executed as root late in the boot process of the instance's first boot (similar to a traditional 'rc.local' script). Output from the script is directed to the console.

For example, create a file named ud.txt containing:

#!/bin/sh
echo ========== Hello World: $(date) ==========
echo "I have been up for $(cut -d\  -f 1 < /proc/uptime) sec"

Now start an instance with the –user-data-file option:

euca-run-instances $EMI -k mykey -t m1.small –user-data-file=ud.txt

Wait now for the system to come up and console to be available. To see the result of the data file commands enter:

euca-get-console-output $EMI | grep --after-context=1 Hello
========== Hello World: Mon Mar 29 18:05:05 UTC 2010 ==========
I have been up for 28.26 sec

Your output may vary.

The simple approach shown above gives a great deal of power. The user-data can contain a script in any language where an interpreter already exists in the image (#!/bin/sh, #!/usr/bin/python, #!/usr/bin/perl, #!/usr/bin/awk … ).

For many cases, the user may not be interested in writing a program. For this case, cloud-init provides «cloud-config», a configuration based approach towards customization. To utilize the cloud-config syntax, the supplied user-data must start with a '#cloud-config'.

For example, create a text file named cloud-config.txt containing:

#cloud-config
apt_upgrade: true
apt_sources:
- source: "ppa:ubuntu-server-edgers/server-edgers-apache "

packages:
- build-essential
- pastebinit

runcmd:
- echo ======= Hello World =====
- echo "I have been up for $(cut -d\  -f 1 < /proc/uptime) sec"

Create a new instance:

euca-run-instances $EMI -k mykey -t m1.small –user-data-file=cloud-config.txt

Now, when the above system is booted, it will have:

  Added the Apache Edgers PPA.

  Run an upgrade to get all updates available

  Installed the 'build-essential' and 'pastebinit' packages

  Printed a similar message to the script above

The Apache Edgers PPA, in the above example, contains the latest version of Apache from upstream source repositories. Package versions in the PPA are unsupported, and depending on your situation, this may or may not be desirable. See the Ubuntu Server Edgers web page for more details.

The 'runcmd' commands are run at the same point in boot that the '#!' script would run in the previous example. It is present to allow you to get the full power of a scripting language if you need it without abandoning cloud-config.

For more information on what kinds of things can be done with cloud-config, see doc/examples in the source.

How to use the Storage Controller

Controlling eucalyptus services:

  sudo service eucalyptus [start|stop|restart] (on the CLC/CC/SC/Walrus side)

  sudo service eucalyptus-nc [start|stop|restart] (on the Node side)

Locations of some important files:

  Log files:

      /var/log/eucalyptus

  Configuration files:

      /etc/eucalyptus

  Database:

      /var/lib/eucalyptus/db

  Keys:

      /var/lib/eucalyptus

      /var/lib/eucalyptus/.ssh

Don't forget to source your ~/.euca/eucarc before running the client tools.

  For information on loading instances see the Eucalyptus Wiki page.

  Eucalyptus Project Site (forums, documentation, downloads).

  Eucalyptus on Launchpad (bugs, code).

  Eucalyptus Troubleshooting (1.5).

  Register your cloud with RightScale.

  You can also find help in the #ubuntu-virt, #eucalyptus, and #ubuntu-server IRC channels on Freenode.