BackgroundThis document describes how to build an image that can be deployed to your platform, which allows you to create servers based on the image. For information about deploying an image to your platform, see the Creating an Image article.
Images are essentially byte for byte copies of a disk. They are deployed onto new servers by thin provisioning. They differ from snapshots only in that they do not retain a reference to the disk from which they were created, and in that they are included in the list presented to the user of images on which servers can be based.
Customers can create private images, and then share them with other customers, a specific billing entity, or the entire platform (subject to the configuration dictated by the platform licensee). The licensee can associate them with a product offer that will generate appropriate billing if required.
Two steps are thus required to create a new public image:
- Creation of a functional private image.
- Publishing of that private image into a shared/public image.
The second of these steps is detailed in the Managing an Image article, so this document focuses on the first of these steps.
There are two main ways to build an image:
- Configuration and snapshotting of a server or disk.
- Off-server image creation.
These are alternative processes. The first method is the easiest and the most user-friendly. The second is the most flexible.
Configuration and Snapshotting of a server or disk
This is the simplest method. The steps are as follows:
- Create a server using an existing operating system image as supplied on your platform. You will have to abide by the licensing terms of this operating system.
Install your own custom components.
- Test your server, and make any required adjustments.
- Stop your server.
- Using the image widget, make an image of the server’s primary disk.
- Create a new server, based on this image.
- Test this; if it is unsuccessful, delete the server created in step 6 and the image, and repeat from step 3.
In order for your image to present well to end-users, you will have to take some care to ensure that inappropriate data is not included within it. This is especially relevant for operating system images (rather than appliance images). This means that you should ensure that:
- Your own test scripts are erased.
- No network-accessible logins are available (so, on a UNIX platform, there are no entries in
/etc/passwd
or /etc/shadow
with fixed passwords). - Logs and usage information are cleared (consider
/var/log
, utmp
, wtmp
, and dmesg
files). - Information which is supposed to be unique per server is regenerated (for instance ssh keys).
The easiest way to ensure these happen is by a “first boot” script. This can also set a unique initial password for one user (see section 4). As an alternative, it is possible to modify the disk directly. To do this:
- Stop your test server.
- Create a new server (call this “utility server”).
- Detach the disk from your test server.
- Attach the disk to your utility server.
- Start the utility server.
- Mount the new disk (e.g. ‘
mount /dev/sdb /mnt
’). - Make the appropriate modifications to the mounted disk.
- Unmount the new disk (e.g. ‘
umount /dev/sdb
’). - Stop the utility server.
- Detach the disk from the utility server.
- Reattach the disk to the test server (If you used the above workaround, also configure it as the primary disk).
- Create the image as described above.
Off-Server image creation
Brinkster Cloud Services will boot any disk that a normal x86 server will boot, provided it will run as a virtual machine. It is thus possible to create images on a separate server either manually, or using packages such as vmbuilder. You can even use a virtual machine running on the platform to do this, which makes transfer for images faster.
This process consists of two parts:
- Making the binary disk image.
- Transferring it into your platform.
Making the binary disk image
You can use any method you like to make a binary disk image (even a ‘dd’ from a server on your desk), but we suggest you use our tool-chain. This currently consists of two tools:sparsecopy and fsmaker. These tools are optimised for and only tested on a Linux platform, though sparsecopy should work on other platforms. However, this does not restrict your choice of operating system within your image; for instance, there is nothing to stop you using this to create an OpenBSD based image.
In short, what these do is as follows:
sparsecopy
is a block copy utility rather like dd, which works with sparse files and does intelligent things with partition tables. Sparse file support means it works more quickly, uses less disk space for the resultant images, and is more compatible with thin provisioning.fsmaker
takes a filing system root partition that does not necessarily have a kernel or a boot sector (such as you might use in Amazon's EC-2 service, or on Eucalyptus), and makes it into a bootable hard disk image by adding grub and a optionally a kernel and an initrd.
You will need sfdisk
and getext2fs
installed to use the fsmaker
utility (both of these are available in Ubuntu and Debian stock distributions, and may be available elsewhere).fsmaker also requires that sparsecopy is installed.
Both utilities will give appropriate usage instructions if invoked with no parameters.
To use fsmaker
, you will first require a copy of your file-system (to be installed in the first partition) of your image. You could create a 2GB file-system, for instance, by:
sparsecopy -f2G /dev/ null myfs
mkfs -t ext3 myfs
mount -o loop myfs /mnt
rsync -axSHAXD /my/files/to/copy /mnt
umount -d /mnt
|
Alternatively, you could use a utility such as vmbuilder to perform this operation, or you could use the img file from an existing AMI image (you may wish to modify it to removeec2init and ec2init-userdata which are Amazon specific items).
Then use fsmaker to build your image. In its simplest form this is:
fsmaker
will use grub, a recent known good kernel and initrd to boot your image, though on first boot you may wish to replace these with kernels inside your file-system. It is possible to specify your own kernel, ram-disk, and even grub installation through command-line options.
You can upload images to your platform using the Create Disk or Create Image buttons within the Disk and Image widgets. The only difference between these two buttons is that the former makes a disk and the latter makes an image. It is possible to make a disk/server into an image by using the “New Image” button on the Images widget. An image can be made into a disk either by provisioning a new server using that image, or by cloning it.
- For information about creating a disk, see the article on Creating a Disk.
- For information about creating an image, see the article on Creating an Image.
Transferring your image to the platform using the API.
Exactly the same functionality as set out above is available through the API. Please refer to the API call FetchDisk.
For more details, please refer to the API documentation (coming soon!)
For completeness, images can also be uploaded manually. This is occasionally useful for debugging, or when developing on the platform itself. To achieve this:
- Create a utility server (i.e. a Linux server of any sort).
- Create a new 20GB blank disk.
- Attach the blank disk to the server.
- Boot the server.
- If you have not done so already, install the tool-chain on the server (as above).
- Compress your image (e.g. ‘gzip myimage.img’).
- Transfer your compressed image (myimage.img.gz in the above example) to your server, using normal UNIX tools (e.g. scp, rsync –e ssh, wget, etc.).
- Uncompress the image directly onto the second hard disk (e.g. 'gzcat myimage.img.gz | sparsecopy - /dev/sdb').
- Reread the partition table (e.g. 'sfdisk –R /dev/sdb'.
- Stop the server.
- Create an image of the second disk.
First password creation
As any server created from a given image on Brinkster Cloud Services will have an identical disk configuration, there is an inherent risk to storing passwords (encrypted or otherwise) in a public image, as the password will be the same for all users. Unless the image is appropriately locked down (so, for instance, passwords can only be entered through the console interface) this would present a security risk as any person familiar with the passwords for that image could log into a freshly deployed server using that image via its network interface; this is clearly undesirable.
To avoid this problem, the Brinkster Cloud Services platform generates a unique initial password that a ‘first boot’ script can obtain and assign as the password to any user on the system.
These scripts work by using the Metadata Service to retrieve the password information, among other details that are available.
ISO support
The platform can also support disks in ISO format (subject to hypervisor support). This is particularly useful in order to upload ISO images of install CDs or DVDs. The platform will recognize an ISO format disk as such, and mounts it as a virtual CD drive rather than a virtual hard disk. Remaining disks will be mounted as hard disks in disk order. Boot order is set to CD then hard disk.