Discussion Forums



Thread: HOWTO Building a self-bundling Debian AMI

Welcome, Guest Help
Login Login


Permlink Replies: 13 - Pages: 1 - Last Post: Jun 22, 2009 2:35 PM by: Eric Hammond
Paul Morris

Posts: 13
Registered: 9/30/06
HOWTO Building a self-bundling Debian AMI
Posted: Sep 15, 2007 9:57 AM PDT
  Click to reply to this thread Reply

Hi all

Quite some time ago I posted instructions for making a Debian AMI on these forums. Since then I've been providing and regularly updating a public Debian Etch image for anyone to use.

Given the regular questions appearing on the forum about using Debian/Ubuntu with EC2 I thought it might be a useful exercise to post a complete HOWTO should people wish to roll their own AMI rather than use my public one. I suspect that these instructions will work for Ubuntu as well although I've not actually tested this.

To start with we need to construct a simple Debian installation on a host machine. I'm using my local Etch at home but this should also work fine on the EC2 public Etch instances.

1. Create a 1Gb disk image, format it ext3 and mount it:

dd if=/dev/zero of=debian.fs count=1024 bs=1M
mke2fs -F -j debian.fs
mount -o loop debian.fs /mnt

2. Using ‘debootstrap’ we install Debian Etch onto the mounted image, update the apt sources list and chroot into the image:

debootstrap -arch i386 etch /mnt http://ftp.debian.org
nano /mnt/etc/apt/sources.list (I just added the security updates line)
chroot /mnt

3. We need to create some devices:

mount /proc
cd /dev
/sbin/MAKEDEV console
/sbin/MAKEDEV std
/sbin/MAKEDEV generic (it's been suggested this isn't actually necessary but I left it in anyway just in case)

4. Setup the root password, network interfaces and fstab:

passwd (this is 'etch' on the public AMI)

nano /etc/network/interfaces

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp

nano /etc/fstab

/dev/sda1 / ext3 defaults 0 1
/dev/sda2 /mnt ext3 defaults 1 2
/dev/sda3 swap swap defaults 0 0

5. Now we use apt to update and add sshd:

apt-get update
apt-get dist-upgrade (don't worry about the locale warnings here, we'll fix this later)
apt-get install openssh-server
6. This simple install should now be capable of booting on EC2. Let's test this out before going any further:
exit (out of the chroot environment)
sync
umount -l /mnt (for lazy umount)
ec2-bundle-image -i debian.fs -k private_key -c certificate -u aws_id
ec2-upload-bundle -b bucket -m /tmp/debian.fs.manifest.xml -a s3_access_key -s s3_secret_key
ec2-register bucket /debian.fs.manifest.xml
7. This intermediate stage AMI should launch fine. If not take a look at the console output to debug. Once we have this basic image we then need to make it a bit more  EC2 friendly by installing the following packages onto it:
apt-get install locales libc6-xen  curl rsync ruby libopenssl-ruby1.8 module-init-tools openssl
dpkg-reconfigure locales (I selected en_US.UTF-8)
8. Download the EC2 AMI rpm tools package from the Developer Resources section of the AWS site and use 'alien' to convert it to a Debian package, then:
dpkg -i ec2-ami-tools_1.2-7222_all.deb
RUBYLIB=/usr/lib/site_ruby/
add the following to /root/.profile
RUBYLIB=/usr/lib/site_ruby/
export RUBYLIB

9. We need to modify the AMI tools slightly due to differences in MAKEDEV:

nano /usr/lib/site_ruby/aes/amiutil/image.rb

changing

exec( ‘for i in console null zero ; do /sbin/MAKEDEV -d ‘ + dev_dir + ‘ -x $i ; done’ )

on around line 150 to

exec("cd #{dev_dir} && /sbin/MAKEDEV console && /sbin/MAKEDEV std && /sbin/MAKEDEV generic")

 10. We also need the AWS kernel modules:

cd /

wget http://s3.amazonaws.com/ec2-downloads/modules-2.6.16-ec2.tgz

tar -xzvf modules-2.6.16-ec2.tgz

rm modules-2.6.16-ec2.tgz

depmod -a

11. Finally, in order to secure the image a bit we will disable root password login:

nano /etc/ssh/sshd_config

changing

PermitRootLogin yes

to

PermitRootLogin without-password

12. and enable the AWS public key login feature by appending the following to rc.local:

nano /etc/rc.local

if [ ! -d /root/.ssh ] ; then

    mkdir -p /root/.ssh

    chmod 700 /root/.ssh

fi

curl http://169.254.169.254/1.0//meta-data/public-keys/0/openssh-key > /tmp/my-key

if [ $? -eq 0 ] ; then

    cat /tmp/my-key >> /root/.ssh/authorized_keys

    chmod 600 /root/.ssh/authorized_keys

    rm /tmp/my-key

fi

exit 0

13. If the image is being bundled for public consumption you may wish to do the following where appropriate:

rm /root/.ssh/authorized_keys

rm /root/.bash_history && touch /root/.bash_history

14. We can now use the new 'self-bundling' ability of the instance:

ec2-bundle-vol -d /mnt -p ami_name -k private_key -c certificate -u aws_id -s 3072

ec2-upload-bundle -b bucket -m /mnt/ ami_name .manifest.xml -a s3_access_key -s s3_secret_key

ec2-register bucket / ami_name .manifest.xml

15. Again, if this AMI doesn't work on launch take a look at the console to figure out why (don't forget that root password login was disabled).

If you notice any bugs in this HOWTO please let us know. Otherwise the public  ami-30f11459 was made today using these same instructions if you want to try it out.

Best regards

Paul



Shawn Presser

Posts: 12
Registered: 4/12/07
Re: HOWTO Building a self-bundling Debian AMI
Posted: Sep 15, 2007 12:15 PM PDT   in response to: Paul Morris
  Click to reply to this thread Reply

This is amazing!  I set out to do exactly this today, but I didn't expect a full guide.  Thanks a lot!.


"tarot00-20"

Posts: 66
Registered: 6/22/07
Re: HOWTO Building a self-bundling Debian AMI
Posted: Sep 24, 2007 2:26 AM PDT   in response to: Paul Morris
  Click to reply to this thread Reply

Thanks!
I am looking for this kind of guide that can allow me to install:
Debian
Apache
php
Mysql
phpMyadmin
webmin
Squid

Thanks again


Meng Wong
RealName(TM)

Posts: 3
Registered: 6/27/07
64-bit Debian AMI
Posted: Oct 20, 2007 9:58 AM PDT   in response to: Paul Morris
  Click to reply to this thread Reply

I'd love to run Debian on a Large Instance.  Has anyone built one from the stock 64-bit Fedora AMI?  If not, that might become a weekend project.


Paul Morris

Posts: 13
Registered: 9/30/06
Re: 64-bit Debian AMI
Posted: Oct 24, 2007 1:57 PM PDT   in response to: Meng Wong
  Click to reply to this thread Reply

I haven't given it lots of thought but might this be as simple as specifying the appropriate system architecture during the debootstrap phase?

linuxtrek

Posts: 3
Registered: 8/7/07
Re: 64-bit Debian AMI
Posted: Nov 12, 2007 5:52 AM PST   in response to: Paul Morris
  Click to reply to this thread Reply

I can try this. What should be the arch? Should it be amd64?


linuxtrek

Posts: 3
Registered: 8/7/07
Re: 64-bit Debian AMI
Posted: Nov 13, 2007 4:27 PM PST   in response to: linuxtrek
  Click to reply to this thread Reply

I was able to put up a public AMI for Large Instance (ami-76cb2e1f). Please test.

Minor modifications to Paul's steps:

- Did debootstrap from an Ubuntu x86_64 public AMI (ami-99c025f0 - http://ec2onrails.rubyforge.org/)

NOTE: Do _not_ mount it on /mnt, /dev/sdb is already mounted on it.
- debootstrap command: debootstrap --arch amd64 etch /mnt1 http://ftp.debian.org

- as /etc/fstab entry:
# Supplied by: ec2-ami-tools-1.3-14051
/dev/sda1 /     ext3    defaults 1 1
/dev/sdb  /mnt  ext3    defaults 0 0
none      /proc proc    defaults 0 0
none      /sys  sysfs   defaults 0 0

- modify /usr/lib/site_ruby/aes/amiutil/image.rb as follows:
# MAKEDEV is incredibly variable across distros, so use mknod directly.
    #exec("mknod #{dev_dir}/null    c 1 3")
    #exec("mknod #{dev_dir}/zero    c 1 5")
    #exec("mknod #{dev_dir}/tty     c 5 0")
    #exec("mknod #{dev_dir}/console c 5 1")
    #exec("ln -s null #{dev_dir}/X0R")
    #Prem: modified for Debian:
    exec("cd #{dev_dir} && /sbin/MAKEDEV console && /sbin/MAKEDEV std && /sbin/MAKEDEV generic")

Hope this helps.


ultri

Posts: 2
Registered: 4/12/08
Re: HOWTO Building a self-bundling Debian AMI
Posted: May 14, 2008 9:24 PM PDT   in response to: Paul Morris
  Click to reply to this thread Reply

I tried this multiple time just now using a public etch image.

This works fine:
dd if=/dev/zero of=debian.fs count=1024 bs=1M
mke2fs -F -j debian.fs
mount -o loop debian.fs /mnt

The this doesn't seem to fail or complain, but no filesytem exists.
debootstrap -arch i386 etch /mnt http://ftp.debian.org

There must be something real obvious (to others) that I'm missing.

BEW




feedbayes

Posts: 11
Registered: 5/23/08
Re: HOWTO Building a self-bundling Debian AMI
Posted: Jun 7, 2008 1:59 AM PDT   in response to: Paul Morris
  Click to reply to this thread Reply

Hi all,

Sorry for replying to a very old topic, but I just wanted to add that if you umount /proc before exiting the chroot, you can umount the image without the -l (lazy). Much cleaner.

--
Kind regards,
Tim

jean-david hsu

Posts: 10
Registered: 6/14/08
Re: 64-bit Debian AMI
Posted: Jun 23, 2008 1:13 PM PDT   in response to: linuxtrek
  Click to reply to this thread Reply

changing fstab as stated, and apt-get install udev got the x86_64 ext3 going for me
Working on reiserfs


linuxconfig

Posts: 1
Registered: 8/22/08
Re: HOWTO Building a self-bundling Debian AMI
Posted: Sep 1, 2008 7:43 PM PDT   in response to: ultri
  Click to reply to this thread Reply

Hi BEW,

>> debootstrap -arch i386 etch /mnt http://ftp.debian.org

you need to use --arch instead of -arch. I also had the same problem.

also check this howto:
http://www.linuxconfig.org/Howto_CREATE_BUNDLE_UPLOAD_and_ACCESS_custom_Debian_AMI_using_ubuntu

lubo


muenchnerbuecherwurm

Posts: 2
Registered: 10/10/07
Re: HOWTO Building a self-bundling Debian AMI
Posted: May 22, 2009 6:49 AM PDT   in response to: Paul Morris
  Click to reply to this thread Reply

Thanks a lot for this howto!

This is a really great help (even after quite some time since you posted this).

Keep up the good work!

Cheers,
  Michael



Allen

Posts: 5,320
Registered: 3/19/07
Re: HOWTO Building a self-bundling Debian AMI
Posted: May 22, 2009 6:51 AM PDT   in response to: muenchnerbueche...
  Click to reply to this thread Reply

I believe alestic.com publishes a script they use to create debian & ubuntu ami's


Eric Hammond
RealName(TM)


Posts: 1,180
Registered: 7/7/07
Re: HOWTO Building a self-bundling Debian AMI
Posted: Jun 22, 2009 2:35 PM PDT   in response to: Allen
  Click to reply to this thread Reply

Yes, Allen, you're referring to this:

  http://code.google.com/p/ec2ubuntu/

This code is used to build the Ubuntu and Debian images for EC2 published on http://alestic.com

Here's a short intro to using it:

  http://alestic.com/2007/11/ec2ubuntu-build-ami

--
Eric Hammond




Point your RSS reader here for a feed of the latest messages in all forums