Solved the problem myself!

For some reason the ec2-cert.pem was missing from its default location.

I fired up another instance with another (public) AMI. There I found the ec2-cert.pem file. I copied the file to the instance from which I was trying to build the ami.

Retried the ec2-bundle-vol and presto!

I don't know wether or not this is the right way, but it worked.

I would love to know the underlying reason for this as I just ran into the same problem!

I never deleted /opt/aws/amitools/ec2/etc/ec2/amitools/cert-ec2.pem, so how did it disappear with the first bundling?

This happened to me, perhaps you're running into the same filtering issue as me.

When you bundle your instance, ec2-bundle-vol omits all *.pem files, including those within ec2-api-tools and ec2-ami-tools. So if you bundle your instance, launch a new one, then attempt to bundle again, your tools installation will be missing the certificate file (/opt/aws/amitools/ec2/etc/ec2/amitools/cert-ec2.pem) and will fail as you describe.

You can see the list of file filters that are built in in lib/ec2/platform/base/constants.rb. You can forcibly use the --include option, to override this and include files and avoid this problem.

--Ben Mishkin

Based on the filter rules, I would not advise any individual to use the current ec2-bundle-vol on any Ubuntu or Debian AMI.

The filters have at least three major problems:
1. They remove system-installed SSL Pem files used to check the validity of SSL certificates. That means that if your stack uses any SSL authentication, you'll see errors when fetching from HTTPS locations. If you install your own certs, then rebundling will completely remove the certs. Additionally, the rules only remove *pem certificate types (such as der or pkc12). If the certificates use a different file name (i.e. cert-pem.crt, cert-pem.key) they will remain. This rule is completely nonsensical.
2. They remove all GPG keys, which means that you will get software authentication errors when installing new software. For Ubuntu, after bundling, you will not be able to install any software in the end result.
3. They remove most of the SSH host keys, but fail to remove the id_ecsda (Eliptical Curve) which is preferred by newer OpenSSH. That means that the identity of the new AMI will be that of the old AMI, and could represent a significant security vulnerability.

Edited by: B. Howard on Apr 27, 2012 11:01 AM

On debian, the following hack works:

apt-get remove --purge ec2-ami-tools
cd /tmp
apt-get install ruby libopenssl-ruby curl rpm alien
wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm
alien --to-deb ec2-ami-tools.noarch.rpm
dpkg -i ec2-ami-tools_1.4-1.7_all.deb
rm ec2-ami-tools*

This purges the ec2-ami-tools and then re-installs it, along with the missing certificate.

Attention AWS -- having ec2-bundle-vol omit pem files by default is the most idiotic thing ever. My instance is now missing all of its pem files and I have to waste time cleaning this up now. Lame.

+1 on this. This is pretty annoying behavior.

+1. And as usual I found this problem too late. Can this be fixed ASAP?

+1 Why is there no explanation anywhere that this is done. No default excludes in help pages or anything and no easy way to override this

+1, this is extremely annoying.

+1. I'm having to work with this on CentOS 6.2 also...

Jeff

Message : "No such file or directory - /opt/aws/amitools/ec2/etc/ec2/amitools/cert-ec2.pem"

A - If you already have the above message when re bundeling your AMI, you could (as root) :

1 - get it back from here : wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm
2 - extract it from the archive : tar xvf ./ec2-ami-tools.noarch.rpm ./etc/ec2/amitools/cert-ec2.pem
3 - copy it to the appropriate dir : cp ./etc/ec2/amitools/cert-ec2.pem /opt/aws/amitools/ec2/etc/ec2/amitools/
4 - delete the temp extract : rm -rf ./etc

B - If you don't want to have this error, just include the file when bundeling your AMI : ec2-bundle-vol -i /opt/aws/amitools/ec2/etc/ec2/amitools/cert-ec2.pem ...

<!-- EDIT : the include doesn't work for the cert-ec2.pem | A and C are still corect -->

C - You could include any other files like /home/MyAppUser/.ssh/authorized_keys with the same include option : ec2-bundle-vol -i /home/MyAppUser/.ssh/authorized_keys

WARNING : It was just a "How to", But always be really careful when including any sensitive data in your AMI ...

Edited by: xavm on Jan 10, 2013 3:02 PM

This was incredibly annoying given that what we were trying to do is create a backup image that we could recover from in the event of a failure - or to stand up a new test instance. The existing image script made that a real pain and it took us a bit to recover. Once we realized what was going on - (thanks to the earlier responses to this post) it wasn't hard to fix - but it was a bit time consuming.

Hopefully this post helps save someone time… and I do hope someone adds an option to the ec2 imaging tools so that people don't get hurt by this in the future… perhaps a non-optional argument on the command line?

---

If you've gotten bit by this problem - this process will only help if you have the old host lying around.

Take a look at "/usr/lib/ec2-ami-tools/lib/ec2/platform/base/constants.rb" on the old host. You should see a list of filtered files towards the bottom of the file. It'll look something like this:

...
        module Security
          FILE_FILTER = [
              '"*/#*#"',
              '"*/.#*"',
              '"*.sw"',
              '"*.swo"',
              '"*.swp"',
              '"*~"',
              '"*.pem"',
              '"*.priv"',
              '"*id_rsa*"',
              '"*id_dsa*"',
              '"*.gpg"',
              '"*.jks"',
              '"*/.ssh/authorized_keys"',
              '"*/.bash_history"']
...

Using this filtered list as our guide - we used rsync to fix the existing image - running a line like this on the new host for each type of file we wanted to pull back in (e.g. here's how you'd pull all pem files across ):

rsync -amvz --include "**/" --exclude "proc/**" --include "*.pem" \
    --exclude "*" old-host:/  /tmp/missing/ 

Run this command for each file type you want to include - (e.g. substituting "gpg" for "pem" above to pull across gpg files) - dropping everything in /tmp/missing per the command above.

We took a look at what we collected as a sanity check deleting anything we didn't want to apply out of /tmp/missing:

find /tmp/missing/ 

We then used the following to apply the files to the new system ignoring any files that already exist:

rsync -amvz --ignore-existing /tmp/missing/ / 

If you're like us you may want to be sure this doesn't happen to you again. Note that if you allow access to your .ssh/ key files and so forth anyone with this AMI may be able to get into your systems - - be very careful here, be sure you know what you're doing and proceed at your own risk. Edit the constants.rb file and comment out any filters you don't want applied to exclude files during AMI creation:

    sudo vi /usr/lib/ec2-ami-tools/lib/ec2/platform/base/constants.rb

---

I hope that helps someone.

All the best,
-Darrin

Edited by: darrinplex on Mar 5, 2013 8:19 AM

If you are using an Amazon Linux ami as your base, you can fix this problem with a single command:

yum reinstall aws-amitools-ec2.noarch

Edited by: "bobertthebold" on May 30, 2013 10:30 AM

Getting error during AMI through ec2-bundle-vol ..

Error:
mke2fs 1.42.9 (4-Feb-2014)
loop: can't delete device /dev/loop0: No such device or address
Bundling image file...
ERROR: error reading certificate file cert_ami.pem: error reading certificate: header too long

Please suggest.