11

I am trying to create an AMI based on ami-3b054701 in the ap-southeast-2 region. The Packer config has one simple provisioner:

#!/bin/bash -e

info() {
    echo -e "[INFO] $1\n"
}

exec_cmd() {
    echo -e "\n[INFO] $1\n"
    eval $1
}

exec_cmd "apt-get update"

exec_cmd "apt-get -y upgrade"

exec_cmd "apt-get -y install build-essential git zip unzip wget default-jre"

This sometimes work with no problem but sometimes fails with this error:

ami: [INFO] apt-get -y install build-essential git zip unzip wget default-jre
ami:
ami: Reading package lists... Done
ami: Building dependency tree
ami: Reading state information... Done
ami: Package build-essential is not available, but is referred to by another package.
ami: This may mean that the package is missing, has been obsoleted, or
ami: is only available from another source
ami:
ami: Package default-jre is not available, but is referred to by another package.
ami: This may mean that the package is missing, has been obsoleted, or
ami: is only available from another source
ami:
ami: Package zip is not available, but is referred to by another package.
ami: This may mean that the package is missing, has been obsoleted, or
ami: is only available from another source
ami:
ami: E: Package 'build-essential' has no installation candidate
ami: E: Package 'zip' has no installation candidate
ami: E: Package 'default-jre' has no installation candidate

I wasted one full day to find a way to avoid this. There is a suggested solution on Stack Exchange but it is more of a hack than a solution. Is there a way to prevent this error from happening intermittently?

Here's the full output of Packer

==> ami: Prevalidating AMI Name...
==> ami: Inspecting the source AMI...
==> ami: Creating temporary keypair: packer 55d33324-e319-bd49-24c5-c06a18b4fe09
==> ami: Launching a source AWS instance...
    ami: Instance ID: i-8b78cd55
==> ami: Waiting for instance (i-8b78cd55) to become ready...
==> ami: Waiting for SSH to become available...
==> ami: Connected to SSH!
==> ami: Provisioning with shell script: scripts/provision.sh
    ami:
    ami: [INFO] apt-get update
    ami:
    ami: Ign http://security.ubuntu.com trusty-security InRelease
    ami: Ign http://archive.ubuntu.com trusty InRelease
    ami: Get:1 http://security.ubuntu.com trusty-security Release.gpg [933 B]
    ami: Ign http://archive.ubuntu.com trusty-updates InRelease
    ami: Get:2 http://security.ubuntu.com trusty-security Release [63.5 kB]
    ami: Hit http://archive.ubuntu.com trusty Release.gpg
    ami: Get:3 http://archive.ubuntu.com trusty-updates Release.gpg [933 B]
    ami: Hit http://archive.ubuntu.com trusty Release
    ami: Get:4 http://archive.ubuntu.com trusty-updates Release [63.5 kB]
    ami: Get:5 http://security.ubuntu.com trusty-security/main amd64 Packages [333 kB]
    ami: Get:6 http://security.ubuntu.com trusty-security/restricted amd64 Packages [8875 B]
    ami: Hit http://archive.ubuntu.com trusty/main amd64 Packages
    ami: Get:7 http://security.ubuntu.com trusty-security/universe amd64 Packages [114 kB]
    ami: Hit http://archive.ubuntu.com trusty/restricted amd64 Packages
    ami: Hit http://archive.ubuntu.com trusty/universe amd64 Packages
    ami: Get:8 http://security.ubuntu.com trusty-security/multiverse amd64 Packages [3686 B]
    ami: Hit http://archive.ubuntu.com trusty/multiverse amd64 Packages
    ami: Get:9 http://security.ubuntu.com trusty-security/main Translation-en [181 kB]
    ami: Hit http://archive.ubuntu.com trusty/main Translation-en
    ami: Hit http://security.ubuntu.com trusty-security/multiverse Translation-en
    ami: Hit http://archive.ubuntu.com trusty/multiverse Translation-en
    ami: Hit http://security.ubuntu.com trusty-security/restricted Translation-en
    ami: Hit http://archive.ubuntu.com trusty/restricted Translation-en
    ami: Get:10 http://security.ubuntu.com trusty-security/universe Translation-en [66.6 kB]
    ami: Hit http://archive.ubuntu.com trusty/universe Translation-en
    ami: Get:11 http://archive.ubuntu.com trusty-updates/main amd64 Packages [605 kB]
    ami: Get:12 http://archive.ubuntu.com trusty-updates/restricted amd64 Packages [15.3 kB]
    ami: Get:13 http://archive.ubuntu.com trusty-updates/universe amd64 Packages [308 kB]
    ami: Get:14 http://archive.ubuntu.com trusty-updates/multiverse amd64 Packages [11.9 kB]
    ami: Get:15 http://archive.ubuntu.com trusty-updates/main Translation-en [292 kB]
    ami: Hit http://archive.ubuntu.com trusty-updates/multiverse Translation-en
    ami: Hit http://archive.ubuntu.com trusty-updates/restricted Translation-en
    ami: Get:16 http://archive.ubuntu.com trusty-updates/universe Translation-en [163 kB]
    ami: Fetched 2232 kB in 16s (134 kB/s)
    ami: Reading package lists... Done
    ami:
    ami: [INFO] apt-get -y upgrade
    ami:
    ami: Reading package lists... Done
    ami: Building dependency tree
    ami: Reading state information... Done
    ami: Calculating upgrade... Done
    ami: The following packages have been kept back:
    ami: linux-headers-generic linux-headers-virtual linux-image-virtual
    ami: linux-virtual
    ami: 0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
    ami:
    ami: [INFO] apt-get -y install build-essential git zip unzip wget default-jre
    ami:
    ami: Reading package lists... Done
    ami: Building dependency tree
    ami: Reading state information... Done
    ami: Package build-essential is not available, but is referred to by another package.
    ami: This may mean that the package is missing, has been obsoleted, or
    ami: is only available from another source
    ami:
    ami: Package default-jre is not available, but is referred to by another package.
    ami: This may mean that the package is missing, has been obsoleted, or
    ami: is only available from another source
    ami:
    ami: Package zip is not available, but is referred to by another package.
    ami: This may mean that the package is missing, has been obsoleted, or
    ami: is only available from another source
    ami:
    ami: E: Package 'build-essential' has no installation candidate
    ami: E: Package 'zip' has no installation candidate
    ami: E: Package 'default-jre' has no installation candidate
==> ami: Terminating the source AWS instance...
==> ami: No AMIs to cleanup
1
  • 1
    I have been struggling with this exact problem. Glad to know im not crazy.
    – Chris W.
    Mar 16, 2017 at 17:47

4 Answers 4

15

Most likely, you ran into a race condition with cloud-init. This packer issue gives some insights, and I have tried the following suggestion given by the OP with success:

echo "waiting 180 seconds for cloud-init to update /etc/apt/sources.list"
timeout 180 /bin/bash -c \
  'until stat /var/lib/cloud/instance/boot-finished 2>/dev/null; do echo waiting ...; sleep 1; done'
1
6

In my case prepending the text /usr/bin/cloud-init status --wait && to the execute_command guy solved the issue,

json file:

"provisioners" : [
    {
        "type" : "shell",
        "script" : "./setup.sh",
        "execute_command" : "/usr/bin/cloud-init status --wait && \
            sudo -E -S sh '{{ .Path }}'"
    }
]
2
  • 1
    Made my day. Thanks so much for posting this answer!
    – singleton
    Jun 17, 2020 at 5:10
  • This is the best practice. It should be marked as the solution
    – Carlos B
    Dec 16, 2021 at 20:39
1

This still seems to be a problem. I resolved it by creating a copy of the stock Ubuntu image and putting an apt-get wrapper on it that will wait for the existing installs to finish before continuing (instead of failing).

Steps to solve

  1. Login to the AWS console and navigated to the EC2 dashboard.
  2. Create a new instance with the Ubuntu image you want.
  3. SSH into the new instance and put the new apt-get script (see below) at /usr/local/sbin/apt-get
  4. Make it executable: sudo chmod 755 /usr/local/sbin/apt-get
  5. Ensure that /usr/local/sbin/ is at the front of the PATH variable in /etc/environment
  6. Logout of the machine.
  7. Back in the AWS cons
    1. Stop the ES2 instance: Actions > Instance State > Stop
    2. Create a new image from this instance: Actions > Image > Create Image
  8. Use the generated AMI for all new instances.

Script: apt-get

Script adapted from this

#!/bin/bash

#
# Wait for the package manager locks to be released before trying to call apt-get
#

i=0
tput sc

while fuser /var/lib/dpkg/lock >/dev/null 2>&1 || fuser /var/lib/apt/lists/lock >/dev/null 2>&1; do
    case $(($i % 4)) in
        0 ) j="-" ;;
        1 ) j="\\" ;;
        2 ) j="|" ;;
        3 ) j="/" ;;
    esac
    tput rc
    echo -en "\r[$j] Waiting for other software managers to finish..."
    sleep 0.5
    ((i=i+1))
done

/usr/bin/apt-get "$@"
0

I've asked the server team to take a look at this and while they work out an answer I can say that the solution on StackExchange pertains to "Hash Sum Mismatch" issues which are not seen in your output.

For some reason the archive metadata doesn't appear to have a match for the packages you're asking to install (which are in main) but apt knows that other packages depend on them so it's telling you that to say that it doesn't think you're crazy either. ;-)

2
  • Thanks Robert! I think I encountered the hash sum mismatch a couple of times as well. Anyway, I will update the post if I encounter it again. Also is there a better solution available for the hash sum mismatch? Because even by following that solution I still encountered it a couple of times. Aug 23, 2015 at 23:19
  • In EC2 there are in-cloud mirrors designed to minimize the mismatches. One updates a copy of the metadata & then move all of the region peers together. At its core, when the archives are updated, the release file is written & contains sums of the metadata. If a client grabs it & then pulls other newer files (timing issue) you get a mismatch. The mirrors are better but, there is still a small timing window & apt-get doesn't try again. For that, follow the example of deleting the local copy of the metadata & updating again.
    – rcj
    Aug 24, 2015 at 2:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .