28

i am running Ubuntu 11.04 instance for my Web Server on AWS cloud, now i am getting there is no disk space in / partition of my server. df -ah say this

Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            7.9G  7.8G   97M  99% /
proc                     0     0     0   -  /proc
none                     0     0     0   -  /sys
fusectl                  0     0     0   -  /sys/fs/fuse/connections
none                     0     0     0   -  /sys/kernel/debug
none                     0     0     0   -  /sys/kernel/security
none                  3.7G  112K  3.7G   1% /dev
none                     0     0     0   -  /dev/pts
none                  3.7G     0  3.7G   0% /dev/shm
none                  3.7G   80K  3.7G   1% /var/run
none                  3.7G     0  3.7G   0% /var/lock
/dev/xvdb             414G   16G  377G   4% /mnt

Now i have Tried these thing for getting some extra space on / partition

  • Clean up All Log files for Apache.
  • Removed all unnecessary files from server.
  • Home directory Cleanup.

But Still I am not getting enough space. This Instance type is m1.large with 8GB EBS. Now i am getting i have enough disk space in /dev/xvdb.

Is there a way i can allocate some diskspace to / from /dev/xvdb or Any other Ways. Please suggest me the possible solution for this.Is it possible to use the same /dev/xvdb partition with another instance.

1

4 Answers 4

26

The answer is twofold.

Workaround: use /dev/xvdb (/mnt) for temporary data

This is the so called ephemeral storage of your Amazon EC2 instance and its characteristics are vastly different than the of the persistent Amazon EBS storage in use elsewhere. In particular, this ephemeral storage will be lost on stop/start cycles and can generally go away, so you definitely don't want to put anything of lasting value there, i.e. only put temporary data there you can afford to lose or rebuild easily, like a swap file or strictly temporary data in use during computations. Of course you might store huge indexes there for example, but must be prepared to rebuild these after the storage has been cleared for whatever reason (instance reboot, hardware failure, ...).

Solution: resize /dev/xvda1 (/) to gain desired storage

This is the so called Root Device Storage of your Amazon EBS-backed EC2 instance, which facilitates Amazon EBS for flexibility and durability in particular, i.e. data put there is reasonably safe and survives instance failures; you can increase flexibility and durability even further by taking regular snapshots of your EBS volume, which are stored on Amazon S3, featuring the well known 99.999999999% durability.

This snapshot features enables you to solve your problem in turn, insofar you can replace your current 8GB EBS root storage (/dev/xvda1) with one more or less as large as you desire. The process is outlined in Eric Hammond's excellent article Resizing the Root Disk on a Running EBS Boot EC2 Instance:

As long as you are ok with a little down time on the EC2 instance (few minutes), it is possible to change out the root EBS volume with a larger copy, without needing to start a new instance.

If you properly prepare the steps he describes (I highly recommend to test them with a throw away EC2 instance first to get acquainted with the procedure, or automate it via a tailored script even), you should be able to finish the process with a few minutes downtime only indeed.

Most of the outlined steps can be performed via the AWS Management Console as well, which avoids dealing with the Amazon EC2 API Tools; this boils down to:

  • stop (not terminate!) the EC2 instance
  • detach the EBS volume from the stopped instance
  • create a snapshot of the detached EBS volume
  • create a new (larger) EBS volume from the created snapshot
  • attach the new EBS volume to the EC2 instance (Important! If this is your root device be sure it to name it exactly as the root device of the instance as it was mentioned e.g (/dev/sda1) or (/dev/xdva1) otherwise it will be attached as a block device and not a root device and you will not be able to start the instance as there will be no root device listed for the instance.)
  • SSH into the running instance and confirm everything is in order via df -ah
    • in case your system hasn't automatically resized the file system, you'll need to do this manually as explained in Eric's article

Good luck!


Alternative

Given the versatility and ease of use of these EBS volumes, an additional option would be to attach more EBS volumes to your instance and move clearly separable areas of concern over there.

For example, we are using a couple of pretty heavyweight Java applications, each consuming 1-2GB storage per version; to ease upgrading versions and generally be able to move these apps to different instances at my discretion, I've placed them on dedicated EBS volumes each, mount these to an instance and soft link them to the desired location, e.g. usually /var/lib/<app>/<version> and /usr/local/<app>/<version>.

With this method, we are currently running EC2 instances with the root device storage still at its default size of 8GB (just like yours), but sometimes up to 8 EBS volumes with varying sizes (1-15GB) attached as well.

You need to be aware of potential network performance issues though, insofar all these EBS volumes are using the very same LAN for their I/O, which might yield respective performance gains even, or saturate your network in extreme cases - so as usual this depends on the use case and workload at hand.

4
  • I am using /dev/xvdb to keep my database which is of size almost 16GB now, One background process is running to keep it up to date. So what should be best permanent storage for this database. Should i go for Amazon RDS or Amazon DynamoDB. whats your suggestion. i am running PHP server on this instance.
    – Sumant
    Apr 3, 2012 at 4:56
  • 2
    @Sumant: That's not good, so you did exactly what's dangerous, namely putting data to be persisted on a disk that can basically go away anytime (it usually doesn't, put one should treat it like so)? I hope I haven't bee misleading in this regard - please be extra careful when mitigating this in order to avoid data loss during the process (you do have database backups regardless, do you?)! Apr 3, 2012 at 8:53
  • @Sumant: Regarding your question - You shouldn't need to change your app architecture (or DB for that matter) at all to remedy the storage problem, just resize you root disk or attach more EBS volumes as suggested. If, however, you want to improve and decouple you DB tier as well, which is a good thing in principle with future growth in mind (but comes with respective costs from the start), and assuming you are currently running MySQL, then Amazon RDS would be the perfect and convenient choice. Amazon DynamoDB requires a complete new app architecture and applies to specific use cases only. Apr 3, 2012 at 8:56
  • 1
    @Sumant: Be advised though, that migrating your DB to a m1.small RDS instance might actually exhibit slower performance than your current MySQL on EC2, which is running m1.large with respective CPU and I/O performance benefits - whether this applies, depends on your current DB workload though. Of course, you can use larger RDS instances as well to remedy this, but your cost will increase accordingly. Apr 3, 2012 at 8:57
1

Yep a simple way it to fstab it and then mount it to say /var/www/html/files2/

then mkdir /var/www/html/files2/website then ln -s -d /var/www/html/website /var/www/html/files2/website

1
  • use UUID to mount the partition using command blkids and fdisk say '/dev/vxds/' to make the partition. Use midnight commander to move the files with F6 from one folder to the other, make sure you choose the correct folder under the mount position oh and of course you will need to 'mount -a' after adding to fstab Nov 23, 2015 at 22:27
0

Today I occurred the same problem , when you ceate the new ec2 intance by default EBS is 8GB. You can modify the size of the attached EBS without creating new intace or taking snapshot or detaching EBS.. Here are the three steps you can follow:

  1. Resize EBS Volume
  2. Resize partition
  3. Resize partition For the first step go to your AWS console and click EBS and change the desired size and click modify.

For the rest of the steps please follow this article if you have any quetion feel free to ask.

Thanks!

0

I suffered for hours today with this issue. So here is a simple guide on what I did to resolve an issue when the file system is full on AWS.

  1. Create a new volume in AWS
  2. Attach additional volume to instance
  3. Format volume to required filesystem
  4. Find EMPTY folder on drive that is full (e.g. /mnt)
  5. Mount the new volume to the drive that is full (mount new volume to /mnt)
  6. Move the largest file you can find to the new volume
  7. Create a symbolic link to the new file

This will create space on the old file system and you will still have access to all the files.

Have fun!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .