How I Rescued My Ubuntu VM from a Full Disk Issue

 Running Linux virtual machines (VMs) often requires careful monitoring of disk space. Recently, I faced a serious issue: one of my Ubuntu VMs filled up its disk entirely. After rebooting, it became stuck at the "Reached target cloud-init.target" message.

An Engineer sitting beside his computer and thumbing up - UFOtechs

In this post, I’ll walk you through the exact problem, what caused it, and how I fixed it without losing data.


What Happened?

The VM’s root partition (/) became 100% full.

When Linux cannot write even temporary files, services start failing — including essential ones like cloud-init, which runs early during boot.

Even after expanding the virtual disk size in vCenter, the VM refused to boot normally.


The root causes were:

  • Disk full (no space for temporary system operations)
  • Partition was still at its original size (500 GB) even though the disk was extended to 1 TB.
  • Filesystem was not resized.


Symptoms

When rebooting the VM, the last message displayed was:

Reached target cloud-init.target - Cloud-init target{codeBox}

And the system hung indefinitely after that.


Step-by-Step Solution

Step 1: Boot into Recovery Mode or Emergency Mode

To fix the VM, I needed a minimal root shell.

  1. Reboot the VM.
  2. When the system starts, hold Shift (for BIOS VMs) or press ESC (for UEFI VMs) to access the GRUB menu.
  3. Choose Advanced options for Ubuntu > (recovery mode).

    How I Rescued My Ubuntu VM from a Full Disk Issue - Step 1.2

    How I Rescued My Ubuntu VM from a Full Disk Issue - Step 1.3

  4. If GRUB does not appear automatically, you may need to modify the GRUB config temporarily and append systemd.unit=emergency.target to the kernel boot line.


Step 2: Mount the Root Filesystem as Read-Write

By default, recovery shells mount / as read-only, so remount it.

mount -o remount,rw /{codeBox}
 
How I Rescued My Ubuntu VM from a Full Disk Issue - Step 2


Step 3: Confirm the Disk Extension

Check the actual disk size using:

lsblk
fdisk -l{codeBox}
 
How I Rescued My Ubuntu VM from a Full Disk Issue - Step 3

I saw that /dev/sda was correctly showing 1TB, but the root partition /dev/sda2 was still 500GB.


Step 4: Resize the Partition

Using parted, I resized the partition to fill the disk:

parted /dev/sda{codeBox}
 
How I Rescued My Ubuntu VM from a Full Disk Issue - Step 4.1

At the (parted) prompt:

print
resizepart 2
100%
quit{codeBox}
 
How I Rescued My Ubuntu VM from a Full Disk Issue - Step 4.2

This safely resized partition 2 to use 100% of the available disk space.


Step 5: Expand the Filesystem

Depending on the filesystem type

For ext4 (most common):

resize2fs /dev/sda2{codeBox}
 
How I Rescued My Ubuntu VM from a Full Disk Issue - Step 5

For XFS:

xfs_growfs /{codeBox}

After this, the root filesystem grew to match the full partition size.


Step 6: Free Up Immediate Space (Optional but Recommended)

If the filesystem was still full, I cleared unnecessary files:

apt clean
rm -rf /var/log/*.gz /var/log/*.1
rm -rf /var/log/journal/*{codeBox}


Step 7: Reboot

After resizing and cleaning up, I rebooted:

reboot{codeBox}


The VM booted normally without hanging at cloud-init.target 👌


Conclusion

Facing a stuck Ubuntu VM can be nerve-wracking, but with recovery mode, parted, and resize2fs, you can often solve disk problems without restoring from backup.

I hope this guide helps others who face a similar situation.

Post a Comment

Previous Post Next Post

Contact Form