Multipass instance stuck in unknown state - QEMU window output states image not found

[greg-pedder]

Describe the bug
I have been having issues with Multipass recently whereby the (SSH) connection between VSCode and my instance kept dropping.

I had decided to update Multipass to 1.15.0+mac-Darwin (from 1.12 if I remember) and all was initially well after restarting Multipass and the instance - however, the dropped connection between VSCode and my instance began again, and more frequently.

Thinking a restart of the Mac might correct things, I had stopped the running instance and then restarted the Mac.
This did not solve the problem.
When trying to start the instance, using either the GUI or command line, it ended up failing to start correctly, and reported 'unknown' in the State column in the GUI. The cli responded with "timed out waiting for response".

At the same time, the Mac needed updating to MacOS 15.2 Sequoia - which was a work related thing.

Trying to start the Multipass instance continues to see it end up in the 'unknown' state.

Having followed the advice from https://discourse.ubuntu.com/t/how-to-troubleshoot-launch-start-issues/48104 - including using the QEMU tool, I am unable to resolve getting the instance to start.

I'd rather not delete and recreate the instance, as I have work I'd rather not lose.

I have made snapshots, but as these are fairly old, I suspect they won't contain the work I need.

The section on using the QEMU tool in the troubleshooting guide produces the result that an image is not found during the initial boot sequence.

To Reproduce
How:
multipass start docker-dev

and what happened?
start failed: The following errors occurred:
docker-dev: timed out waiting for response

Expected behavior
What did you expect to happen?
In the GUI - State 'running' to then allow me to ssh into my instance.

Logs
Please provide logs from the daemon, see accessing logs on where to find them on your platform.

[2025-01-24T10:09:03.090] [debug] [docker-dev] process working dir ''
[2025-01-24T10:09:03.091] [info] [docker-dev] process program 'qemu-system-aarch64'
[2025-01-24T10:09:03.091] [info] [docker-dev] process arguments '-machine, virt,gic-version=3, -accel, hvf, -drive, file=/Library/Application Support/com.canonical.multipass/bin/../Resources/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on, -cpu, host, -nic, vmnet-shared,model=virtio-net-pci,mac=52:54:00:73:d7:c2, -device, virtio-scsi-pci,id=scsi0, -drive, file=/var/root/Library/Application Support/multipassd/qemu/vault/instances/docker-dev/ubuntu-22.04-server-cloudimg-arm64.img,if=none,format=qcow2,discard=unmap,id=hda, -device, scsi-hd,drive=hda,bus=scsi0.0, -smp, 4, -m, 14336M, -qmp, stdio, -chardev, null,id=char0, -serial, chardev:char0, -nographic, -cdrom, /var/root/Library/Application Support/multipassd/qemu/vault/instances/docker-dev/cloud-init-config.iso'
[2025-01-24T10:09:03.098] [debug] [qemu-system-aarch64] [5417] started: qemu-system-aarch64 -machine virt,gic-version=3 -nographic -dump-vmstate /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/multipassd.tsKXmy
[2025-01-24T10:09:03.157] [info] [docker-dev] process state changed to Starting
[2025-01-24T10:09:03.159] [info] [docker-dev] process state changed to Running
[2025-01-24T10:09:03.159] [debug] [qemu-system-aarch64] [5418] started: qemu-system-aarch64 -machine virt,gic-version=3 -accel hvf -drive file=/Library/Application Support/com.canonical.multipass/bin/../Resources/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -cpu host -nic vmnet-shared,model=virtio-net-pci,mac=52:54:00:73:d7:c2 -device virtio-scsi-pci,id=scsi0 -drive file=/var/root/Library/Application Support/multipassd/qemu/vault/instances/docker-dev/ubuntu-22.04-server-cloudimg-arm64.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 4 -m 14336M -qmp stdio -chardev null,id=char0 -serial chardev:char0 -nographic -cdrom /var/root/Library/Application Support/multipassd/qemu/vault/instances/docker-dev/cloud-init-config.iso
[2025-01-24T10:09:03.159] [info] [docker-dev] process started
[2025-01-24T10:09:03.160] [debug] [docker-dev] Waiting for SSH to be up
[2025-01-24T10:09:03.162] [debug] [arp] [5419] started: arp -an
[2025-01-24T10:09:03.396] [debug] [docker-dev] QMP: {"QMP": {"version": {"qemu": {"micro": 1, "minor": 2, "major": 8}, "package": ""}, "capabilities": ["oob"]}}

[2025-01-24T10:09:03.448] [debug] [docker-dev] QMP: {"return": {}}

[2025-01-24T10:09:05.206] [debug] [arp] [5423] started: arp -an
[This is repeated numerous times]
[2025-01-24T10:13:13.081] [debug] [arp] [5631] started: arp -an
[2025-01-24T10:13:13.265] [debug] [async task] fetch manifest periodically
[2025-01-24T10:13:13.272] [info] [VMImageHost] Did not find any supported products in "appliance"

Additional info

• OS: macOS 15.2 Sequoia
• multipass version 1.15.0+mac-Darwin
• multipass info
  Name: primary
  State: Stopped
  Snapshots: 0
  IPv4: --
  Release: --
  Image hash: 5167c1b13cb3 (Ubuntu 22.04 LTS)
  CPU(s): --
  Load: --
  Disk usage: --
  Memory usage: --
  Mounts: --

Name: docker-dev
State: Unknown
Snapshots: 2
IPv4: --
Release: --
Image hash: eafa7742ce5f (Ubuntu 22.04 LTS)
CPU(s): --
Load: --
Disk usage: --
Memory usage: --
Mounts: --

• multipass get local.driver - qemu

Additional context
Have followed the instructions for using the QEMU window to follow the boot process for VM boot failure - for the primary this ended up at the login prompt - for the instance that is failing, after creating a screen recording of the boot process, it reports:

[giuliazanchi]

Hi @greg-pedder , thank you for your detailed report, I'm really sorry you're experiencing this issue on a VM you were actively using.

This looks like a corrupted image problem. I see you've already gone through our troubleshooting page, did you also see the part on image corruption here? If qemu-img and fsck on a mounted image don't work, the last option would be to mount the image manually and recover the data, as explained at the end of the section.

Which command did you use to produce the output shown in your screenshot?

[greg-pedder]

Hi @giuliazanchi - thanks for getting back to me.

This looks like a corrupted image problem. I see you've already gone through our troubleshooting page, did you also see the part on image corruption here? If qemu-img and fsck on a mounted image don't work, the last option would be to mount the image manually and recover the data, as explained at the end of the section.

I am trying this as we speak, however, I am unable to reach the GRUB menu - I can get to the UEFI page, but on the Mac, holding down Shift as suggested doesn't get me in. Is there any other key combination that would get me to this?

Which command did you use to produce the output shown in your screenshot?
For the screen shot, I had run the qemu command as per the troubleshooting guide under https://discourse.ubuntu.com/t/how-to-troubleshoot-launch-start-issues/48104#p-119710-vm-boot-failure:

/Library/Application\ Support/com.canonical.multipass/bin/qemu-system-aarch64 -machine virt,gic-version=3, -accel hvf -drive file=/Library/Application\ Support/com.canonical.multipass/bin/../Resources/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -cpu host -nic vmnet-shared,model=virtio-net-pci,mac=52:54:00:02:7a:c6 -device virtio-scsi-pci,id=scsi0, -drive file=/var/root/Library/Application\ Support/multipassd/qemu/vault/instances/primary/ubuntu-22.04-server-cloudimg-arm64.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 1 -m 1024M -qmp stdio -cdrom /var/root/Library/Application\ Support/multipassd/qemu/vault/instances/primary/cloud-init-config.iso

If it helps, I have also ran:
/Library/Application\ Support/com.canonical.multipass/bin/qemu-img check -r all /var/root/Library/Application\ Support/multipassd/qemu/vault/instances/docker-dev/ubuntu-22.04-server-cloudimg-arm64.img

which fixed a number of issues, and now returns:

No errors were found on the image.
1761832/2457600 = 71.69% allocated, 16.51% fragmented, 0.02% compressed clusters
Image end offset: 153799884800

[greg-pedder]

Hi @giuliazanchi - would you have any other suggestions for how to reach the GRUB menu at all?