server,kvm: Avoid device id conflicts between data volumes, CD-ROMs and the config drive ISO by weizhouapache · Pull Request #14407 · apache/cloudstack

weizhouapache · 2026-10-15T11:01:02Z

Description

This PR fixes #13484 and #14321 (the fix for 4.22 is in #14073).

On KVM, a data volume could get the same device id as a CD-ROM or as the config drive ISO. When both devices end up with the same sd* target name (SATA CD-ROMs on UEFI/q35 Instances plus SCSI or SATA data disks), libvirt rejects the domain XML with target 'sde' duplicated for disk sources ...:

ConfigDrive: duplicated target for disk sources with Q35 machine type #13484: the config drive ISO is attached with the device id 4, but the allocators of volume device ids did not know about it.
KVM: Data disks and pre-allocated CD-ROMs can get conflicting target device names (multiple-ISO support, UEFI) #14321: with multiple ISO support, the KVM agent defines all the CD-ROMs supported by the host at boot (device ids 3, 4, ...), but the allocators only reserved the primary CD-ROM (3). Also, the second ISO and the config drive ISO both wanted the device id 4.

Besides the attach volume API, the same conflicts could happen when attaching volumes to a stopped Instance (the conflict only shows up when it is started), when restoring volumes/Instances from backup, and when importing Instances (unmanaged Instances and VMware to KVM), which numbered the data disks 1, 2, 3, 4, ...

Device ids of an Instance

Device	Device id
Root volume	`0`
ISO 1 (primary CD-ROM)	`3`
Config drive ISO (if the Instance uses config drive)	`4`, always
ISO 2, ISO 3, ...	`4`, `5`, ... without config drive; `5`, `6`, ... with config drive (the device id of the config drive is skipped)
Data volumes	`1`, `2`, then the next device ids which are not used by a CD-ROM. The maximum number of data volumes is not reduced by the reserved device ids.

vm.iso.max.count is the maximum number of ISOs, the config drive ISO is not counted. It is limited by the number of CD-ROMs supported by the host (host.cdrom.max.count: 2 for KVM, 1 if the host does not advertise it).
The ISO slots stored for an Instance (user_vm.iso_id and vm_iso_map) are not changed: they are 3, 4, 5, ... They are converted to device ids when the CD-ROMs are defined for the Instance and when an ISO is attached or detached. So an Instance which already has an ISO stored in the second slot does not need to be updated, and the config drive keeps the device id 4.
The KVM agent defines all the CD-ROMs supported by the host at boot, whether ISOs are attached or not, so their device ids are reserved too. The config drive ISO is one more CD-ROM.
An Instance uses config drive if the Dhcp, Dns or UserData service of any of its networks is provided by ConfigDrive (checked on the network providers, so it also works for Instances which have never been started), or if it has the config drive location detail.
CloudStack supports 2 CD-ROMs per Instance on the IDE bus (hdc and hdd, as the first two slots are kept for the disks). The CD-ROMs with a device id beyond them (e.g. ISO 2 after the config drive ISO) are put on the SATA bus by the KVM agent. UEFI Instances have all their CD-ROMs on the SATA bus.

Examples (KVM host which supports 2 CD-ROMs, default maximum of 6 data volumes):

`vm.iso.max.count`	Config drive	CD-ROM device ids	Reserved device ids	Data volumes
1 (default)	no	`3` ISO 1, `4` (empty)	`3`, `4`	`1`, `2`, `5`, `6`, `7`, `8`
2	no	`3` ISO 1, `4` ISO 2	`3`, `4`	`1`, `2`, `5`, `6`, `7`, `8`
1 (default)	yes	`3` ISO 1, `4` config drive	`3`, `4`, `5`	`1`, `2`, `6`, `7`, `8`, `9`
2	yes	`3` ISO 1, `4` config drive, `5` ISO 2	`3`, `4`, `5`	`1`, `2`, `6`, `7`, `8`, `9`

Libvirt targets of the last row: BIOS (IDE) Instance: ISO 1 hdc, config drive hdd, ISO 2 sdf; UEFI Instance with SCSI disks: ISO 1 sdd, config drive sde, ISO 2 sdf, data volumes sdb, sdc, sdg, sdh, ...

For a host which does not advertise the number of CD-ROMs (older agents, other hypervisors): one ISO CD-ROM (3), and the config drive ISO (4) if the Instance uses config drive.

Changes

VolumeApiService.getReservedDeviceIds(vm) returns the device ids reserved for the CD-ROMs (including the config drive ISO). It is used when allocating device ids in:
- attach volume (auto-allocated device ids, and validation of the deviceid parameter, whose error message now lists the reserved ids). This also covers Instances which are stopped.
- restore of volumes and Instances from backup (KVMGuru)
- import of Instances (unmanaged Instances, VMware to KVM). The NICs are not imported yet at that point, so the config drive is always assumed (getReservedDeviceIds(vm, true)).
TemplateManager.getVmCdromCount(vm) returns the number of CD-ROM devices of an Instance (including the config drive ISO). TemplateManagerImpl converts the stored ISO slots to device ids (getIsoDeviceSeq) when defining the CD-ROMs at boot (prepareIsoForVmProfile) and when attaching or detaching an ISO.
The KVM agent defines the CD-ROMs beyond the 2 IDE slots on the SATA bus.

Notes

The volumes which already use a reserved device id (e.g. created before this change) are not changed: such an Instance still fails to start until the volume is detached.
On an Instance using config drive, a second ISO can only be attached while it is running if vm.iso.max.count was at least 2 when it was started (the CD-ROM has to exist in the domain). Otherwise the Instance has to be restarted.
The number of ISOs shown by listVirtualMachines does not take the config drive ISO into account (it is not counted in vm.iso.max.count anyway).
Not changed here: q35 set only through kvm.guest.os.machine.type (without UEFI) still puts the CD-ROMs on the IDE bus, which q35 does not support.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Unit tests (TemplateManagerImplTest, VolumeApiServiceImplTest, ConfigDriveNetworkElementTest, KVMGuruTest, UnmanagedVMsManagerImplTest, LibvirtComputingResourceTest, LibvirtVMDefTest) cover the reserved device ids, the number of CD-ROMs, the config drive detection (from the network providers and from the location detail), the conversion of ISO slots to device ids, the device id allocation for attach volume, restore and import, and the SATA bus for the CD-ROMs beyond the IDE slots.

Manually tested on a KVM host (management server and agent built from this branch, Ubuntu 24.04 template, L2 networks with and without config drive, vm.iso.max.count 1 and 2):

BIOS (i440fx), no config drive: 1 and 2 ISOs, data volumes 1, 2, 5, 6, 7 on a running Instance, a third ISO rejected, explicit deviceid 3/4 rejected
BIOS, config drive, 2 ISOs: ISO 1 hdc, config drive hdd, ISO 2 sdf, data volumes 1, 2, 6; detaching ISO 2 does not touch the config drive; the ISOs are kept after a stop/start; adding a NIC re-attaches the config drive
UEFI (q35), SCSI disks, with and without config drive (the setups of ConfigDrive: duplicated target for disk sources with Q35 machine type #13484 and KVM: Data disks and pre-allocated CD-ROMs can get conflicting target device names (multiple-ISO support, UEFI) #14321): no duplicated target
Instance deployed stopped on a config drive network, 4 data volumes attached while stopped (1, 2, 6, 7), then started
Windows Server 2022 guest OS with secure UEFI (all SATA, data volumes attached while stopped), and "Other (64-bit)" with an IDE root disk
VM import of an unmanaged Instance with 4 data disks: device ids 1, 2, 6, 7 (instead of 1, 2, 3, 4), and the Instance starts with a valid domain

How did you try to break this feature and the system with this change?

Instances with config drive and vm.iso.max.count set to 2, including raising it while the Instance is running
Instances which have never been started, with config drive networks
Instances with ISOs stored in the second slot before and after the config drive is used
hosts which do not advertise the number of CD-ROMs

github-actions · 2026-10-15T11:08:34Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

Data volumes could be assigned the same device id as one of the CD-ROM devices or the config drive ISO, which end up with the same libvirt target name (e.g. sde) when they are on the same bus namespace (SATA with UEFI/Q35). With multiple CD-ROM support, the KVM agent defines all the CD-ROMs supported by the host at boot, so any slot after the primary one can conflict. Add VolumeApiService.getReservedDeviceIds(), which returns the device ids of the CD-ROM devices and of the config drive ISO when the Instance uses config drive. The number of CD-ROM devices is returned by TemplateManager.getVmCdromCount(), based on the CD-ROM cap advertised by the host and the ISOs attached. The config drive is detected from the network providers (Dhcp, Dns, UserData), so Instances which have never been started are covered as well as the ones with the config drive location detail. The reserved ids are skipped when allocating device ids in: - attach volume - restore of volumes and Instances from backup - import of Instances (unmanaged Instances and VMware to KVM) The max number of data volumes is preserved when device ids are reserved, and the error message of an invalid deviceId lists the reserved ids.

The config drive ISO is attached as a CD-ROM, but its device id was hardcoded to 4, which is also the device id of the second ISO CD-ROM when multiple ISOs are supported. With vm.iso.max.count >= 2 the Instance could get two CD-ROMs on the same device id. The config drive ISO is now the CD-ROM following the ones of the ISOs, so its device id is the primary CD-ROM device id plus the number of the CD-ROMs of the ISOs (TemplateManager.getConfigDriveDeviceId). As the config drive ISO takes one of the CD-ROMs supported by the host, the maximum number of ISOs of an Instance using config drive is lowered by one. The number of CD-ROM devices of an Instance (TemplateManager.getVmCdromCount) includes the config drive ISO, so the device ids reserved for the volumes follow it, and the volume device id allocators do not need to know about the config drive any more. The KVM agent finds the config drive CD-ROM by the path of its ISO, and keeps its label and bus when refreshing it, instead of using the hardcoded id 4 and the label hdd (which does not match the SATA CD-ROMs of UEFI Instances).

codecov · 2026-10-15T11:26:44Z

Codecov Report

❌ Patch coverage is 86.86869% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.41%. Comparing base (d3f0700) to head (de91dd4).

Files with missing lines	Patch %	Lines
...n/java/com/cloud/template/TemplateManagerImpl.java	89.13%	3 Missing and 2 partials ⚠️
...er/src/main/java/com/cloud/hypervisor/KVMGuru.java	70.00%	3 Missing ⚠️
...om/cloud/hypervisor/kvm/resource/LibvirtVMDef.java	60.00%	0 Missing and 2 partials ⚠️
.../apache/cloudstack/vm/UnmanagedVMsManagerImpl.java	86.66%	2 Missing ⚠️
...n/java/com/cloud/storage/VolumeApiServiceImpl.java	95.45%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #14407      +/-   ##
============================================
+ Coverage     20.39%   20.41%   +0.01%     
- Complexity    21283    21322      +39     
============================================
  Files          6481     6481              
  Lines        588040   588108      +68     
  Branches      72021    72034      +13     
============================================
+ Hits         119957   120049      +92     
+ Misses       455052   455029      -23     
+ Partials      13031    13030       -1

Flag	Coverage Δ
uitests	`3.70% <ø> (ø)`
unittests	`21.71% <86.86%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

vm.iso.max.count is the maximum number of ISOs of an Instance, so it should not take the config drive ISO into account: the config drive ISO is attached as an additional CD-ROM, following the ones of the ISOs. As the IDE bus only has the slots of the first MAX_CDROMS_PER_VM CD-ROMs (hdc and hdd), the KVM agent attaches the CD-ROMs beyond these slots (e.g. the config drive ISO following the maximum number of ISOs) on the SATA bus.

vishesh92 · 2026-10-15T12:33:46Z

        }
    }

+    @Override


The changes in this file looks good but I am not sure if this is the place where these methods should exist.

vishesh92

just one minor comment. otherwise the changes looks good.

The config drive ISO keeps the device id 4, whatever the number of CD-ROMs of the Instance. The CD-ROMs of the ISOs are attached around it. This reverts the commits: - server,kvm: Do not count the config drive ISO in vm.iso.max.count - server,kvm: Calculate the device id of the config drive ISO

The config drive ISO keeps the device id 4, whatever the number of ISOs of the Instance. When an Instance uses config drive, the CD-ROMs of the ISOs which would use that device id or the next ones are moved to the next device ids, i.e. the CD-ROMs are, by device id: ISO 1 (3), config drive (4), ISO 2 (5), ISO 3 (6), ... The ISO slots stored for the Instance (user_vm.iso_id and vm_iso_map) are not changed. They are converted to device ids when the CD-ROMs are defined for the Instance, and when an ISO is attached or detached, so Instances which already have an ISO in the slot following the first one do not need to be updated. The number of CD-ROM devices of an Instance (TemplateManager.getVmCdromCount) includes the one of the config drive ISO, so the device ids reserved for the volumes cover all of them. When the networks of an Instance are not known yet (import, restore from backup), the config drive is always assumed. CloudStack supports 2 CD-ROMs per Instance on the IDE bus (hdc and hdd, the first two slots are kept for the disks). The KVM agent defines the CD-ROMs beyond these slots on the SATA bus.

TemplateManager exposed the number of CD-ROMs of an Instance, which the only caller (VolumeApiServiceImpl) converted back to the device ids starting from the primary CD-ROM device id. Return the device ids directly (TemplateManager.getCdromDeviceIds), so the callers do not have to assume that they are consecutive, and the number of CD-ROMs becomes an internal detail of TemplateManagerImpl.

Damans227 · 2026-10-15T16:58:50Z

overall the changes look good.

nit: what happens if a running vm is added to a config drive network and then gets a second iso? looks like the iso goes to a drive that was never set up at start, so the attach fails until a restart.

weizhouapache · 2026-10-15T18:50:12Z

overall the changes look good.

nit: what happens if a running vm is added to a config drive network and then gets a second iso? looks like the iso goes to a drive that was never set up at start, so the attach fails until a restart.

good point.

if so, I would advise to reserve device id 3 as primary CD-ROM, and device id 4 as configdrive ISO.

if the VM use IDE controller which support 2 CDROMs, the additional CD-ROMs (from 2nd CDROM) will use SATA controller or 2nd IDE controller, due to hardcoded MAX_CDROMS_PER_VM = 2 , right ?

Damans227 · 2026-10-15T20:30:43Z

if the VM use IDE controller which support 2 CDROMs, the additional CD-ROMs (from 2nd CDROM) will use SATA controller or 2nd IDE controller, due to hardcoded MAX_CDROMS_PER_VM = 2 , right ?

@weizhouapache right, the limit of 2 is ours, not qemu's. with 4 always kept for the config drive, the ide bus is full after the first cd, so extra isos go on sata or a second ide controller. that also covers the running vm case, since the second iso no longer moves when a config drive shows up.

boring-cyborg Bot added component:api component:kvm component:networking component:storage labels Oct 15, 2026

github-actions Bot added the status:has-conflicts label Oct 15, 2026

weizhouapache added 2 commits October 15, 2026 13:15

weizhouapache force-pushed the main-fix-13484-14321 branch from 9fa2580 to 3076489 Compare October 15, 2026 11:18

weizhouapache changed the title ~~Main fix 13484 14321~~ server,kvm: Avoid device id conflicts between data volumes, CD-ROMs and the config drive ISO Oct 15, 2026

github-actions Bot removed the status:has-conflicts label Oct 15, 2026

vishesh92 reviewed Oct 15, 2026

View reviewed changes

weizhouapache added 3 commits October 15, 2026 16:31

weizhouapache mentioned this pull request Oct 15, 2026

server: Avoid device id conflicts between data volumes, the CD-ROM and the config drive ISO #14073

Draft

2 of 14 tasks

Navigation Menu

Repository navigation

server,kvm: Avoid device id conflicts between data volumes, CD-ROMs and the config drive ISO - #14407

server,kvm: Avoid device id conflicts between data volumes, CD-ROMs and the config drive ISO#14407
weizhouapache wants to merge 6 commits into
apache:mainfrom
weizhouapache:main-fix-13484-14321

weizhouapache commented Oct 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

codecov Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

vishesh92 Oct 15, 2026

Uh oh!

vishesh92 left a comment

Uh oh!

Damans227 commented Oct 15, 2026

Uh oh!

weizhouapache commented Oct 15, 2026 •

edited

Loading

Uh oh!

Damans227 commented Oct 15, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

weizhouapache commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Device ids of an Instance

Changes

Notes

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

codecov Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vishesh92 Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

vishesh92 left a comment

Choose a reason for hiding this comment

Uh oh!

Damans227 commented Oct 15, 2026

Uh oh!

weizhouapache commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Damans227 commented Oct 15, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

weizhouapache commented Oct 15, 2026 •

edited

Loading

codecov Bot commented Oct 15, 2026 •

edited

Loading

weizhouapache commented Oct 15, 2026 •

edited

Loading