Skip to content

Fix Qemu probes for disk format and zero-byte skipping support - #14272

Open
sureshanaparti wants to merge 3 commits into
apache:4.22from
shapeblue:qemu-disk-format-and-zero-bytes-skipping-probe
Open

sureshanaparti wants to merge 3 commits into
apache:4.22from
shapeblue:qemu-disk-format-and-zero-bytes-skipping-probe

Conversation

@sureshanaparti

Copy link
Copy Markdown
Contributor

Description

This PR fixes two Qemu feature detection probes that broke between Qemu 8.x and Qemu 10.x:

  • Disk format support
  • Zero-bytes skipping

Disk format support

Due to changes in the format of qemu-img --help in Qemu 10.x, the previous logic always returns false:

$ qemu-img --help
...
Supported image formats:

  blkdebug blklogwrites blkverify compress copy-before-write copy-on-read
  file ftp ftps host_cdrom host_device http https io_uring luks nbd null-aio
  null-co nvme nvme-io_uring preallocate qcow2 quorum raw rbd
  snapshot-access throttle vdi vhdx virtio-blk-vfio-pci
  virtio-blk-vhost-user virtio-blk-vhost-vdpa vmdk vpc

The approach of doing a create -f $format -o help and relying on the exit code is more robust as a first level probe

--target-is-zero support

Note: From what I can tell, this is unused in native CloudStack plugins, but I figured this is easier to upstream if fixing this at the same time

Due to changes in the format of qemu-img --help in Qemu 10.x, the previous logic always returns false, as --target-is-zero is no longer a top-level suggested flag.

The flag continues showing in qemu-img convert --help however. The whole skipZero feature appears unused by native CloudStack, but this preserves the support while remaining backwards compatible with Qemu 8.x

ie:

$ qemu-img --version && qemu-img convert --help | grep target-is-zero
qemu-img version 10.1.0 (qemu-kvm-10.1.0-17.el9_8.5)
Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers
        [-n] [--target-is-zero] [-c]
  --target-is-zero
$ qemu-img --version && qemu-img convert --help | grep target-is-zero
qemu-img version 8.2.0 (qemu-kvm-8.2.0-11.el9_4)
Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers
  convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file [-F backing_fmt]] [-o options] [-l snapshot_param] [-S sparse_size] [-r rate_limit] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Tristan971 and others added 3 commits September 30, 2026 17:09
…pport probe

Due to changes in the format of `qemu-img --help` in Qemu 10.x, the previous logic always returns false, as --target-is-zero is no longer a top-level suggested flag.

The flag continues showing in qemu-img convert --help however. The whole skipZero feature appears unused by native CloudStack, but this preserves the support while remaining backwards compatible with Qemu 8.x

ie:
```
$ qemu-img --version && qemu-img convert --help | grep target-is-zero
qemu-img version 10.1.0 (qemu-kvm-10.1.0-17.el9_8.5)
Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers
        [-n] [--target-is-zero] [-c]
  --target-is-zero
```

```
$ qemu-img --version && qemu-img convert --help | grep target-is-zero
qemu-img version 8.2.0 (qemu-kvm-8.2.0-11.el9_4)
Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers
  convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file [-F backing_fmt]] [-o options] [-l snapshot_param] [-S sparse_size] [-r rate_limit] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename
```
Due to changes in the format of `qemu-img --help` in Qemu 10.x, the previous logic always returns false:
```
$ qemu-img --help
...
Supported image formats:

  blkdebug blklogwrites blkverify compress copy-before-write copy-on-read
  file ftp ftps host_cdrom host_device http https io_uring luks nbd null-aio
  null-co nvme nvme-io_uring preallocate qcow2 quorum raw rbd
  snapshot-access throttle vdi vhdx virtio-blk-vfio-pci
  virtio-blk-vhost-user virtio-blk-vhost-vdpa vmdk vpc
```

The approach of doing a create -f $format -o help and relying on the exit code is more robust as a first level probe
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@sureshanaparti

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@sureshanaparti sureshanaparti added this to the 4.22.2 milestone Sep 30, 2026
@codecov

codecov Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.99%. Comparing base (2e63c60) to head (eef39c8).

Files with missing lines Patch % Lines
...java/org/apache/cloudstack/utils/qemu/QemuImg.java 0.00% 6 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.22   #14272      +/-   ##
============================================
- Coverage     18.00%   17.99%   -0.02%     
+ Complexity    16219    16214       -5     
============================================
  Files          5936     5936              
  Lines        535716   535721       +5     
  Branches      65596    65597       +1     
============================================
- Hits          96459    96392      -67     
- Misses       428268   428346      +78     
+ Partials      10989    10983       -6     
Flag Coverage Δ
uitests 4.02% <ø> (ø)
unittests 19.06% <0.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19354

// - prints supported format options with rc=0 on known format, and
// - error message with rc=1 on unsupported format
final Script createProbe = new Script(_qemuImgPath, timeout);
createProbe.add("create", "-f", format.toString(), "-o", "help");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this still needed? #13587 already handles the new help header on 4.22, and the current check finds qcow2 in the 10.x output from the description

if (skipZeroIfSupported) {
final Script s = new Script(_qemuImgPath, timeout);
s.add("--help");
s.add("convert", "--help");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main already checks this flag a different way since #14256. should this match it so the forward merge doesnt conflict?

@andrijapanicsb

andrijapanicsb commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

+1 to both of @Damans227 points.

Note that zero-skipping is used on 4.22: backup restore, ScaleIO/PowerFlex, Linstor (zeroed devices) and StorPool encryption all ask for it. With qemu 10.x they currently write zeros into thin targets, so the second fix is worth having on 4.22.

One small request: could you fill in "How Has This Been Tested?" with the hosts it was checked on? For example, the qemu versions (8.2 / 10.1), whether host.volume.encryption shows correctly after an agent restart, and whether a copy that asks for zero-skipping (e.g. ScaleIO or Linstor) passes --target-is-zero in the agent log. That would make it easy to approve.

Happy to add a unit test for the probe when backporting, since main lacks one too.

@andrijapanicsb

andrijapanicsb commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

@sureshanaparti @nvazquez As offered, a unit test for the --target-is-zero probe is ready. It needs a small refactor of isTargetZeroFlagSupported(): the decision moves into a static method that takes qemu-img's output, with no change in behaviour. The new QemuImgTargetZeroFlagTest has six cases built from real qemu-img output: qemu 5.x–8.x that lists the flag in --help (EL8, EL9.4, Ubuntu 22.04 and 24.04), qemu 10.x where only the probe finds it, and a qemu-img that doesn’t know the option (none of the KVM hosts supported since 4.22, all of which ship qemu 6.2 or newer).

It needs neither qemu-img nor libvirt, so it runs in CI. That matters here: the smoke tests run on OL8, whose qemu still lists the flag in --help, so the qemu 10 probe path never runs there.

I'll open it as a separate PR against main. For 4.22 it can go in together with the backport here; I'm happy to share the patch for this branch if that's easier.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants