Skip to content

Extra lsilogic SCSI controller on aarch64 causes last disk unrecognized #12090

Closed
Bug
#12452
Closed
Extra lsilogic SCSI controller on aarch64 causes last disk unrecognized#12090
Bug
#12452
Assignees
vishesh92
Labels
status:needs-investigationtype:bug
Milestone
4.20.3

Description

@gw769
gw769
opened on Nov 18, 2025

problem

Environment

  • CloudStack Version: 4.20.1.0
  • CPU Architecture: aarch64
  • Hypervisor: KVM
  • UEFI Configuration (relevant for aarch64 VM boot, from /etc/cloudstack/agent/uefi.properties):
# CloudStack Agent UEFI Configuration for ARM64
# This file configures UEFI boot support for ARM64 virtual machines

# Secure boot mode with Microsoft keys
guest.nvram.template.secure=/usr/share/AAVMF/AAVMF_VARS.ms.fd
guest.loader.secure=/usr/share/AAVMF/AAVMF_CODE.ms.fd

# Secure boot mode without Microsoft keys
guest.nvram.template.secboot=/usr/share/AAVMF/AAVMF_VARS.fd
guest.loader.secboot=/usr/share/AAVMF/AAVMF_CODE.secboot.fd

# Standard UEFI mode (default)
guest.nvram.template.legacy=/usr/share/AAVMF/AAVMF_VARS.fd
guest.loader.legacy=/usr/share/AAVMF/AAVMF_CODE.fd

# No secure boot mode (explicitly disabled)
guest.nvram.template.nosecboot=/usr/share/AAVMF/AAVMF_VARS.fd
guest.loader.nosecboot=/usr/share/AAVMF/AAVMF_CODE.no-secboot.fd

# NVRAM storage path
guest.nvram.path=/var/lib/libvirt/qemu/nvram/

Problem Description

When attaching multiple disks to an aarch64 VM, CloudStack incorrectly adds an extra lsilogic SCSI controller after allocating virtio-scsi controllers for every 7 disks. This causes the last disk to be unrecognized after VM reboot.

Key Evidence: The second SCSI controller switches from virtio-scsi to lsilogic after reboot.

Reproduction Steps

  1. Deploy an aarch64 VM with 5 disks → all recognized correctly under virtio-scsi controller
  2. Hot-add a 6th disk while VM running → temporarily works, creates second virtio-scsi controller
  3. Shutdown and restart the VM → second controller becomes lsilogic, 6th disk disappears

Expected Behavior

All disks assigned to virtio-scsi controllers, no lsilogic controller added.

Actual Behavior (with Evidence)

After reboot - WRONG
<controller type='scsi' index='1' model='lsilogic'>
Should be:  <controller type='scsi' index='1' model='virtio-scsi'>

Related Code

PR #9823 (bug at the end of implementation)

Attachments

Step 1: Initial State (5 disks attached)
root@NODE159:/var/log/cloudstack/agent# virsh dumpxml i-2-971-VM  | grep controller
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='5'/>
      <address type='drive' controller='0' bus='0' target='0' unit='6'/>
    <controller type='usb' index='0' model='qemu-xhci'>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
    </controller>
    <controller type='virtio-serial' index='0'>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='7' model='pcie-to-pci-bridge'>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
    </controller>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>

Step 2: Hot-add 6th disk while VM running,witch device id is 7
root@NODE159:/var/log/cloudstack/agent# virsh dumpxml i-2-971-VM  | grep controller
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='5'/>
      <address type='drive' controller='0' bus='0' target='0' unit='6'/>
      <address type='drive' controller='1' bus='0' target='0' unit='0'/>
    <controller type='usb' index='0' model='qemu-xhci'>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
    </controller>
    <controller type='scsi' index='1' model='virtio-scsi'>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
    </controller>
    <controller type='virtio-serial' index='0'>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='7' model='pcie-to-pci-bridge'>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
    </controller>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
Step 3: Reboot VM
root@NODE159:/var/log/cloudstack/agent# virsh dumpxml i-2-971-VM  | grep controller
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='5'/>
      <address type='drive' controller='0' bus='0' target='0' unit='6'/>
      <address type='drive' controller='1' bus='0' target='0' unit='0'/>
    <controller type='usb' index='0' model='qemu-xhci'>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
    </controller>
    <controller type='scsi' index='1' model='lsilogic'>    <!-- BUG: Should be virtio-scsi -->
    </controller>
    <controller type='virtio-serial' index='0'>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='7' model='pcie-to-pci-bridge'>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
    </controller>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
root@NODE159:/var/log/cloudstack/agent#

versions

  • CloudStack Version: 4.20.1.0
  • CPU Architecture: aarch64
  • Hypervisor: KVM

The steps to reproduce the bug

1.Initial State 5 disks attached
2.Hot-add 6th disk while VM running
3.reboot VM
...

Image

What to do about it?

No response

Activity

  1. gw769 commented on Nov 26, 2025

    @gw769
    gw769
    on Nov 26, 2025
    Author

    Hi all,

    I've found a temporary workaround for this issue (see #12090) using CloudStack's libvirt VM XML transformation hook. Here's the properly formatted details:

    Workaround Approach

    Use the libvirt-vm-xml-transformer.groovy hook to force-inject 4 persistent virtio-scsi controllers (index=1~4) into the VM's XML. This prevents the unwanted lsilogic controller from being added after reboot.

    Steps to Apply

    1. Create/Update the Hook File
      Create/overwrite the hook at:

      /etc/cloudstack/agent/hooks/libvirt-vm-xml-transformer.groovy

    2. Add the Workaround Script
      Paste this code (properly wrapped in a Groovy code block):

      import org.apache.logging.log4j.Logger

class VmWwnHook {
    /**
     * Transform XML by injecting required virtio-scsi controllers
     * @param log Logger instance for logging
     * @param xml Original XML string to process
     * @return Modified XML (or original on exception)
     */
    public String transform(Logger log, String xml) {
        try {
            log.info("[VM-WWN-HOOK] Starting: Inject virtio-scsi controllers")
            log.debug("[VM-WWN-HOOK] Original XML:\n${xml}")  
            String modifiedXml = xml
            modifiedXml = injectVirtioScsiControllers(log, modifiedXml)

            log.debug("[VM-WWN-HOOK] Modified XML:\n${modifiedXml}")
            return modifiedXml
        } catch (Exception e) {
            log.error("[VM-WWN-HOOK] Execution error: ${e.getMessage()}", e)
            return xml
        }
    }

    /**
     * Force inject 4 virtio-scsi controllers (index=1~4), skip if exists
     * @param log Logger instance
     * @param xml Original XML string
     * @return XML with virtio-scsi controllers injected
     */
    private String injectVirtioScsiControllers(Logger log, String xml) {
        try {
            log.info("[VM-WWN-HOOK] Injecting virtio-scsi controllers (index=1~4)")
            String modifiedXml = xml
            String controllersToInject = ""

            // Check & inject controllers 1-4
            for (int i = 1; i <= 4; i++) {
                String checkStr = "controller type='scsi' index='${i}'"
                if (!modifiedXml.contains(checkStr)) {
                    controllersToInject += """
<controller type='scsi' index='${i}' model='virtio-scsi'>
  <driver queues='4'/>
</controller>"""
                    log.info("[VM-WWN-HOOK] Injecting missing controller (index=${i})")
                } else {
                    log.info("[VM-WWN-HOOK] Controller (index=${i}) exists - skipping")
                }
            }

            // Inject into XML
            if (!controllersToInject.isEmpty()) {
                def existingScsiMatcher = modifiedXml =~ /(?s)(<controller type='scsi'.*?<\/controller>)/
                if (existingScsiMatcher.find()) {
                    String lastController = existingScsiMatcher[existingScsiMatcher.count - 1][1]
                    modifiedXml = modifiedXml.replace(lastController, lastController + "\n" + controllersToInject)
                    log.info("[VM-WWN-HOOK] Injected after existing controllers")
                } else {
                    modifiedXml = modifiedXml.replace("<devices>", "<devices>\n" + controllersToInject)
                    log.info("[VM-WWN-HOOK] Injected into <devices> tag")
                }
            } else {
                log.info("[VM-WWN-HOOK] All controllers exist - no injection needed")
            }
            return modifiedXml
        } catch (Exception e) {
            log.error("[VM-WWN-HOOK] Injection error: ${e.getMessage()}", e)
            return xml
        }
    }
}

def vmWwnHook = new VmWwnHook()

    Verification

    After applying:

    • Reboot the VM, then run virsh dumpxml <VM_NAME> | grep controller
    • All SCSI controllers will show model='virtio-scsi' (no more lsilogic), and all disks remain recognized.
    root@NODE159:/etc/cloudstack/agent/hooks# virsh dumpxml i-2-971-VM | grep controll
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='5'/>
      <address type='drive' controller='0' bus='0' target='0' unit='6'/>
      <address type='drive' controller='1' bus='0' target='0' unit='0'/>
    <controller type='usb' index='0' model='qemu-xhci'>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
    </controller>
    <controller type='scsi' index='1' model='virtio-scsi'>
    </controller>
    <controller type='scsi' index='2' model='virtio-scsi'>
    </controller>
    <controller type='scsi' index='3' model='virtio-scsi'>
    </controller>
    <controller type='scsi' index='4' model='virtio-scsi'>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
    </controller>
    <controller type='virtio-serial' index='0'>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='10' model='pcie-to-pci-bridge'>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
    </controller>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
  2. DaanHoogland
    added
    status:needs-investigation
    on Nov 26, 2025
  3. DaanHoogland
    assigned
    vishesh92
    on Nov 26, 2025

  4. kiranchavala commented on Dec 23, 2025

    @kiranchavala
    kiranchavala
    on Dec 23, 2025
    Member

    @gw769 Could you please let us know the vm guest os type (windows or linux )

    Also can you provide the root disk controller details of the vm

    Image

  5. gw769 commented on Dec 23, 2025

    @gw769
    gw769
    on Dec 23, 2025
    Author
    Image

    Hi,
    Thanks for checking. Quick details:

    1. Guest OS: Linux (UOS Server 20 1070a, a RHEL-based distro). CloudStack UI shows "Ubuntu 22.04 LTS" because UOS isn't in the dropdown.
    2. Root Disk Controller: SCSI
      Let me know if you need anything else.

  6. kiranchavala commented on Dec 23, 2025

    @kiranchavala
    kiranchavala
    on Dec 23, 2025
    Member

    Thanks @gw769

    cc @DaanHoogland @vishesh92

    Able to reproduce the issue even on a KVM amd64 host

    1. Enable UEFI mode on the kvm host
    2. Create a vm with 5 data disks with dataDiskController as SCSI and rootDiskController as osdefault
    Image Image
    1. Start the vm

    Check the dumpxml on the kvm host

    
<disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/8d2f700c-1483-493b-a4c7-522b3554aaae' index='6'/>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <serial>8d2f700c1483493ba4c7</serial>
      <alias name='scsi0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/97f81648-7abc-4a22-800c-1675b1ef3eec' index='5'/>
      <backingStore/>
      <target dev='sdc' bus='scsi'/>
      <serial>97f816487abc4a22800c</serial>
      <alias name='scsi0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/dd061ab2-ab6d-4d2a-93f4-f368e9c2c62e' index='4'/>
      <backingStore/>
      <target dev='sde' bus='scsi'/>
      <serial>dd061ab2ab6d4d2a93f4</serial>
      <alias name='scsi0-0-4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/0d4a3931-95c2-443a-86a6-64d1087febb7' index='3'/>
      <backingStore/>
      <target dev='sdf' bus='scsi'/>
      <serial>0d4a393195c2443a86a6</serial>
      <alias name='scsi0-0-5'/>
      <address type='drive' controller='0' bus='0' target='0' unit='5'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/6f5b952e-ed5b-49dc-9057-f5ba7064e4fe' index='2'/>
      <backingStore/>
      <target dev='sdg' bus='scsi'/>
      <serial>6f5b952eed5b49dc9057</serial>
      <alias name='scsi0-0-6'/>
      <address type='drive' controller='0' bus='0' target='0' unit='6'/>
    </disk>
    1. Add a sixth data disk and check the dumpxml

    The sixth disk get the bus driver as "virtio"

    
<disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/aa2e55b0-4637-47b8-a0aa-97f0eab903ce' index='9'/>
      <backingStore/>
      <target dev='vdh' bus='virtio'/>
      <serial>aa2e55b0463747b8a0aa</serial>
      <alias name='virtio-disk7'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/8d2f700c-1483-493b-a4c7-522b3554aaae' index='6'/>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <serial>8d2f700c1483493ba4c7</serial>
      <alias name='scsi0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/97f81648-7abc-4a22-800c-1675b1ef3eec' index='5'/>
      <backingStore/>
      <target dev='sdc' bus='scsi'/>
      <serial>97f816487abc4a22800c</serial>
      <alias name='scsi0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/dd061ab2-ab6d-4d2a-93f4-f368e9c2c62e' index='4'/>
      <backingStore/>
      <target dev='sde' bus='scsi'/>
      <serial>dd061ab2ab6d4d2a93f4</serial>
      <alias name='scsi0-0-4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/0d4a3931-95c2-443a-86a6-64d1087febb7' index='3'/>
      <backingStore/>
      <target dev='sdf' bus='scsi'/>
      <serial>0d4a393195c2443a86a6</serial>
      <alias name='scsi0-0-5'/>
      <address type='drive' controller='0' bus='0' target='0' unit='5'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/69c0c442-1b98-3f15-a62d-b371ae1ae4f6/6f5b952e-ed5b-49dc-9057-f5ba7064e4fe' index='2'/>
      <backingStore/>
      <target dev='sdg' bus='scsi'/>
      <serial>6f5b952eed5b49dc9057</serial>
      <alias name='scsi0-0-6'/>
      <address type='drive' controller='0' bus='0' target='0' unit='6'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <alias name='ide0-1-0'/>
      <address type='drive' controller='0' bus='1' target='0' unit='0'/>
    </disk>
  7. kiranchavala
    added
    type:bug
    on Dec 23, 2025
  8. kiranchavala
    added this to Apache CloudStack BugFest - Issueson Dec 23, 2025
  9. DaanHoogland
    moved this to Todo in Apache CloudStack BugFest - Issueson Dec 23, 2025
  10. DaanHoogland
    added this to the 4.20.3 milestone on Dec 23, 2025
  11. DaanHoogland
    added the
    Bug
    issue type on Jan 5, 2026
  12. DaanHoogland
    changed the title [-][CloudStack 4.20.1.0]Extra lsilogic SCSI controller on aarch64 causes last disk unrecognized[/-] [+]Extra lsilogic SCSI controller on aarch64 causes last disk unrecognized[/+] on Jan 6, 2026
  13. shwstppr
    added a commit that references this issue on Jan 16, 2026

    kvm: honour data disk controller for vm during attach volume

    99ca627

  14. shwstppr commented on Jan 16, 2026

    @shwstppr
    shwstppr
    on Jan 16, 2026
    Contributor

    While I created #12452 but I think there could be two issues.
    In @gw769 's case, dataDiskController is not specified for the VM but @kiranchavala reproduced the somewhat similar behaviour with amd64 while specifying a dataDiskController. I'll try to check for aarch64 if #12452 solves the issue

  15. DaanHoogland
    moved this from Todo to Discuss in Apache CloudStack BugFest - Issueson Jan 16, 2026
  16. DaanHoogland
    linked a pull request that will close this issuekvm: honour disk controller for vm during attach volume #12452on Jan 19, 2026

  17. abh1sar commented on Feb 4, 2026

    @abh1sar
    abh1sar
    on Feb 4, 2026
    Contributor

    Fixed by #12452

  18. abh1sar
    closed this as completedon Feb 4, 2026
  19. github-project-automation
    moved this from Discuss to Done in Apache CloudStack BugFest - Issueson Feb 4, 2026
  20. DaanHoogland
    removed this from Apache CloudStack BugFest - Issueson Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

status:needs-investigationtype:bug

Type

Bug

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions