[HyperV2012R2+] The underlying connection was closed

[berkut1]

Describe the bug

The error likely happened due to a temporary connection loss between Hyper-V and the server. Another possible reason is that the node is overloaded (with 400 VM objects), causing PowerShell to struggle. It took 3 minutes just to create a VM object (check logs).

Expected behavior
When an error occurs, there should be an option to retry the creation process, either automatically or manually.

SolidCP Info

• SolidCP Version: 1.5.1 (but probable any)

Additional context

00:03:55 - Creating Virtual Machine object
00:06:39 - Error creating Virtual Machine object: The underlying connection was closed: An unexpected error occurred on a receive.
The underlying connection was closed: An unexpected error occurred on a receive.
at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
at Microsoft.Web.Services3.WebServicesClientProtocol.GetResponse(WebRequest request, IAsyncResult result)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at SolidCP.Providers.Virtualization2012.VirtualizationServer2012.CreateVirtualMachine(VirtualMachine vm)
at SolidCP.EnterpriseServer.Code.Virtualization2012.Tasks.CreateVirtualMachineTask.CreateVirtualMachineInternal(String taskId, VirtualMachine vm, LibraryItem osTemplate, Int32 externalAddressesNumber, Boolean randomExternalAddresses, Int32[] externalAddresses, Int32 privateAddressesNumber, Boolean randomPrivateAddresses, String[] privateAddresses, Int32 dmzAddressesNumber, Boolean randomDmzAddresses, String[] dmzAddresses, String summaryLetterEmail)
00:06:39 - VPS was not created

I see two possible solutions here:

1. Implement the ability to resume an interrupted task from where it stopped. This would require splitting the installation process into steps and storing them in the database. If an issue occurs, the system should attempt to retry the procedure from the last completed step.
2. Rewrite the VM object creation function from PowerShell to WMI, as it is not acceptable for PowerShell to take so much time on a heavily loaded nod

[berkut1]

It looks like the object was created but not fully, which suggests the issue is in this function:

SolidCP/SolidCP/Sources/SolidCP.Providers.Virtualization.HyperV-2012R2/HyperV2012R2.cs

Line 550 in 28ff929

public VirtualMachine UpdateVirtualMachine(VirtualMachine vm)

Specifically, this part:

SolidCP/SolidCP/Sources/SolidCP.Providers.Virtualization.HyperV-2012R2/HyperV2012R2.cs

Line 564 in 28ff929

var realVm = GetVirtualMachineEx(vm.VirtualMachineId);

This function is too resource-intensive. I remember a similar issue in WebsitePanel when they decided to switch from WMI to PowerShell. That’s when the first problems appeared—retrieving VM data took up to 2 minutes. This was eventually fixed by simply adding caching. I still remember it because I created an issue about it back then. :)

If the problem is indeed with PowerShell, switching back to WMI for retrieving VM data could not only partially fix the current issue but also significantly improve the panel’s responsiveness when working with VMs.

[berkut1]

Haha, after fixing this bug: #146 the error The underlying connection was closed started appearing more often. I suspect that this error was actually the original cause of #146 in the first place.

So, the only real solution here is to rewrite GetVirtualMachineEx() using WMI.