VMware Horizon takes a long time to provision the Desktop virtual machines
We detected a strange situation when changing the sizing (number of desktop VMs) or publishing a new image on the Instant clone Desktop Pool.
The highlighted situation is a very long time in creating one or more VMs from the Gold Image. Following investigation we found that the problem is also present when cloning a VM that is present in the same vSphere environment where the instant clone VDIs are allocated.
In our case it was a vSAN environment, having carried out the first routine checks where no network, disk or compatibility problems were found, we went into the details of the logs and in the case of the clone we found this error message in the logs of the VM that was being cloned.
We have found a workaround and a permanent resolution:
Attenzione è una mia valutazione….quindi non sparate sul sistemista
Broadcom ha acquisito VMware, ormai lo sappiamo tutti. La situazione di incertezza aleggia ovunque soprattutto sul mondo vSphere e sui costi (con tanti competitor che provano a ritagliarsi la loro fetta di mercato togliendole al leader indiscusso di questi anni)
Finalmente in questi giorni incomincio ad avere i primi dati effettivi (Prezzi ecc…) su cui iniziare a fare i primi ragionamenti. !!Attenzione non voglio dare giudizi ma voglio solo paragonare due offerte fatte allo stesso cliente che abbiamo dovuto rivedere a seguito del nuovo listino (E parliamo di prezzi di listino.. senza eventuali scontistiche)!!
Ragioniamo su un cluster vSphere con 3 nodi da 2 processori ciascuno da 16 core.
Con le precedenti licenze e il vecchio listino nello scenaro ipotizzato dovevamo considerare:
Licenza VMWARE VCENTER SERVER 8 STANDARD
Licenza VMWARE VSPHERE 8 STANDARD FOR 1 PROCESSOR
Support/Subscription
Support/Subscription
Con il nuovo listino e le nuove tipologie di licenze invece dobbiamo considerare:
VMWARE VSPHERE STANDARD per core (Che comprende la licenza di vCenter)
Entrambe le soluzioni con 5 anni.
Da una prima analisi le prime valutazioni sono: Con il nuovo listino si viene a pagare circa 30-35% in meno. Ho una semplificazione nella quotazione (una sola voce rispetto alle 4 precedenti)
Ovviamente:
Non abbiamo le licenze perpetue (comunque chi non vuole il supporto sul proprio ambiente di produzione o la possibilità di effettuare aggiornamenti?)
é una prima offerta e la quotazione può dipendere da vari fattori e i prezzi potrebbero nuovamente cambiare
Le funzionalità all’interno dei bundle possono essere leggermente differenti (il link per vedere le funzionalità presenti nei nuovi bundle VMware vSphere® Product Line Comparison)
Posso aver sbagliato i calcoli 🙂
Possono avermi dato dei prezzi sbagliati 🙂 spero di no per il cliente 🙂
ma aspettavo di avere due informazioni reali per fare le mie prime considerazioni.
L’unica cosa che posso dire è di valutare con attenzione il cambio …. (io sono il primo che accetta nuove sfide..) ma attenzione a tutti i prezzi nascosti e valutate bene!
P.S. se qualcuno ha delle esperienze in merito … condividiamole.
In recent days, a customer reported an anomaly on an HPE SimpliVity cluster hosting instant clone Horizon VDIs. In detail:
vSphere with seven hosts present, two were always at 98% CPU utilization and 90% RAM utilization.
Continuous vMotion generated by the VM DRS to and from those two HOSTS.
After a careful analysis, we identified that there were no problems at the vSphere infrastructure level. The issue was due to a Simplivity feature called IWO.
By disabling IWO and keeping DRS active (Full automatic) I have an optimal balance of CPU and RAM load between hosts at the expense of a slight increase in I/O trip times
Scenario – Even VM Load Distribution
I want even VM load across my cluster in terms of CPU and memory. Data locality and I/O performance are not top priorities. Most applications are CPU and memory intensive, and adding 1ms to 2ms to I/O trip times will not impact application performance.
In this scenario, IWO can be disabled thus ensuring no DRS affinity rules are populated into vCenter server. Suppressing DRS affinity rules will allow VMware DRS or allow you to directly distribute VMs across the cluster as desired to ensure all VMs are adequately resourced in terms of CPU and memory. The ‘Data Access Not Optimized’ alarm can be suppressed within vCenter server.
For us VMware systems engineers who every day find ourselves “dialoguing” with those who manage the network ecosystem, we can only find the vSphere Distributed Switch health check function useful.
What these checks allow us to highlight:
These are some of the common configuration errors that health check identifies:
Mismatched VLAN trunks between a vSphere distributed switch and a physical switch.
Mismatched MTU settings between physical network adapters, distributed switches, and physical switch ports.
Mismatched virtual switch teaming policies for the physical switch port-channel settings.
The network health check in vSphere monitors the following three network parameters at regular intervals:
VLAN: Checks whether vSphere distributed switch VLAN settings match trunk port configuration on the adjacent physical switch ports.
MTU: Checks whether the physical access switch port MTU setting based on per VLAN matches the vSphere distributed switch MTU setting.
Network adapter teaming: Checks whether the physical access switch ports EtherChannel setting matches the distributed switch distributed port group IP Hash teaming policy settings.
How to activate:
Access the network section of our vCenter
Select the vDS on which we want to activate health checks
And enable the check that interests us:
Where to check the outcome of the checks?
Wait a few minutes and already first feedback we can have it on ESXi hosts using the vDS in question, where if there are problems the classic red dot will be displayed
For more details, access the network section of our vCenter and select the vDS in question
And we can see that on the vmnic0 and vmnic3 of the first host, there are vLANs of which we have a Portgroup but which are not proposed correctly on all the ports of the switches to which we have attested our hosts. Then we have to have the configuration verified by our colleagues in the network.
How to turn it off:
Repeat the enabling steps but this time select disable.
Risks in activating it (we always consider activating it for a short time)
Depending on the options that you select, the vSphere Distributed Switch Health Check can generate a significant number of MAC addresses for testing teaming policy, MTU size, vLAN configuration, resulting in extra network traffic.
Ensure the number of MAC addresses to be generated by the health check will be less than the size of the physical switch(es) MAC table. Otherwise, there is a risk that the switches will run out of memory, with subsequent network connectivity failures. After you disable vSphere Distributed Switch Health Check, the generated MAC addresses age out of your physical network environment according to your network policy.
Copy and paste operations between the guest operating system and remote console are deactivated by default.
To enable it:
Browse to the virtual machine in the vSphere Client inventory
Right-click the virtual machine and click Edit Settings.
Select Advanced Parameters.
Add or edit the following parameters.
isolation.tools.copy.disable False isolation.tools.paste.disable False isolation.tools.setGUIOptions.enable True These options override any settings made in the guest operating system’s VMware Tools control panel.
Click OK.
(Optional) If you made changes to the configuration parameters, restart the virtual machine.
If you see such an error on the Cluster object of a vSAN (in my case it appeared on two vSAN clusters managed by the same vCenter)
vSphere DRS functionality was impacted due to an unhealthy state vSphere Cluster Service …….
an unhealthy state of the Service cluster
Errors such as the following in the EAM log. vCenter LOG
EAM.log:
2023-01-26T13:16:39.996Z | INFO | vim-monitor | VcListener.java | 131 | Retrying in 10 sec.
2023-01-26T13:16:41.432Z | ERROR | vlsi | DispatcherImpl.java | 468 | Internal server error during dispatch
com.vmware.vim.binding.eam.fault.EamServiceNotInitialized: EAM is still loading from database. Please try again later.
at com.vmware.eam.vmomi.EAMInitRequestFilter.handleBody(EAMInitRequestFilter.java:57) ~[eam-server.jar:?]
at com.vmware.vim.vmomi.server.impl.DispatcherImpl$SingleRequestDispatcher.handleBody(DispatcherImpl.java:373) [vlsi-server.jar:?]
at com.vmware.vim.vmomi.server.impl.DispatcherImpl$SingleRequestDispatcher.dispatch(DispatcherImpl.java:290) [vlsi-server.jar:?]
at com.vmware.vim.vmomi.server.impl.DispatcherImpl.dispatch(DispatcherImpl.java:246) [vlsi-server.jar:?]
at com.vmware.vim.vmomi.server.http.impl.CorrelationDispatcherTask.run(CorrelationDispatcherTask.java:58) [vlsi-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_345]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_345]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_345]
2023-01-26T13:16:50.007Z | INFO | vim-monitor | ExtensionSessionRenewer.java | 190 | [Retry:Login:com.vmware.vim.eam:b55a7f93b59f0f7e] Re-login to vCenter because method: currentTime of managed object: null::ServiceInstance:ServiceInstance failed due to expired client session: null
2023-01-26T13:16:50.007Z | INFO | vim-monitor | OpId.java | 37 | [vim:loginExtensionByCertificate:913aec585658e328] created from [Retry:Login:com.vmware.vim.eam:b55a7f93b59f0f7e]
2023-01-26T13:16:51.440Z | ERROR | vlsi | DispatcherImpl.java | 468 | Internal server error during dispatch
com.vmware.vim.binding.eam.fault.EamServiceNotInitialized: EAM is still loading from database. Please try again later.
And you see the lack of vCLS VMs in the two vSANs
To resolve the anomaly you must proceed as follows:
vCenter Snapshots and Backup
Log in to the vCenter Server Appliance using SSH.
Run this command to enable access the Bash shell:
shell.set --enabled true
Type shell and press Enter.
Run this command to retrieve the vpxd-extension solution user certificate and key:
Note: If this produces the error “Hostname mismatch, certificate is not valid for ‘localhost'”, change ‘localhost’ to the FQDN or IP of the vCenter. The process is checking this value against the SAN entries of the certificate.
Note: The default user and domain is Administrator@vsphere.local. If this was changed during configuration, change the domain to match your environment. When prompted, type in the Administrator@domain.local password.
Restart EAM and start the rest of the services with these commands:
Well, in recent weeks we have often talked about how to heal vCenters from the log4j vulnerability. I guess the first thing we all thought was “What a show VMware support released scripts to run to solve the problem …” and then every one to use WinSCP or similar tools/commands to copy the file …. but many will have found it impossible to copy files using the Root user …. but how SSH works but the SCP command does not work! Well, the problem comes from the shell associated with the Root user. It is not the classic BASH but the APPLIANCESH. Then we proceed as follows:
Let’s connect in SSH to the vCenter Virtual Appliance
We access the Bash SHELL with the command SHELL
We enable BASH as the default shell for the root user
Whenever we installed a new vCenter the activity always included integration with Active Directory and normally IWA (Integrated Windows Authentication) was used. Since vSphere 7.0 version this possibility has been deprecated so it is good to start with the integration of the vCenter with Active Directory via LDAP. In our case, we will use LDAPS which uses a certificate
For first the step we need to create the certificate:
Copy the certificate output with —–BEGIN CERTIFICATE—– and —–END CERTIFICATE—–
Past on Notepad and save with .crt extension
Now we will go to configure the Identity Sources on vCenter:
Login as Single Sign-On Administrator to vCenter
Navigate to Menu > Administration > Single Sign-On > Configuration
In the Identity Provider tab, open Identity Sources
Click ADD
Select Active Directory over LDAP or OpenLDAP, depending on your directory type.
Fill out the remaining fields as follows: Identity Source Name: Label Base DN for users: The Distinguished Name (DN) of the starting point for directory server searches. Example: “DC=pollaio,DC=lan”. Base DN for groups: The Distinguished Name (DN) of the starting point for directory server searches. Domain name: Your domain name. Example: “pollaio.lan” Domain alias: Your NetBIOS name. Example: “pollaio.lan” Username: Domain user with at least browse privileges. Example: “pollaio\administrator”. Connect to: “ldaps://<DC FQDN>”.
Click Browse next to SSL Certificate
Select the .cer file created in before step
Now we are ready to login to the vCenter with domain user (remember to assign the correct permission to domain group or user group)
If you want check the correct use of SSL certificate on the authentication to Active Directory with LDAP connection check the websso.log:
VMware has had a product for a while now called VMware Skyline that provides proactive monitoring, analysis, and support for your VMware environment. It monitors your VMware installation and will notify you when issues arise.
Skyline Advisor will be available to customers and partners with active Production and Premier Support, VMware Success 360 and vRealize Cloud Universal subscriptions at no additional cost.
Create a Cloud Services Organization
Login with My VMware account associate to Production and Premier Support on the site:
https://console.cloud.vmware.com/
After clicking Get Started, a new web browser page, or tab, will open. You will be asked to sign-in
with your VMware account. If you have an existing My VMware account, you can use those same
account details (email address/password) to sign in to Cloud Services.
If you are existing VMware Cloud Services customer, you can choose an existing Cloud Services
Organization for Skyline. If you have never used VMware Cloud Services, click Create New
Organization.
Enter a Organization Name.
Name your Organization something meaningful, that can be easily
referenced by both you, and VMware. For example, name your Organization after you Company, or
Business name. You can also append a line-of-business, division, or team, to the end of your
Company or Business name.
The following are example Organization Names:
The company, LOB, Company LOB, Company-vSphere, Company-Desktop
Enter an Address for your Organization.
Click Add Address. You can also choose an existing
address if one was found for your account. If you choose an existing address, skip to substep f.
During the creation of your Cloud Services Organization, your country currency, and Tax ID, may be
displayed. The displaying of this information is a construct of Cloud Services. Skyline is available at
no additional cost, and you will not be required to enter any payment details while adopting Skyline.
Select a Country from the drop-down menu.
Enter your street address on Address Line 1, and Address Line 2 (optional).
Enter your City.
Enter your State/Province.
Enter your Zip/Postal Code.
Review the Cloud Services Terms of Service. Click the checkbox to agree to the Terms of Service.
Click Continue.
Now on service, we have Skyline Advisor, click on this service
Link the Entitlement
Now, after clicking on LINK, we have the correct status LINKED
Deploy Skyline Collector and configure the connection to Cloud Services
Copy and paste the token on Skyline Connect and register it.
Add source Data to Skyline Advisor
After complete the step 5 and 6 (I suggest to enable the auto-upgrade), we can access to skyline collector to configure the connection to vCenter (or multiple vCenter)
One of the conveniences of administering VMware solutions is being able to use code to create scripts to perform repetitive tasks or automate processes
One of the vSphere Web Client features that can help those new to the PowerCli is the Capture Code, it basically allows you to list and save the Powercli commands of the actions you are doing with the vSphere Web Client.
To activate it just access the vSphere Web Client, from the Menu select Developer Center
Select Code Capture and enable it by placing the “Enable Code Capture” flag on the right (which turns green)
At this point, a space will appear in our frame where the commands will be listed with some operations, such as Clear and start another, Copy and Download
Where the Download option generates you the ps1 file with the Powercli commands of the recorded operations
To start and stop a recording session you can use the buttons:
Or the red button that appears at the top of the WebClient once “Enable Code Capture” is enabled
