Horizon Instant Clone -VM Replica and Template in inaccessible state

In the various maintenance activities of a Horizon infrastructure, it can happen to find VMs of the instant clone chain in an inaccessible state. (Caused by issues on hosts or vCenter such as sudden shutdowns without properly maintaining Horizon Desktop Pools.)

Image that contains text

Auto-generated description

In the VMware Horizon solution there is a tool, from the command line, that allows the cleaning of these objects.

The tool is present in the directory

C:\Program Files\VMware\VMware View\Server\tools\bin>

of one of the connection servers of the Horizon infrastructure

The command is iccleanup.cmd

The first step is to connect to the vCenter in question by launching the following command

iccleanup.cmd -vc <fqdn of vCenter> -uid <administrative user>

Once you have entered the password you will have the possibility to list the VMs of the instant clone infrastructures implemented on that vCenter with the LIST command

Image that contains text

Auto-generated description

Or delete objects in an inaccessible state, for example:

With the delete –index 2 command

Image that contains text

Auto-generated description

After the completion of the cleaning, the situation will be as follows:

Image that contains text

Auto-generated description

Horizon Instant Clone -VM Replica and Template in inaccessible state

Script to see Datastore Permission

Last day in the VMware Community I saw a request for:

“I have AD group like mydomain\mygroup.

This group have access for many datastores.

How i can use powercli to get full list of datastores which the group can manage?”

I made this PowerCLI script:

$cred = Get-Credential
Connect-ViServer <vcenter-FQDN>; -Credential $cred
$datastores = Get-Datastore | Select Name
$groupAD = "domain\group"
$report = @()
foreach ($datastore in $datastores) {
  $report +=  Get-VIPermission
| Where-Object {($_.Entity.Name -Like $datastore.Name) -and ($_.Principal -eq $groupAD)} |Select Principal,Role,@{n='Datastore';E={$datastore.Name}},@{n='Entity';E={$_.Entity.Name}},@{N='Entity Type';E={$_.EntityId.Split('-')[0]}},@{N='vCenter';E={$_.Uid.Split('@:')[1]}}
}
$report | Export-Csv <path\csvfile> -NoTypeInformation

Script to see Datastore Permission

Copy-VMGuestFile

I needed to copy a folder to virtual machine on the DMZ network segment.
The firewall rule blocks any access to VM, well I used the Copy-VMGuestFile Powercli Command.

Connect-VIServer <vcenter server FQDN or IP>

$vm = Get-VM -Name <VM target>

Get-Item “<Source Path>” | Copy-VMGuestFile -Destination “<Destination Path on VM>” -VM $vm -LocalToGuest -GuestUser <User VM Guest> -GuestPassword <Password User VM Guest>

After the first tentative I receive this error:

Copy-VMGuestFile : 09/12/2021 11:08:40 Copy-VMGuestFile The request was aborted: The request was cancelled.
At line:1 char:27

Probably it is a time out error and I try to change the WebOperation Timeout Seconds.

PS C:\Windows\system32> Set-PowerCLIConfiguration -WebOperationTimeoutSeconds -1

Scope ProxyPolicy DefaultVIServerMode InvalidCertificateAction DisplayDeprecationWarnings WebOperationTimeout
Seconds
—– ———– ——————- ———————— ————————– ——————-
Session UseSystemProxy Multiple Unset True -1
User Multiple
AllUsers -1

After the change the error it is resolved

Copy-VMGuestFile

LDAP Identity source and vCenter

Whenever we installed a new vCenter the activity always included integration with Active Directory and normally IWA (Integrated Windows Authentication) was used.
Since vSphere 7.0 version this possibility has been deprecated
so it is good to start with the integration of the vCenter with Active Directory via LDAP.
In our case, we will use LDAPS which uses a certificate

For first the step we need to create the certificate:

  • Use SSH to vCenter connection

On shell use this command

openssl s_client -connect <DC FQDN>:636 -showcerts

Copy the certificate output with  —–BEGIN CERTIFICATE—– and —–END CERTIFICATE—–

Past on Notepad and save with .crt extension

Now we will go to configure the Identity Sources on vCenter:

  • Login as Single Sign-On Administrator to vCenter
  • Navigate to Menu > Administration > Single Sign-On Configuration
  • In the Identity Provider tab, open Identity Sources
  • Click ADD
  • Select Active Directory over LDAP or OpenLDAP, depending on your directory type.

Fill out the remaining fields as follows:
Identity Source Name: Label
Base DN for users: The Distinguished Name (DN) of the starting point for directory server searches. Example: “DC=pollaio,DC=lan”.
Base DN for groups: The Distinguished Name (DN) of the starting point for directory server searches.
Domain name: Your domain name. Example: “pollaio.lan”
Domain alias: Your NetBIOS name. Example: “pollaio.lan”
Username: Domain user with at least browse privileges. Example: “pollaio\administrator”.
Connect to:  “ldaps://<DC FQDN>”.

  • Click Browse next to SSL Certificate
  • Select the .cer file created in before step
Now we are ready to login to the vCenter with domain user (remember to assign the correct permission to domain group or user group)

If you want check the correct use of SSL certificate on the authentication to Active Directory with LDAP connection check the websso.log:

LDAP Identity source and vCenter

Capture Code – vSphere Web Client

One of the conveniences of administering VMware solutions is being able to use code to create scripts to perform repetitive tasks or automate processes

One of the vSphere Web Client features that can help those new to the PowerCli is the Capture Code, it basically allows you to list and save the Powercli commands of the actions you are doing with the vSphere Web Client.

To activate it just access the vSphere Web Client, from the Menu select Developer Center

Select Code Capture and enable it by placing the “Enable Code Capture” flag on the right (which turns green)

At this point, a space will appear in our frame where the commands will be listed with some operations, such as Clear and start another, Copy and Download

Where the Download option generates you the ps1 file with the Powercli commands of the recorded operations

To start and stop a recording session you can use the buttons:

Or the red button that appears at the top of the WebClient once “Enable Code Capture” is enabled

Bye

Capture Code – vSphere Web Client

Ingest your VMware VCSA Appliance logs into Azure Sentinel

In an old post, I described how to send ESXi logs to Azure Log Analytics to ingest at Azure Sentinel, now I describe Step to Step how to send vCenter logs.

The first step is to do step by step this configuration:

After this we need to:
  • Change to the settings of VCSA Appliance to send the logs to Syslog Gateway Server
  • Configure the Log Analytics Agent, installed on Syslog Gateway Server to process the Facility Local0
  • Change la function VMwareESXi (It was created for ESXi Log check my old post) or create a query custom to parse the log on Azure Log Analytics

Change to the settings of VCSA Appliance to send the logs to Syslog Gateway Server

For configuring the VCSA you can use this VMware KB

Forward vCenter Server Log Files to Remote Syslog Server (vmware.com)

Protocol and port depend on your infrastructure configuration (you need to enable communication from VCSA to Syslog Gateway Server on the select TCP/UDP port)

and enable send events (it is enabled by default, but a check is a good idea)

Configure Streaming of Events to a Remote Syslog Server (vmware.com)

Now you can connect to the Syslog Gateway Server and check if the Syslog server received the logs from the VCSA Appliance

Use SSH to connect at the Syslog Gateway Server and use this command

cat /var/log/syslog | grep <fqdn vCenter> | more

in my situation

cat /var/log/syslog | grep vcenter | more

Configure the Log Analytics Agent, installed on Syslog Gateway Server to process the Facility Local0

Connect to Azure Portal and on Azure Log Analytics Service enable the correct facility (local0)

After 10/15 minutes the new configuration will be applied on Syslog Gateway Server (you can check the file /etc/rsyslog.d/95-omsagent.conf on Syslog Gateway)

Change the VMwareESXi function (It was created for ESXi Log check my old post) or create a query custom to parse the log on Azure Log Analytics

Finally, you can query the data on Azure Log Analytics

Syslog | where HostName contains “<FQDN vCenter>”

or optionally you can edit the function create for Ingest ESXi log (check my old POST) and insert the vCenter FQDN Name in the same position where there is the ESXi FQDN Name.

Currently, on Azure Sentinel there are no specific Workbooks for VMware, all queries are to be created

Ingest your VMware VCSA Appliance logs into Azure Sentinel

Check HW VMware Compatibility Matrix

I need to check the Compatibility Matrix for the network IO device of ESXi HOST

Connect to ESXi with SSH and start this command

vmkchdev -l | grep vmnic

and the value are:

If you want check storage IO device change vmnic to vmhba

Determining Network/Storage firmware and driver version in ESXi (1027206) (vmware.com)

Check HW VMware Compatibility Matrix

VMware Horizon and Adobe Flash

I found myself in the need to carry out some checks on a horizon infrastructure that I could not access the administration console due to the now-famous problems of Adobe FLASH So I found it convenient to use the powercli, I report some scripts used. Running the scripts requires installing the necessary components which I have already discussed in a previous post of mine.

Script to show last user login to VMware Horizon in the last month

$connectionServer=Connect-HVServer -Server $hvserver -User $hvuser -Password $hvPassword -Domain $hvDomain

$Services1=$connectionServer.ExtensionData

$eventdb=Connect-HVEvent -DbPassword $eventDbPassword

$events=Get-HVEvent -HvDbServer $eventdb -TimePeriod month -SeverityFilter AUDIT_SUCCESS

 $events.events | Export-Csv C:\temp\VCSMonthLogin.csv

Script to display Horizon Session

Connect-HvServer -server $hvserver -User $hvuser -Password $hvPassword -Domain $hvDomain

$query = New-Object “Vmware.Hv.QueryDefinition”

$query.queryEntityType = ‘SessionLocalSummaryView’

$qSrv = New-Object “Vmware.Hv.QueryServiceService”

$qSRv.QueryService_Query($global:DefaultHVServers[0].ExtensionData,$query) |

Select -ExpandProperty Results |

Select -ExpandProperty NamesData |

Select-Object -Property UserName,DesktopType,DesktopName,MachineOrRDSServerDNS

Script to show user and assigned Computer

Connect-HvServer -server $hvserver -User $hvuser -Password $hvPassword -Domain $hvDomain

$AllVDIInfo = get-hvmachinesummary -PoolName $PoolName

$AllVDIInfo | Format-Table -AutoSize

a special thanks :

Horizon View API – The SLOG – SimonLong/Blog

VMware Horizon and Adobe Flash