Copy file to VCSA with SCP

Well, in recent weeks we have often talked about how to heal vCenters from the log4j vulnerability.
I guess the first thing we all thought was “What a show VMware support released scripts to run to solve the problem …” and then every one to use WinSCP or similar tools/commands to copy the file …. but many will have found it impossible to copy files using the Root user …. but how SSH works but the SCP command does not work!
Well, the problem comes from the shell associated with the Root user. It is not the classic BASH but the APPLIANCESH.
Then we proceed as follows:

  • Let’s connect in SSH to the vCenter Virtual Appliance
  • We access the Bash SHELL with the command SHELL
  • We enable BASH as the default shell for the root user
  • We run our SCP
  • We re-enable APPLIANCESH for the root user

Copy file to VCSA with SCP

Log4j, Horizon Connection Server Workaround

We continue to look at how to mitigate the log4j vulnerability, in this post we look at horizon connection servers in detail.
As indicated by the VMware KB


only the connection servers where the HTML Access Portal is active are vulnerable. But all versions are subject to vulnerability.
I recommend applying the workaround even if the HTML Access Portal is not active.
Again as indicated in the previously cited KB we have two possibilities:

  • Change the following registry key

1. Edit this registry value:
HKLM\Software\VMware, Inc.\VMware VDM\plugins\wsnm\TomcatService\Params\JVMOptions
2. Append a single space character followed by this text: -Dlog4j2.formatMsgNoLookups=true
3. Exit the registry editor and restart the Connection Server service or reboot the machine

  • Run the following script as administrator.
@echo off
setlocal
goto start
__________________________________________________

CVE-2021-44228 - Prevent log4j parameter expansion
Horizon Connection Server 7.x, 8.x
VMware, Inc. 2021
__________________________________________________

:start
set sigpath=HKLM\Software\VMware, Inc.\VMware VDM\plugins\wsnm\TomcatService
for /f "delims=" %%g in ('reg.exe query "%sigpath%" /v Filename') do set sigval=%%g
if "%sigval%"=="" goto notneeded
set killflag=-Dlog4j2.formatMsgNoLookups=true
set svcpath=HKLM\Software\VMware, Inc.\VMware VDM\plugins\wsnm\TomcatService\Params
for /f "tokens=2*" %%v in ('reg.exe query "%svcpath%" /v JVMOptions') do set svcval=%%w
echo %svcval%|find " %killflag%" >nul
if not errorlevel 1 goto notneeded
reg add "%svcpath%" /v JVMOptions /d "%svcval% %killflag%" /f
net stop wsbroker /y && net start wsbroker
echo Completed.
goto :EOF

:notneeded
echo Not required.
goto :EOF

I will proceed with the script.
I create a fix-log4j.bat file in the c: \ temp folder of my connection server and copy the script text to it.


I launch the command from a PowerShell with administrator rights:

I reboot the server

I verify that the workaround is applied by relaunching the bat file.

Obviously, I have to do this on all the Horizon Connection Servers present
in the Horizon infrastructure

Log4j, Horizon Connection Server Workaround

Automate workaround for mitigating Log4j exploit on VCSA (vCenter Virtual Appliance)

After the post where I apply the VMware workaround for mitigating the Log4j exploit on the UAG appliance, now I suggest using this VMware KB to apply the workaround on vCenter.

Python script to automate the workaround steps of VMSA-2021-0028 vulnerability on vCenter Server Appliance (87088) (vmware.com)

The Python script attached at the KB check the vCenter version (6.5, 6.7 and 7) and apply the correct workaround indicate from VMware [see the Workaround instructions to address CVE-2021-44228 in vCenter Server and vCenter Cloud Gateway (87081) (vmware.com)]

Automate workaround for mitigating Log4j exploit on VCSA (vCenter Virtual Appliance)

Exploit Log4j mitigate on VMware Unified Access Gateway

## UPDATE 20/12/2021 ##

On December 14, 2021 the Apache Software Foundation notified the community that their initial guidance for CVE-2021-44228 workarounds was not sufficient. We believed the previous instructions in this article to be an effective mitigation for CVE-2021-44228, but in the best interest of our customers, we must assume the earlier workaround may not adequately address all attack vectors. 

We need to run a script:

#!/bin/bash

# Log contents to file by prefixing timestamp. Maximum file size is 50MB
function log_to_console() {
    echo "$(date +'%Y-%m-%d %T')" "$HOSTNAME" "$@"
}

log_to_console "Running script to remove JndiLookup.class from jars in Unified Access Gateway"

log_to_console "UAG Version: " $(tail -1 /opt/vmware/gateway/logs/version.info 2>/dev/null)

mkdir /tmp/test
mkdir /tmp/bkp

log_to_console "Unpacking archive and removing JndiLookup.class"
cp /opt/vmware/gateway/lib/ab-frontend-0.2.jar /tmp/bkp

unzip -q -o /opt/vmware/gateway/lib/ab-frontend-0.2.jar -d /tmp/test

unzip -q -o /tmp/test/hc.war -d /tmp/test/hc

zip -dq /tmp/test/hc/WEB-INF/lib/log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

rm /tmp/test/hc.war
cd /tmp/test/hc

zip -r -q ../hc.war .

cd ..
rm -rf hc

log_to_console "Repackaging archive"

zip -r -q ab-frontend-0.2.jar .

chown gateway:users ab-frontend-0.2.jar
mv ab-frontend-0.2.jar /opt/vmware/gateway/lib

log_to_console "Replaced updated ab-frontend-0.2.jar, now looking for jndi in other places"

find / -type f \( -name "*.jar" -o -name *.war \) -exec sh -c "zipinfo -1 {} 2>/dev/null | grep 'JndiLookup.class' && echo {}" \; | grep .jar | while read -r line ; do
    jar_path=$line
    log_to_console "Updating $jar_path"
    zip -dq $jar_path org/apache/logging/log4j/core/lookup/JndiLookup.class
    chown gateway:users $jar_path
done


log_to_console "Restarting authbroker"
supervisorctl restart authbroker

log_to_console "Cleaning up."
cd /tmp
rm -rf /tmp/test

log_to_console "Verification: We are good if no jars are listed below"
find / -type f \( -name "*.jar" -o -name *.war \) -exec sh -c "zipinfo -1 {} 2>/dev/null | grep 'JndiLookup.class' && echo {}" \;

log_to_console "Verification: Grep authbroker-std-out.log for log4j errors, we are good if no exception is displayed below"
cat /opt/vmware/gateway/logs/authbroker-std-out.log | grep log4j

log_to_console "Done!"

Well we need to connect to UAG and create a uag_rm_log4j_jndilookup.sh file

vi uag_rm_log4j_jndilookup.sh

copy into the file the code, and enable it for execution

chmod +x uag_rm_log4j_jndilookup.sh

running the script

./uag_rm_log4j_jndilookup.sh

now if the UAG version is between 2009 and 2111 it is also necessary to set the -Dlog4j2.formatMsgNoLookups=true option on the authbroker service with the following commands. Note the space between “s/java /java”  and a space after “true /” in the command, these are important to ensure the command works correctly and doesn’t attempt to modify the wrong lines in the configuration file.

sed -i ‘s/java /java -Dlog4j2.formatMsgNoLookups=true /’ /opt/vmware/gateway/supervisor/conf/authbroker.ini

and restart the supervisorctl

supervisorctl update

more info in this VMware KB

Mitigation instructions to address CVE-2021-44228 and CVE-2021-45046 in VMware Unified Access Gateway (UAG) (87092)

#########################

In the middle of December month, we found a “little exploit”……

Ok it is not a joke, for mitigate on UAG (Unified Access Gateway that is a Security Server exposed on the Internet for remote access at Horizon infrastructure) it is necessary (To apply the workaround for CVE-2021-44228 to Unified Access Gateway version 2009 through to 2111):

  • Connect to UAG server with SSH Session

Check if SSH is enabled on UAG server to accept root connection.

Connect from WEB console or VMware Remote Console to UAG virtual appliance and modify in /etc/ssh/sshd_config the following line (for modify use vi commands):

PermitRootLogin no

to

PermitRootLogin yes

Save the file

Restart SSHD service with this command:

service sshd restart

now you are able to create an SSH connection to UAG server, REMEMBER TO DISABLE SSH CONNECTION FOR ROOT USER WITH ROLLBACK THE SETTING INTO SSHD_CONFIG FILE

  • Append the fix -Dlog4j2.formatMsgNoLookups=true

Type this command:

sed -i ‘s/java /java -Dlog4j2.formatMsgNoLookups=true /’ /opt/vmware/gateway/supervisor/conf/authbroker.ini

Reload the service

supervisorctl update

Check if the fix is applied with this command:

ps -ef | grep ab-frontend

the output of the command should need:

root@viuag03 [ ~ ]# ps -ef | grep ab-frontend
gateway 2799 849 99 09:51 ? 00:00:12 /usr/lib/jvm/zre-8/bin/java -Dlog4j2.formatMsgNoLookups=true -Dfile.encoding=UTF8 -Dport=8877 -Dlog4j.configuration=file:/opt/vmware/gateway/conf/log4j-authbroker.properties -Dspring.profiles.active=accesspoint -jar /opt/vmware/gateway/lib/ab-frontend-0.2.jar
root 2817 2622 0 09:51 pts/0 00:00:00 grep –color=auto ab-frontend

Ref: Workaround instructions to address CVE-2021-44228 in VMware Unified Access Gateway (87092)

Exploit Log4j mitigate on VMware Unified Access Gateway

VMware, CVE-2021-44228 and log4j (version 2)

Since the end of last week, a new critical vulnerability has spread, present in many programs (Many VMware applications use this Java Logging including vCenter, Horizon, etc.)

“an exploit in the popular Java logging library log4j (version 2) was discovered that results in Remote Code Execution (RCE) by logging a certain string.” as indicated in this link:

https://www.lunasec.io/docs/blog/log4j-zero-day/

After a few hours, the CVE also released its bulletin (CVE-2021-44228).

https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-44228

On the day of 10/12/2021, VMware released its security advisor (VMSA-2021-0028) with the workarounds to limit the vulnerability pending the release of patched versions to fix it:

https://www.vmware.com/security/advisories/VMSA-2021-0028.html

So good application everyone!

By the way, for the uninitiated:

What is CVE? CVE, short for Common Vulnerabilities and Exposures, is a list of publicly disclosed computer security flaws. When someone refers to a CVE, they mean a security flaw that’s been assigned a CVE ID number.
Security advisories issued by vendors and researchers almost always mention at least one CVE ID. CVEs help IT professionals coordinate their efforts to prioritize and address these vulnerabilities to make computer systems more secure.

VMware, CVE-2021-44228 and log4j (version 2)

Copy-VMGuestFile

I needed to copy a folder to virtual machine on the DMZ network segment.
The firewall rule blocks any access to VM, well I used the Copy-VMGuestFile Powercli Command.

Connect-VIServer <vcenter server FQDN or IP>

$vm = Get-VM -Name <VM target>

Get-Item “<Source Path>” | Copy-VMGuestFile -Destination “<Destination Path on VM>” -VM $vm -LocalToGuest -GuestUser <User VM Guest> -GuestPassword <Password User VM Guest>

After the first tentative I receive this error:

Copy-VMGuestFile : 09/12/2021 11:08:40 Copy-VMGuestFile The request was aborted: The request was cancelled.
At line:1 char:27

Probably it is a time out error and I try to change the WebOperation Timeout Seconds.

PS C:\Windows\system32> Set-PowerCLIConfiguration -WebOperationTimeoutSeconds -1

Scope ProxyPolicy DefaultVIServerMode InvalidCertificateAction DisplayDeprecationWarnings WebOperationTimeout
Seconds
—– ———– ——————- ———————— ————————– ——————-
Session UseSystemProxy Multiple Unset True -1
User Multiple
AllUsers -1

After the change the error it is resolved

Copy-VMGuestFile