Security Advisory - log4j Remote Code Execution (CVE-2021-44228)
Resolved
The infrastructure upgrades for all IaaS, DaaS and Internal Management vCenters have been completed. Upon first notice our Security Operations (SOC), Cloud Infrastructure and Engineering teams have conducted a thorough investigation of any potentially affected vendors and systems. Any known required remediation steps have been completed. As vendors release additional updated versions this will be worked into our standard upgrade cycles. If you have any questions, please email support@greenclouddefense.com
Posted Feb 18, 2022 - 17:50 EST
Update
VMware recently released update 3Q for vCenter Server 6.7 which permanently resolves the Log4J vulnerabilities. The temporary workaround remediation steps have been in place since 12/12/2021. Beginning tonight, Tuesday, February 15th, and continuing through Saturday, February 19th at 6 am, Green Cloud Defense will be performing the necessary infrastructure patching required to remediate this vulnerability between the hours of midnight and 6 am. Patching of private cloud environments will be individually scheduled through proactively opened tickets. There is no expected impact to running VMs during this process, but administrative access via vCloud or vSphere for private cloud will be impacted during service restart intervals. If you have any questions, please email support@greenclouddefense.com
Posted Feb 15, 2022 - 15:40 EST
Update
VMware has provided a hotfix for Horizon DaaS 8 tenants that has been successfully tested by Green Cloud Defense. This hotfix is in replacement for the temporary workarounds that were put in place on December 12th. In order to implement this hotfix, the tenant appliances services will need to be restarted for each appliance. We will be beginning that activity between midnight and 6:00 am EST tonight. Any DaaS users in ATL or HOU that are connected when the services are restarted may disconnect but should be able to reconnect within a few minutes. If you have any questions please support@greenclouddefense.com.
Posted Dec 21, 2021 - 17:43 EST
Update
The VMware remediation steps have been completed across all NSX-V manager components including the Atlanta datacenter. We will continue updating this advisory as relevant information is made available and maintenance is completed. If you have any questions, please email support@greenclouddefense.com.
Posted Dec 17, 2021 - 04:04 EST
Update
The VMware remediation steps have been completed across all NSX-V manager components outside of the Atlanta datacenter. Green Cloud Defense is continuing to work with VMware to complete the required changes and will be temporarily suspending administrative access through vCloud in Atlanta starting at 6:00pm EST. There is no expected impact to NSX Edge Gateways during this process, but administrative tasks and console access via vCloud will be temporarily unavailable during the maintenance. We will continue updating this advisory as relevant information is made available and maintenance is completed. If you have any questions, please email support@greenclouddefense.com.
Posted Dec 16, 2021 - 16:59 EST
Update
VMware has updated the VMSA-2021-0028 advisory to include NSX-V manager components and released temporary remediation steps. This advisory does not affect customer Edge Gateways themselves but the backend components that manage the firewalls. The internal NSX-V manager components are protected by our SIEM, EDR, and Continuous Risk Scanning platforms and are in no way externally exposed. Green Cloud Defense is starting on this remediation now. There is no expected impact to NSX Edge Gateways during this process, but administrative tasks via vCloud on Edge Gateways may be temporarily unavailable during the maintenance. No other type of firewall or vCloud administration tasks will be affected. We will continue updating this advisory as relevant information is made available and maintenance is completed. If you have any questions, please email support@greenclouddefense.com.
Posted Dec 15, 2021 - 14:13 EST
Update
The temporary remediation steps for vCenter have been implemented across our platforms including private cloud. We will continue updating this advisory as relevant information is made available. In the meantime, if you have any questions, please email support@greenclouddefense.com.
Posted Dec 14, 2021 - 14:51 EST
Update
Green Cloud Defense is starting on temporary remediation steps provided by VMware for private cloud environments. There is no expected impact to running VMs during this process, but access via vSphere will be impacted briefly during service restart intervals. We will continue updating this advisory as relevant information is made available and maintenance is completed. Additional If you have any questions, please email support@greenclouddefense.com.
Posted Dec 13, 2021 - 14:30 EST
Update
Green Cloud Defense has concluded implementing the temporary workaround across the affected external facing Horizon DaaS components. This workaround will be superseded by VMware’s official recommendation once that is made available. We would recommend partners test logins to their DaaS environments and if possible only allow traffic to DaaS from known good source IPs. If you have any questions please email support@greenclouddefense.com.
Posted Dec 12, 2021 - 19:46 EST
Update
Green Cloud Defense has identified a temporary workaround for the most critical Horizon DaaS components affecting end customer DaaS environments. This workaround should reduce the risk of exploitation from external sources while we wait for guidance from VMware. We believe that there may be additional steps needed to fully remediate this vulnerability. In order to implement this workaround, the tenant appliances services will need to be restarted for each appliance. We will be beginning that activity immediately. Any users that are connected when the services are restarted will disconnect but should be able to reconnect within a few minutes. If you have any questions please support@greenclouddefense.com.
Posted Dec 12, 2021 - 15:57 EST
Update
The temporary remediation steps for vCenter have been implemented across our IaaS and DaaS platforms. We will continue updating this advisory as relevant information is made available. In the meantime, if you have any questions, please email support@greenclouddefense.com.
Posted Dec 12, 2021 - 00:01 EST
Update
VMware has updated the VMSA-2021-0028 advisory with temporary remediation steps for vCenter. Given the severity of the CVE, Green Cloud Defense is immediately starting on this remediation across our IaaS and DaaS platforms. There is no expected impact to running VMs during this process, but administrative tasks via vCloud will be impacted briefly during service restart intervals. We will continue updating this advisory as relevant information is made available and maintenance is completed. Additional If you have any questions, please email support@greenclouddefense.com.
Posted Dec 11, 2021 - 17:19 EST
Identified
Green Cloud was notified today of the possibility of exploit for the log4j vulnerability detailed in this CVE: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228. Our Security Operations (SOC), Cloud Infrastructure and Engineering teams immediately began our incident management processes and started to assess potential impact. We reached out to our vendors to determine which may be vulnerable and conducted our own internal investigation. We believe we have a fairly good idea at this time of what components may be affected.
To summarize, VMware has issued an advisory that notes vCenter and Horizon DaaS components as being potentially exploitable while VMware Cloud Director, VMware Cloud Director Availability and NSX-V are not. (https://www.vmware.com/security/advisories/VMSA-2021-0028.html) We will be waiting for VMware to issue patches or guidance on workarounds and implementing them as quickly as we can. Our Portal does not use the vulnerable log4j package and the rest of our critical vendors have either confirmed they are not vulnerable or will be issuing guidance as soon as they can confirm. Of particular note, both Veeam and Zerto have confirmed they do not use the vulnerable code or anything related. We foresee no impact to any Green Cloud backup or DR services and we are taking every precaution to make sure this continues to be the case. As always, protecting your data is our top priority.
Our SOC team has implemented specific monitoring and alerting for any possible exploits using our SIEM, EDR and Continuous Risk Scanning platforms and we are confident we can take appropriate action should we be affected. Due to the commonality of this component in applications across all industries, this is going to remain a fluid and ever-evolving situation for quite some time and we will be re-assessing our approach and protections regularly. We will be updating this advisory as relevant information is made available.
As always, please do not hesitate to reach out to support@greenclouddefense.com if you have questions or concerns.
Posted Dec 10, 2021 - 20:53 EST
This incident affected: Network (Network - Greenville, SC, Network - Nashville, TN, Network - Houston, TX, Network - Atlanta, GA, Network - Phoenix, AZ, Network - Minneapolis, MN), DaaS (DaaS - Greenville, SC, DaaS - Nashville, TN, DaaS - Atlanta, GA, DaaS - Houston, TX), BaaS (BaaS with Veeam - Greenville, SC, BaaS with Veeam - Nashville, TN, BaaS with Veeam - Atlanta, GA, BaaS with Veeam - Phoenix, AZ), DRaaS (DRaaS with Zerto - Atlanta, GA, DRaaS with Zerto - Greenville, SC, DRaaS with Zerto - Nashville, TN, DRaaS with Zerto - Houston, TX, DRaaS with Zerto - Minneapolis, MN, DRaaS with Zerto - Phoenix, AZ, DRaaS with StorageCraft - Greenville, SC, DRaaS with StorageCraft - Nashville, TN, DRaaS with StorageCraft - Atlanta, GA, DRaaS with StorageCraft - Phoenix, AZ, DRaaS with Veeam - Atlanta, GA), Office 365 BaaS (Office 365 BaaS - Atlanta, GA), BaaS with Ransomware Protection (BaaS with Ransomware Protection - West-1, BaaS with Ransomware Protection - East-1), IaaS (IaaS - Nashville, TN, IaaS - Greenville, SC, IaaS - Houston, TX, IaaS - Atlanta, GA, IaaS - Phoenix, AZ, IaaS - Minneapolis, MN), Managed Security (Managed SIEM), and Security.