HOL-2042-01-NET - Secure Data Center Endpoints with VMware AppDefense
Note: There are only two modules in this lab.The expected time of completion is 45-60 minutes.
The Table of Contents can be accessed in the upper right-hand corner of the Lab Manual.
VMware AppDefense is a data center endpoint security product that protects applications running in virtualized environments. Rather than chasing after threats, AppDefense understands an application's intended state and behavior, then monitors for changes to that intended state that indicate a threat. When a threat is detected, AppDefense automatically responds. This maximizes efficiency and effectiveness in Security Operations. It also streamlines the application security readiness review process.
Lab Module List:
Lab Captains & Support:
This lab would not have been possible without the dedication of the AppDefense engineering team.Their support and assistance to make this something suitable for the VMworld HOL environment was instrumental. We would like to thank the following members:
This lab manual can be downloaded from the Hands-on Labs Document site found here: http://docs.hol.vmware.com
This lab may be available in other languages. To set your language preference and have a localized manual deployed with your lab, you may utilize this document to help guide you through the process:
During this module, you will input text into the Main Console. Besides directly typing it in, there are two very helpful methods of entering data which make it easier to enter complex data.
You can also click and drag text and Command Line Interface (CLI) commands directly from the Lab Manual into the active window in the Main Console.
You can also use the Online International Keyboard found in the Main Console.
In this example, you will use the Online Keyboard to enter the "@" sign used in email addresses. The "@" sign is Shift-2 on US keyboard layouts.
Notice the @ sign entered in the active console window.
When you first start your lab, you may notice a watermark on the desktop indicating that Windows is not activated.
One of the major benefits of virtualization is that virtual machines can be moved and run on any platform. The Hands-on Labs utilizes this benefit and we are able to run the labs out of multiple datacenters. However, these datacenters may not have identical processors, which triggers a Microsoft activation check through the Internet.
Rest assured, VMware and the Hands-on Labs are in full compliance with Microsoft licensing requirements. The lab that you are using is a self-contained pod and does not have full access to the Internet, which is required for Windows to verify the activation. Without full access to the Internet, this automated process fails and you see this watermark.
This cosmetic issue has no effect on your lab.
Please check to see that your lab is finished all the startup routines and is ready for you to start. If you see anything other than "Ready", please wait a few minutes. If after 5 minutes your lab has not changed to "Ready", please ask for assistance.
Module 1 - Overview of VMware AppDefense (15 minutes)
In this section, you will read about VMware's new AppDefense security platform.
AppDefense is a data center endpoint security product that embeds threat detection and response into the virtualization layer regarding applications and data live. Leveraging VMware AppDefense delivers three key advantages over existing endpoint security solutions:
Authoritative knowledge of application intended state When you know whats good, you can detect whats bad.
From inside the vSphere hypervisor, AppDefense has an authoritative understanding of how data center endpoints are meant to behave and is the first to know when changes are made. This contextual intelligence removes the guesswork involved in determining which changes are legitimate and which are real threats. AppDefense does not look at a guest workload in isolation. Instead, it manages workloads as part of broader Security Scopes. These scopes allow AppDefense to have a deeper understanding of complex interactive behaviour patterns in the data center as opposed to simply individual machine behaviour.
Automated, precise threat response The right response at the right time. When a threat is detected, AppDefense can trigger vSphere and VMware NSX to orchestrate the correct response to the threat, without the need for manual intervention. For example, AppDefense can automatically:
Isolation from the attack surface - Protect the protector. The first thing that most malware variants do when they reach an endpoint is disable anti-virus and other agent-based endpoint security solutions. The hypervisor provides a protected location from which AppDefense can operate, ensuring that even if an endpoint is compromised, AppDefense itself is protected.
AppDefense is a foundational security product that has a wide-reaching impact on an organizations security strategy.
Application-centric alerting for the Security Operations Center (SOC) AppDefense doesn't produce a lot of alerts, but when it raises the alarm its smart to listen. The authoritative alerts generated by AppDefense coupled with automated response capabilities allow security administrators to focus on catching and eradicating threats from their environment, rather than sifting through noisy data and investigating threats that aren't there.
Transforming application security readiness reviews In the world of modern application development, applications are launched, changed, and decommissioned rapidly. By the time a security team learns of the existence of a new application, it has often already changed. AppDefense creates a common source of truth between application team and the security teams, streamlining the security review process.
VMware has changed the face of network security with our network virtualization platform, VMware NSX, and its ability to enable micro-segmentation across the data center. NSX architects network and security services such as firewalling directly into the hypervisor, enabling a least privilege model for the network. The net outcome is that network security teams can prevent threats from moving laterally within their environments.
AppDefense layers in threat detection and response capabilities into another core area of the infrastructure, enabling a least privilege model for data center endpoints. Should a threat make it onto an endpoint, AppDefense will immediately detect the threat and automatically respond with precision. Together, NSX and AppDefense offer a robust solution for securing the application infrastructure and thus, the applications and data that live there.
The AppDefense platform provides:
AppDefense's operation from within the hypervisor also provides protection and abstraction not available with traditional end-point protection platforms creating the most effective least-privilege model for the application layer.
Protection of the Protector in a Separate "Trust Zone"
Kernel Level Monitoring
Memory and Process Monitoring
Unlike other endpoint security products, AppDefense is isolated from the attack surface without sacrificing the context necessary to provide accurate security alerts. Furthermore, AppDefense works with NSX and other infrastructure control points to automate the response to detected threats, minimizing the potential for data exfiltration and the impact to the business.
Congratulations on completing Module 1 -- An overview of the VMware AppDefense Platform.
Proceed to the next module.
Lab Module List:
Module 1 - Overview of VMware App Defense (15 minutes) - Basic - This module will walk you through the structure of the platform.
Module 2 - Exploring & Utilizing the AppDefense Platform (45 minutes) - Basic - This module will walk you through the creation of a security scope. Secondly, you will monitor the application after various attacks have been made. Finally, you will perform remediation, quarantine and upgrade actions.
If you would like to end the lab now, you can simply click the "End" button in the upper part of your screen. Otherwise, please proceed to Module 2.
Module 2 - Exploring and Utilizing the AppDefense Platform (45 minutes)
In this section, we will be preparing the lab for us to learn more about AppDefense. While we are preparing the lab, we will explore the environment (hosts and VMs) in the lab.
The preparation workflow will invoke a script to configure the parameters on the AppDefense appliance.
A new window will pop up, showing the status of the preparation script. It will take approximately 5-7 minutes for the preparation workflow to complete.
IMPORTANT: Do not close the window or refresh the browser.
Let's access vSphere Web Client and review vCenter Server Inventory while we wait.
These are the following objects in vCenter Server inventory:
Nested Virtualization: For the purpose of this lab, the two ESXi hosts are actually Nested Virtualization. Nested Virtualization means the ESXi are running inside VMs instead of bare-metal servers. Although Nested Virtualization is not supported for production, it is sufficient for running this lab which is meant for learning AppDefense.
Corde's Cord App: In this lab, we have also provisioned a three-tier web application named Corde's Cord App. The VMs used in this three-tier web application are as follows:
As we proceed with this lab, you will have a hands-on experience of AppDefense's capabilities in alerting, monitoring and integration with NSX as we perform unauthorized and malicious actions on Corde's Cord App.
The window will show that preparation workflow has completed. We can start performing the steps in this module to learn more about AppDefense.
In this section, we will exploring the AppDefense Plugin in vCenter Server and AppDefense Plugin.
Online Trust Analysis - Helps in the analysis of the processes to display the reputation for all the processes that are monitored by AppDefense. The status is displayed as:
AppDefense - Displays connection status with the AppDefense Manager when your connectivity mode is SaaS.
Hosts and VMs
There are four widgets in the AppDefense Plugin in vCenter Server:
In the next few steps, we will explore Process Reputation and Critical Vulnerabilities.
Note: If you don't see suspicious behaviour, read through the image below and proceed to the next steps
You may need to scroll down to see the VM:
You will be able to review the processes running in the VM. Let's look at the suspicious process:
In this view, you will be able to review the details of the suspicious process.
Now we will use App-defense plugin to investigate servers with known vulnerabilities..." This is the same for all widgets described in page 41
In this view, you will be able to review the vulnerabilities of this VM.
We have explored Process Reputation and Critical Vulnerabilities.
For Windows Integrity Checks widget, you may want to review it after completing a later section named "Attack the Application and Validate Automated Response" in this module. In that section, we will compromise the OS integrity of DB-VM, hence you will be able to see the alerts on Windows Integrity Checks widget.
Next, we will explore the AppDefense Appliance.
We will not need the vSphere Web Client until later in the lab, so let's close the browser tab.
NOTE: You may need to scroll down to see the NSX details.
Review the parameters configured by the preparation workflow:
IMPORTANT: Take note of the Manager UUID under the AppDefense Manager section. We will verify the Manager UUID when we login to the AppDefense Manager in the next section.
In this section, we will be exploring the AppDefense Manager.
This is the login page for AppDefense Manager. Next, we will retrieve the email and password required for login.
IMPORTANT: Your assigned email address may be different from the above screenshot. Please use the email address assigned for your lab.
A window will pop-up for you to retrieve the password.
IMPORTANT: Please only copy the characters highlighted in green. Your password should only have six characters.
IMPORTANT: Your assigned email address may not be the same. If your login fail, please check your password. Your password should only have six characters.
The AppDefense Manager is a multi-tenant cloud service that delivers the complete AppDefense feature set. You can use the AppDefense Manager to define the intended behavior and protection rules of your applications and then monitor security events and alerts in real time. In addition to management capabilities, the AppDefense Manager provides process reputation services, machine learning capabilities, and other additional visibility features for your environment.
However, the AppDefense Manager is running in a local instance within the VMware Hands-on Labs environment. Your assigned user is also created in it's own tenant.
In the dashboard view, you are able to review the protection coverage, scopes in discovery, alerts, and provisioning events.
Move your mouse to the left-hand navigation bar and:
You may need to scroll right in the browser to see details of the AppDefense Appliance.
AppDefense Appliance is an on-premises based control point for ingress and egress of data from and to the AppDefense Manager. It brokers connections to the VMware management components like vCenter Server and makes outbound connections to the AppDefense Manager.
Review and validate configurations of AppDefense Appliance:
The UUID is the same as the UUID shown in the AppDefense Appliance shown in the previous steps.
In this lab, we did not provision any container hence there is no inventory on containers.
This view shows the VMs and containers in the inventory that are not assigned to any security scopes in AppDefense. It will also show the operational status of the host and guest modules of the unassigned VMs and containers.
The orange and red areas represent VMs and containers that are either in discovery mode or under protection.
You may need to scroll right in the browser to see details of the Downloads.
The On-Prem build of the AppDefense Manager used in this lab does not support automatic downloads, so the image on this step is from the actual production cloud based AppDefense Manager. You can see that all documentation, OVA files, VIBs and guest modules are available in the management portal itself.
A Security Scope in AppDefense is the foundational component that establishes what the intended state and specific allowed behaviors of an application should be. In this section, we will walk through the steps involved in creating and deleting an AppDefense Security Scope.
You will notice that a security scope named Corde's Cords App has been created during the preparation workflow. We will review Corde's Cords App in the next section. In this section, you will create a new security scope and add members to the security scope. Finally, you will also learn how to delete the security scope.
A Security Scope defines the relevant configuration elements to protect an application and its constituent workloads. These configuration elements constitute a "blueprint" or "birth certificate" for the application. It contains a description, member workloads, rules and behaviors.
This is fundamental to the AppDefense philosophy. By focusing on applications as opposed to just indvidual endpoints, AppDefense derives a greater contextual knowledge of the intended state of the application.
In the Service Description, you can specify other information of your choosing. This is not mandatory, but can be useful in operational environments to denote additional relevant information on the service.
Behaviors are process executions (CLIs) and network activities (inbound and outbound connections) exhibited within a service.
Once scopes and services are created, AppDefense enters Discovery Mode. AppDefense creates a list of allowed behaviors (for example ports and processes) to build a blueprint or a whitelist of the natural state of the application. The system dynamically populates allowed behaviors based on a runtime view of the application over a period. During this time, all relevant activity is recorded as the application is functioning.
During this time, no action is needed as AppDefense is learning the environment automatically.
In cases where you want to specifically define allowed behaviors, you have the option to EXPORT or ADD a behavior.
You have deleted your security scope as it is not needed in the other sections of the lab. In the next section, we will review Corde's Cords App, a security scope that was created during the preparation workflow. Corde's Cords App is also pre-assigned with members and pre-populated with allowed behaviors.
In this section, we will explore a security scope named Corde's Cords App which has been created during the preparation workflow. Corde's Cords App is also pre-assigned with members and pre-populated with allowed behaviors. We will now examine Corde's Cords App.
A security scope defines the relevant configuration elements to protect an application and its constituent workloads. These configuration elements are like a blueprint or a birth certificate for the application. It contains a description, member workloads, rules, and behaviors. Security scopes are a grouping of data center assets (VMs, Containers, and so on) that make up an application or a regulatory scope.
You may need to move around the Topology Canvas to see Web, App and DB Tiers' services.
NOTE: Due to the screen resolution of the lab, you may need to move around the topology to see all the services.
IMPORTANT: Legend provides explanation of the graphics used in the topology map.
The Topology tab enables viewing large amount of complex application behavior data easily in an interactive graphic. A graphical visualization illustrates application behavior, showing the relationships of the services within the application and also the relationships of the connected components (VMs, private address, public address, and so on) to each other. Remote nodes and connectivity information is also graphically displayed in a way to enable users to focus on the application servers that are causal or may have the greatest impact. The tab represents the rules in a graphical manner by using symbols to show how services are related to each other depending on the allowed behaviors and connection rules set for each of the services. The data displayed on the tab is read-only. Any changes made on the Services tab gets reflected on the Topology tab.
AppDefense creates a list of allowed behaviors (e.g. ports, processes, etc.) to build a "blueprint" or "whitelist" of the intended state of the application. AppDefense can create this blueprint with assistance from provisioning interfaces such as vRealize Automation, vRealize Orchestrator, Puppet, or similar engines.
However, when the application is already deployed, AppDefense can also "learn" these behaviors. After a Security Scope is created and applied to an application, it defaults to "Learning Mode". During this time, all relevant activity is recorded as the application is functioning.
Once reviewed, this master list of intended activity can be validated and/or modified by a security operations team or application owner. After the final intended state is determined, the security scope is placed into "protected mode".
Once the scope is moved into this mode, AppDefense will use the allowed behavior list to enforce the correct security context and posture for the workload against any deviation from that list.
IMPORTANT: Do not click on VERIFY AND PROTECT. Normally, the learning period for a workload or application is a recommended 7-14 days. Since we do not have that timeframe from within the HOL lab environment, we have automated most of the "allowed processes" into the creation of the service.
A member is a virtual machine (VM) within a service. Members or VMs in a service must have an identical operating system (means within a service, all the VMs must be homogeneous – either all Microsoft or all Linux). In this lab, there are two app VMs in the App Tier service.
In this section, we will examine the Discovery Mode of AppDefense.
Upon creation, scopes are placed into Discovery Mode. In this mode, when AppDefense recognizes a virtual machine exhibiting a new behavior, it adds it to the allowed behavior list for the associated service. Normally, the learning period for a workload or application is 7–14 days, although it can vary depending on the workload. Learned behaviors include process executions, command-line arguments, network connections, and more. This information is organized into a process-specific card view in the Services tab.
You will notice that Python 2.7.5 and Python 3.5.1 are installed on Web-VM.
You will notice that there is only python2.7 behavior but no python3 behavior in Web Tier. This is because python3 was newly installed and has not been used before.
The Putty session will run a script in Web-VM. The script runs for about a minute and does the following:
IMPORTANT: Do not close the Putty Session until prompted to "Press Enter to close this putty session".
python3 in Web-VM has attempted an outbound connection to python.org. Since the security scope is still in Discovery Mode, AppDefense recognize this new behavior (python3) and adds it to the allowed behavior list for Web-Tier.
In this section, we will move the security scope to Protected Mode. After which, we will review and edit the rules associated with the scope.
Once the allowed behaviors learning is satisfactory, you can move the scope and all services within the scope to the Protected Mode by clicking on "VERIFY AND PROTECT". Protected Mode marks the golden image of the application state and begins locking down the behavior. After moving to Protected Mode, rules are applied. You can view the applied rule under the Rules tab, and any violations generates an alarm.
There are five vectors that are used to alert and remediate. By default, the only action for the Remediation rules is set to Alert and the enforcement is automatic. You can edit the rule settings based on the action that you want AppDefense to take.
Enforce Process Monitoring: How do you want AppDefense to monitor a process execution?
Enforce Outbound Connections: If AppDefense sees a new outbound connection from an allowed process, what do you want it to do?
Enforce Inbound Connections: If AppDefense sees a new inbound connection from an allowed process, what do you want it to do?
Enforce Guest OS Integrity: Windows-only. If AppDefense detects that the integrity of your operating system (OS) has been compromised, what do you like it to do?
Enforce AppDefense Module Integrity: Windows-only. If AppDefense detects the integrity, the AppDefense Module has been compromised (potentially turned off), what do you like it to do?
You cannot set automatic remediation action for the Guest module down alert. Remediation for this action can only be taken manually.
This will change the default behavior for Guest OS integrity issues to provide a Manual Quarantine option for the VM using an NSX policy. The other remediation actions (e.g. Suspend, Power Off, & Snapshot) are done directly at the vSphere/vCenter level.
You will notice that the remediation action in case of violation is to Quarantine the VM. To use the Quarantine action, AppDefense must be integrated with VMware NSX. In the next section, we will examine the NSX Security Group and Policy that are used for AppDefense.
In this section, we will explore the integration between AppDefense and NSX. We will review the NSX Security Group and Policy that are used for AppDefense.
The AppDefense.AnomalyFound (Security Tag) was automatically created when the AppDefense Appliance was integrated with NSX Manager during installation.
The AppDefense Quarantine Group was automatically created when the AppDefense Appliance was integrated with NSX Manager during installation. There is currently no VM being quarantined because there is no violation on Guest OS Integrity.
The AppDefense Quarantine Policy was automatically created when the AppDefense Appliance was integrated with NSX Manager during installation.
When there is a violation of Guest OS Integrity, these firewall rules will be applied to the quarantined VM.
Now that you have defined the intended state of your application, you will use tools provided to attack the application and observe the results.
At this point, you have reviewed the installation of AppDefense and its integration with NSX and vCenter. You have created a security scope and added the web and db services to it.
In addition, we have modified rules of the Guest OS Integrity of the DB service so that it would require a manual interaction prior to being quarantined by an NSX policy that was automatically built by AppDefense.
In this section, we will generate an outbound network connection attempt which will generate an alarm against our security scope. Then we will use scripts to simulate attacks on the kernel and host level AppDefense modules . Once the attacks are executed, we will validate the alarms in AppDefense Manager and perform a Quarantine of the VM. Finally, we will test to ensure the DB VM is isolated.
DB-Tier.rdp will provide remote desktop access to DB-VM of the application.
Open TCP Connection.cmd will run a script to simulate an outbound connection.
The script executed the following command in an attempt to simulate an outbound connection:
powershell -File Open-TCP-Connection.ps1 -dest www.google.com -port 443
As this attempt is an unauthorized outbound connection, the command prompt shows an "Timeout!" output. We will return to AppDefense Manager to review the events that were captured by this attempt.
Although the unauthorized outbound connection is benign, it is not a permitted behavior in the DB-VM. Hence we shall not allow the behavior to be added to AppDefense Manager for DB-VM. This way, we will get updated again if anyone tries to perform an unauthorized outbound connection.
Next, we will clear the unauthorized outbound connection alerts triggered from the previous steps.
Return to the DB-VM.
GIRogue is a process that can be used to simulate different attacks at both the Guest and Host level. The details and specifics of this tool are beyond the scope of this lab.
Two Command Prompt windows will appear. You may need to expand the Command Prompt windows to see the outputs of the scripts.
OS Integrity Attack.cmd executed scripts to do the following:
Note that your Alert ID may be different.
You will be able to see the details of the Alert on this page.
You will notice that the remediation status is shown as "Queued: Appdefense- Quarantine". It may take a while for the remediation to be completed.
Once the Last Remediation Action is shown as "Action taken: Appdefense - Quarantine", we can proceed to the next step.
You will notice that db-01 is being quarantined.
You will notice that the website encountered an error and cannot be loaded. This is because the DB-VM is being quarantined.
You will notice VM Count is 0. This means that DB-VM has been removed from the quarantine. Let's try and access the Web Portal now.
With the DB-VM removed from quarantine, the website loads properly as seen in the screenshot above.
In this section, we will examine how upgrades of application components are identified and processed with the AppDefense platform.
Take note of Hash values of MD5 and SHA256. These Hash values will change after we perform the upgrade of Python3.
You will notice that Python3 version 3.5.1 is currently installed on Web-VM. This script will upgrade the Python3 from version 3.5.1 to 3.6.8 in Web-VM.
The Putty session will run a script in Web-VM. The script runs for about a minute and does the following:
IMPORTANT: Do not close the Putty Session until prompted to "Press Enter to close this putty session".
When the upgrade event appears, you will see that AppDefense has captured Python3 upgrade from version 3.5.1 to 3.6.8 in Web-VM.
You will notice the Hash values of MD5 and SHA256 have changed. This is because the version of Python3 has changed from version 3.5.1 to 3.6.8. The Hash values are based on the current version (3.6.8) of Python3 installed on Web-VM.
You will be able to see the Hash values based on the older version (3.5.1) of Python3.
In this module, you went through the basic workflow of creating/deleting security scopes, service definitions and remediation policies. You then simulated an attack on a test application and observed VMware AppDefense quarantine the virtual machine using VMware NSX security policies. Finally, you upgraded a application component to see how AppDefense deals with upgrade scenarios in an intended state model.
If you would like more information on VMware's AppDefense, please check out our products page at: www.vmware.com/appdefense
You can proceed to any module in the lab below.
(45 minutes) - Basic - This module will walk you through the creation of a security scope. Secondly, you will monitor the application after various attacks have been made. Finally, you will perform remediation, quarantine, and upgrade actions.
If you want to download the manual for this or any other Hands on Lab, please visit http://docs.hol.vmware.com
Thank you for participating in the VMware Hands-on Labs. Be sure to visit http://hol.vmware.com/ to continue your lab experience online.
Lab SKU: HOL-2042-01-NET