App Volumes Architecture
This chapter is one of a series that make up the VMware Workspace ONE and VMware Horizon Reference Architecture, a framework that provides guidance on the architecture, design considerations, and deployment of Workspace ONE and Horizon solutions. This chapter provides information about architecting VMware App Volumes.
The VMware App Volumes™ just-in-time application model separates IT-managed applications and application suites into administrator-defined application containers. App Volumes also introduces an entirely different container used for persisting user changes between sessions.
Figure 1: App Volumes Just-in-Time Application Model
This version of the App Volumes reference architecture was built using App Volumes 4. App Volumes 4 architecture is similar to that of the earlier App Volumes 2.x versions, but there are some notable differences in components, lifecycle management, and terminology. The App Volumes 4 Feature Review is an interactive demo that will help you quickly familiarize yourself with the new concepts. If you are planning to upgrade from App Volumes 2.x to 4, see VMware App Volumes 4 Installation and Upgrade Considerations to learn about the various upgrade paths available, including App Volumes 2.18 and App Volumes 4 co-existence.
App Volumes serves two functions. The first is delivery of software programs that are not in the golden image VM image for VDI and RDSH. App Volumes groups one or more programs into packages based on the requirements of each use case. A package is a virtual disk containing one or more programs that are captured together.
The packages are added to applications. Applications are used to assign packages to AD entities such as user, group, organizational unit (OU), or machine. The packages can be mounted each time the user logs in to a desktop, or at machine startup. For VDI use cases, packages can be mounted at login. With RDSH use cases, because packages are assigned to the machine account, the packages are mounted when the App Volumes service starts.
App Volumes also provides user-writable volumes, which can be used in specific use cases. Writable volumes provide a mechanism to capture user profile data, user-installed applications that are not or cannot be delivered by packages, or both. This reduces the likelihood that persistent desktops would be required for a use case. User profile data and user-installed applications follow the user as they connect to different virtual desktops.
Table 1: Implementation Strategy for App Volumes
App Volumes was deployed and integrated into the VMware Horizon® 7 on-premises environment.
This design was created for an environment capable of scaling to 8,000 concurrent user connections.
This strategy allows the design, deployment, and integration to be validated and documented.
Note: If you are new to App Volumes, VMware recommends the following resources to help you familiarize yourself with the product:
- VMware App Volumes Product Page
- Mastering App Volumes (Activity Path)
- VMware Horizon 7 - App/Desktop Virtualization with Just in Time Management (hands-on lab)
For additional hands-on learning, consider this three-day course on implementing App Volumes and Dynamic Environment Manager on Horizon 7.
The App Volumes Agent is installed in the guest operating system of nonpersistent VMs. The agent communicates with the App Volumes Manager instances to determine package and writable volumes entitlements. Packages and writable volumes virtual disks are attached to the guest operating system in the VM, making applications and personalized settings available to end users.
Figure 2: App Volumes Logical Components
The components and features of App Volumes are described in the following table.
Table 2: App Volumes Components and Concepts
App Volumes Manager
App Volumes Agent
VMware vCenter Server®
The following figure shows the high-level logical architecture of the App Volumes components, scaled out with multiple App Volumes Manager servers using a third-party load balancer.
Figure 3: App Volumes Logical Architecture
Key Design Considerations
- Always use at least two App Volumes Manager servers, preferably configured behind a load balancer.
Note: This setup requires a shared SQL Server.
- An App Volumes instance is bounded by the SQL database.
- Any kernel mode applications should reside in the base image and not in a package.
- Use storage groups (if you are not using VMware vSAN™) to aggregate load and IOPS.
- Note: Packages are very read intensive.
- Storage groups may still be applicable to vSAN customers for replicating packages. See Multi-site Design Using Separate Databases for more information.
- Place packages on storage that is optimized for read (100 percent read).
- Place writable volumes on storage optimized for random IOPS (50/50 read/write).
- Assign as few packages as possible per user or device. See the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354) for the recommended number of packages per VM.
- Note: This KB article was written for App Volumes 2.x AppStacks. Although most of the content is applicable to App Volumes 4, the maximum number of package attachments tested has increased.
- App Volumes 4 defaults to an optimized Machine Managers configuration. Use the default configuration and make changes only when necessary.
Figure 4: Default Machine Managers Configuration
Note: With previous versions of App Volumes, configuring the Mount ESXi option or, mount on host was recommended to reduce the load on vCenter Server and improve App Volumes performance. App Volumes 4 and later provides new optimizations in the communication with vCenter Server. Most implementations will no longer benefit from enabling the Mount ESXi option.
You can enable the Mount Local storage option in App Volumes to check local storage first and then check central storage. Packages are mounted faster if stored locally to the ESXi (vSphere) host. Place VMDKs on local storage and, as a safeguard, place duplicates of these VMDKs on central storage in case the vSphere host fails. Then the VMs can reboot on other hosts that have access to the centrally stored VMDKs.
If you choose to enable Mount ESXi or Mount Local, all vSphere hosts must have the same user credentials. Root-level access is not required. See Create a Custom vCenter Role for more information.
Network Ports for App Volumes
A detailed discussion of network requirements for App Volumes is outside of the scope of this guide. See Network connectivity requirements for VMware App Volumes.
See Network Ports in VMware Horizon for a comprehensive list of ports requirements for VMware Horizon®, App Volumes, and much more.
App Volumes in a Horizon Environment
One key concept in a VMware Horizon® environment design is the use of pods and blocks, which gives us a repeatable and scalable approach. See the Pod and Block section of Horizon Architecture for more information on pod and block design.
Consider the Horizon block design and scale when architecting App Volumes.
Table 3: Strategy for Deploying App Volumes in Horizon Pods
An App Volumes Manager instance was deployed in each pod in each site.
The App Volumes machine manager was configured for communication with the vCenter Server in each resource block.
Standardizing on the pod and block approach simplifies the architecture and streamlines administration.
In a production Horizon environment, it is important to adhere to the following best practices:
Scalability and Availability
As with all server workloads, it is strongly recommended that enterprises host App Volumes Manager servers as vSphere virtual machines. vSphere availability features such as cluster HA, VMware vSphere® Replication™, and VMware Site Recovery Manager™ can all complement App Volumes deployments and should be considered for a production deployment.
In production environments, avoid deploying only a single App Volumes Manager server. It is far better to deploy an enterprise-grade load balancer to manage multiple App Volumes Manager servers connected to a central, resilient SQL Server database instance.
As with all production workloads that run on vSphere, underlying host, cluster, network, and storage configurations should adhere to VMware best practices with regard to availability. See the vSphere Availability Guide for more information.
App Volumes Managers
App Volumes Managers are the primary point of management and configuration, and they broker volumes to agents. For a production environment, deploy at least two App Volumes Manager servers. App Volumes Manager is stateless—all of the data required by App Volumes is located in a SQL database. Deploying at least two App Volumes Manager servers ensures the availability of App Volumes services and distributes the user load.
For more information, see the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354).
Although two App Volumes Managers might support the 8,000 concurrent users design, additional managers are necessary to accommodate periods of heavy concurrent usage, such as for logon storms.
Table 4: Strategy for Scaling App Volumes
Four App Volumes Manager servers were deployed with a load balancer.
This strategy satisfies the requirements for load and provides redundancy.
Multiple vCenter Server Considerations
Configuring multiple vCenter Servers is a way to achieve scale for a large Horizon pod, for multiple data centers, or for multiple sites.
With machine managers, you can use different credentials for each vCenter Server, but vSphere host names and datastore names must be unique across all vCenter Server environments. After you have enabled multi-vCenter-Server support in your environment, it is not recommended to revert back to a single vCenter Server configuration.
Note: In a multiple-vCenter-Server environment, a package is tied to storage that is available to each vCenter Server. It is possible that a package visible in App Volumes Manager could be assigned to a VM that does not have access to the storage. To avoid this issue, use storage groups to replicate packages across vCenter Servers. For instructions, see Configure Storage Groups.
Multiple AD Domain Considerations
App Volumes supports environments with multiple Active Directory domains, both with and without the need for trust types configured between them. See Configuring and Using Active Directory for more information.
An administrator can add multiple Active Directory domains through the Configuration > Active Directories tab in App Volumes Manager. An account with a minimum of read-only permissions for each domain is required. You must add each domain that will be accessed for App Volumes by any computer, group, or user object. In addition, non-domain-joined entities may be allowed by enabling this setting.
Figure 5: Enabling Non-Domain-Joined Entities
Host configurations have significant impact on performance at scale. Consider all ESXi best practices during each phase of scale-out. To support optimal performance of packages and writable volumes, give special consideration to the following host storage elements:
- Host storage policies
- Storage network configuration
- HBA or network adapter (NFS) configuration
- Multipath configuration
- Queue-depth configuration
For best results, follow the recommendations of the relevant storage partner when configuring hosts and clusters.
For more information, see the vSphere Hardening Guide.
Use at least two App Volumes Managers in production and configure each App Volumes Agent to point to a load balancer, or use a DNS server that resolves to each App Volumes Manager in a round-robin fashion.
For high performance and availability, an external load balancer is required to balance connections between App Volumes Managers.
The main concern with App Volumes Managers is handling login storms. During the login process, user-based packages and writable volumes must be attached to the guest OS in the VMs. The greater the number of concurrent attachment operations, the more time it might take to get all users logged in.
For App Volumes 4, the exact number of users each App Volumes Manager can handle will vary, depending on the load and the specifics of each environment. See the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354) for tested limits of users per App Volumes Manager server and login rates.
VMware recommends that you test the load and then size the number of App Volumes Manager servers appropriately. To size this design, we assumed each App Volumes Manager was able to handle 2,000 users.
Table 5: Strategy for App Volumes Scalability and Availability
A load balancer was placed in front of the App Volumes Manager servers.
The NSX Advanced Load Balancer (formerly Avi Networks) was used.
The load balancer properly distributes load and keeps the services available in the event of an issue with one of the managers.
The following figure shows how virtual desktops and RDSH-published applications can point to an internal load balancer that distributes the load to two App Volumes Managers.
Figure 6: App Volumes Managers Load Balancing
In the following list, the numbers correspond to numbers in the diagram.
- No additional configuration is required on the App Volumes Manager servers.
- Load balancing of App Volumes Managers should use the following:
- Ports = 80, 443
- Persistent or session stickiness = Hash all Cookies
- Timeout = 6 minutes
- Scheduling method = round robin
- HTTP headers = X-Forward-For
- Real server check = HTTP
App Volumes 4 uses a Microsoft SQL Server database to store configuration settings, assignments, and metadata. This database is a critical aspect of the design, and it must be accessible to all App Volumes Manager servers.
An App Volumes instance is defined by the SQL database. Multiple App Volumes Manager servers may be connected to a single SQL database.
For nonproduction App Volumes environments, you can use the Microsoft SQL Server Express database option, which is included in the App Volumes Manager installer. Do not use SQL Server Express for large-scale deployments or for production implementations.
App Volumes works well with both SQL Server failover cluster instances (FCI) and SQL Server Always On availability groups. Consult with your SQL DBA or architect to decide which option better fits your environment.
Table 6: Implementation Strategy for the SQL Server Database
A SQL database was placed on a highly available Microsoft SQL Server. This database server was installed on a Windows Server Failover Cluster, and an Always On availability group was used to provide high availability.
An Always On availability group achieves automatic failover. Both App Volumes Manager servers point to the availability group listener for the SQL Server.
A successful implementation of App Volumes requires several carefully considered design decisions with regards to disk volume size, storage IOPS, and storage replication.
Package and Writable Volume Template Placement
When new packages and writable volumes are deployed, predefined templates are used as the copy source. Administrators should place these templates on a centralized shared storage platform. As with all production shared storage objects, the template storage should be highly available, resilient, and recoverable. See Configuring Storage to get started.
Package sizing and writable volume sizing are critical for success in a production environment. Package volumes should be large enough to allow programs to be installed and should also allow for software updates. Packages should always have at least 20 percent free disk space available so administrators can easily update programs without having to resize the package volumes.
Writable volumes should also be sufficiently sized to accommodate all users’ data. Storage platforms that allow for volume resizing are helpful if the total number of writable volume users is not known at the time of initial App Volumes deployment.
Because packages and writable volumes use VMware vSphere® VMFS, the thin-provisioned, clustered Virtual Machine File System from VMware, storage space is not immediately consumed. Follow VMware best practices when managing thin-provisioned storage environments. Free-space monitoring is essential in large production environments.
Writable Volumes Delay Creation Option
Two policy options can complicate free-space management for writable volumes:
- The option to create writable volumes on the user’s next login means that storage processes and capacity allocation are impacted by user login behavior.
- The option to restrict writable volume access (and thus initial creation) to a certain desktop or group of desktops can also mean that user login behavior dictates when a writable volume template is copied.
In a large App Volumes environment, it is not usually a good practice to allow user behavior to dictate storage operations and capacity allocation. For this reason, VMware recommends that you create writable volumes at the time of entitlement, rather than deferring creation.
App Volumes uses a construct called storage groups. A storage group is a collection of datastores that are used to serve packages or distribute writable volumes.
The two types of storage groups are:
- Package storage groups – Used for replication.
- Writable volume storage groups – Used for distribution.
In App Volumes 4, the packages within a storage group can be replicated among its peers to ensure all packages are available. Having a common datastore presented to all hosts in all vCenter Servers allows packages to be replicated across vCenter Servers and datastores.
Two automation options for package storage groups are available:
- Automatic replication – Any package placed on any datastore in the storage group is replicated across all datastores in the group.
- Automatic import – After replication, the package is imported into App Volumes Manager and is available for assignment from all datastores in the storage group.
When using package storage groups, the App Volumes Manager manages the connection to the relevant package, based on location and number of attachments across all the datastores in the group.
Scaling App Volumes with Storage Groups
Once created, packages are read-only. As more and more users are entitled to and begin using a given package, the number of concurrent read operations increases. With enough users reading from a single package, performance can be negatively impacted. Performance can be improved by creating one or more copies of the package on additional datastores, and spreading user access across them.
Packages can be automatically replicated to multiple datastores in a storage group. This replication creates multiple copies of packages. Access is spread across the datastores, ensuring good performance as App Volumes scales to serve more end users. See the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354) for the recommended number of concurrent attachments per package.
Multi-site App Volumes Implementations and Storage Groups
Storage groups can also be used to replicate packages from one site to another in multi-site App Volumes configurations. By using a non-attachable datastore available to hosts in each site, packages created at one site can be replicated to remote sites to serve local users.
A datastore configured as non-attachable is ignored by the App Volumes Manager while mounting volumes, and the storage can be used solely for replication of packages. This means you can use a datastore on slow or inexpensive storage for replication, and use high-speed, low-latency storage for storing mountable volumes.
This non-attachable datastore can also be used as a staging area for package creation before deploying to production storage groups. This topic is covered in more detail in the Configuration of the Non-Attachable Datastore and Storage Group section of App Volumes Configuration.
Writable Volumes and Storage Groups
Writable volume storage groups are used to distribute volumes across datastores to ensure good performance as writable volumes are added. See the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354) for the recommended number of writable volumes per datastore.
Table 7: Implementation Strategy for Storage Groups
Storage groups were set up to replicate packages between datastores.
An NFS datastore was used as a common datastore between the different vSphere clusters.
This strategy allows the packages to be automatically replicated between VMFS datastores, between vSAN datastores and between vSphere clusters.
This section provides guidance about creating, sizing, scaling, provisioning, configuring, and updating packages.
By default, a single 20-GB package template is deployed in an App Volumes environment. This template is thin-provisioned and is provided in both a VMDK and VHD format. This template can be copied and customized, depending on how large the package needs to be for a given deployment scenario. For more information, see the VMware Knowledge Base article Creating a new App Volumes package template VMDK smaller than 20 GB (2116022).
Note: Although this KB article was written for AppStacks, it is applicable to App Volumes 4 packages as well.
If you have packages from a previous App Volumes 4 release, they will continue to work. However, additional features or fixes included in later versions are not applied to packages created with earlier versions.
Packages at Scale
The number of packages that can be attached to a given VM is technically limited by the maximum number of possible drive attachments in Windows and vSphere.For example, ESXi has a limit of 59 VMDKs + 1 OS disk. See the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354) for guidance. In practice, the number of packages attached to a VM will likely be considerably lower than the maximum values.
Attaching packages involves the following processes:
- The disk mount (mounting the package VMDK to the VM)
- The virtualization process applied to the content in the package (merging files and registry entries with the guest OS)
The time required to complete the virtualization process varies greatly, depending on the programs contained in a given package. The more packages that need to be attached, the longer this operation might take to complete.
Packages may be assigned to a number of Active Directory objects, which has implications for the timing and specifics of which volumes are attached. See Working with Applications.
Recommended Practices for Packages in Production Environments
The size of the default package template is 20 GB. The default writable volume template is 10 GB. In some environments, it might make sense to add larger or smaller templates. For information on creating multiple, custom-sized templates, see the VMware Knowledge Base article Creating a New App Volumes package template VMDK smaller than 20 GB (2116022).
App Volumes 4 includes enhancements to the agent compared with 2.x agents. The result is improved logon times and better performance with many packages attached. Although you can likely attach more App Volumes 4 packages than App Volumes 2.x AppStacks, we recommend keeping the total number of packages assigned to a given user or computer relatively small. This can be accomplished by adding multiple programs to each package. Group apps in such a way as to simplify distribution.
The following is a simple example for grouping App Volumes programs into packages:
- Create a package containing core programs (apps that most or all users should receive). This package can be assigned to a large group or OU.
- Create a package for departmental programs (apps limited to a department). This package can be assigned at a group or departmental level.
For traditional storage (VMFS, NFS, and so on):
- Do not place packages and VMs on the same datastore.
- Use storage groups for packages when packages are assigned to a large population of users or desktops. This helps to distribute the aggregated I/O load across multiple datastores, while keeping the assignments consistent and easy to manage.
- Packages and VMs can be placed on a single datastore.
- Storage groups for packages are not applicable in a vSAN implementation.
See the VMware Knowledge Base article VMware App Volumes Sizing Limits and Recommendations (67354).
Packaging Applications Recommended Practices
Consider the following best practices when creating and packaging applications:
- The following characters cannot be used when naming packages: & “ ‘ < >
- For the packaging VM, use a clean golden image that resembles as closely as possible the target environment where the package is to be deployed. For example, the packaging VM and target should be at the same OS patch and service pack level and, if programs are included in the golden image, they should also be in the packaging VM.
- Do not use a packaging machine where you have previously installed and then uninstalled any of the programs that you will capture. Uninstalling a program might not clean up all remnants of the software, and the subsequent App Volumes package capture might not be complete.
- Always take a snapshot of your packaging VM before packaging or attaching any packages to it. If any packages have been assigned to the VM, or if the VM has been used previously for packaging, revert that VM to the clean snapshot before creating a new package.
Updating and Assigning Updated Packages Recommended Practices
App Volumes 4 introduced assignment types called marker and package to improve the administrative process of updating application packages. Using the CURRENT marker enables distribution of the current package to your end-user population; whereas using the package assignment type enables distribution of test versions to a subset of users for validation.
Once a new package has been tested and approved, you can simply change the CURRENT marker to point to the new package. As end users log off their desktop sessions, the old version of the package is detached. When they log on again, the new version is attached.
The following illustration shows the 7-Zip application, which contains three packages with different versions of the software.
Figure 7: Portion of the Application Tab Detailing the Three Packages on the Right
7-Zip 16.04 has the CURRENT marker, so it is distributed to the general population of end users. 7-Zip 19.0 is an updated package that contains a newer version of the 7-Zip program.
Figure 8: Portion of the Application Tab Detailing the Two Assignments on the Right
7-Zip 19.0 is using a package assignment type to directly assign that specific package to a group of test users for validation.
To learn more about assignment types refer to Assign Application Package in App Volumes 4 Feature Review.
To initiate an update to programs in an existing package, use the App Volumes Manager console to invoke the update process. This process clones the original package for you to work with and apply updates. End users continue to work from the original package to prevent user downtime. The new package with the updated programs is distributed by simply moving the CURRENT marker once it has been approved.
Consider the following best practices when updating and assigning updated packages:
- When creating and updating packages, use the Stage drop-down list to select an appropriate value. This makes it easy for you and other App Volumes admins to manage the lifecycle of the package.
- Use the marker assignment type to simplify updates for your general population of users.
- Use the Unset CURRENT option to disable delivery of a package without modifying assignments.
- Use the package assignment type for one-off, explicit assignments of a specific version.
- Note: If both package and marker assignments are made, the package assignment is used.
Although not required, App Volumes is often implemented in a Horizon environment. Consider the following when integrating App Volumes and Horizon.
- Do not attempt to include the Horizon Agent in an App Volumes package. The Horizon Agent should be installed in the golden image VM.
- Do not use a Horizon VM (guest OS with Horizon Agent installed) as a clean packaging VM. You must uninstall the Horizon Agent if it is present. Dependencies previously installed by the Horizon Agent, such as Microsoft side-by-side (SxS) shared libraries, are not reinstalled, and therefore are not captured by the App Volumes packaging process.
- See Installation order of End User Computing Agents for Horizon View, Dynamic Environment Manager, and App Volumes (2118048) for information on agent installation order.
Performance Testing for Packages
Test packages immediately after packaging to determine their overall performance. Using a performance analytics tool, such as VMware vRealize® Operations Manager™, gather virtual machine, host, network, and storage performance information for use when packages are operated on a larger scale. Do not neglect user feedback, which can be extremely useful for assessing the overall performance of an application.
Because App Volumes provides an application container and brokerage service, storage performance is very important in a production environment. Packages are read-only. Depending on utilization patterns, the underlying shared storage platform might have significant read I/O activity. Consider using flash and hybrid-flash storage technologies for packages.
This evaluation can be time-consuming for the administrator, but it is necessary for any desktop- transformation technology or initiative.
ThinApp Integration with Packages
Network latency is often the limiting factor for scalability and performance when deploying ThinApp packages in streaming mode. Yet ThinApp provides exceptional application-isolation capabilities. With App Volumes, administrators can present ThinApp packages as dynamically attached applications that are located on storage rather than as bits that must traverse the data center over the network.
Using App Volumes to deliver ThinApp packages removes network latency due to Windows OS and environmental conditions. It also allows for the best of both worlds—real-time delivery of isolated and troublesome applications alongside other applications delivered on packages.
With App Volumes in a virtual desktop infrastructure, enterprises can take advantage of local deployment mode for ThinApp packages. ThinApp virtual applications can be provisioned inside a package using all the storage options available for use with packages. This architecture permits thousands of virtual desktops to share a common ThinApp package through packages without the need to stream or copy the package locally.
Microsoft Office Application Packages
For deploying Microsoft Office applications through App Volumes, see the VMware knowledge base article VMware App Volumes 2.x with Microsoft Office Products (2146035).
Office Plug-Ins and Add-Ons
The most straightforward method is to include Microsoft Office plug-ins or add-ons in the same package as the Microsoft Office installation.
However, if necessary, you can include plug-ins or add-ons in packages that are separate from the packages that contain the Microsoft applications to which they apply. Before packaging the plug-in or add-on, install the primary application natively in the OS of the packaging VM.
Note: Ensure the plug-in or add-on is at the same version as the Microsoft Office application in the package. This includes any patches or updates.
Recommended Practices for Installing Office
VMware recommends that you install core Microsoft Office applications in the base virtual desktop image, and create one package for non-core Microsoft Office applications, such as Visio, Project, or Visio and Project together.
To create the package with Visio and Project, use a packaging machine with the same core Microsoft Office applications as on the base image. After the package is created, you can assign the package to only the users who require these non-core Microsoft Office applications.
RDSH Integration with Packages
App Volumes supports package integration with Microsoft RDSH-published desktops and published applications. Packages are assigned to RDSH servers rather than directly to users. Packages are attached to the RDSH server when the machine is powered on and the App Volumes service starts. Users are then entitled to the RDSH-published desktops or applications through the Horizon entitlement process.
Note: Writable volumes are not supported with RDSH assignments.
Consider associating packages at the OU level in Active Directory, rather than to individual computer objects. This practice reduces the number of package entitlements and ensures packages are always available as new hosts are created and existing hosts are refreshed.
Entitling packages to an OU where Horizon instant-clone RDSH server farms are provisioned ensures that all hosts are configured exactly alike, which supports dynamic growth of farms with minimal administrative effort.
Create dedicated packages for RDSH servers. Do not reuse a package that was originally created for a desktop OS.
When creating the package, install programs on a packaging machine that has the same operating system as that used on the deployed RDSH servers. Before installing software, switch the RDSH server to RD-Install mode. For more information, see Learn How To Install Applications on an RD Session Host Server.
See Infrastructure and Networking Requirements to verify that the Windows Server version you want to use for RDSH is supported for the App Volumes Agent.
For information about using App Volumes in a Citrix XenApp shared-application environment, see Implementation Considerations for VMware App Volumes in a Citrix XenApp Environment.
Application Suitability for Packages
Most Windows applications work well with App Volumes, including those with services and drivers, and require little to no additional interaction. If you need an application to continue to run after the user logs out, it is best to natively install this application on the desktop or desktop image.
The following sections briefly describe situations and application types where App Volumes might need special attention to work properly or where the application would work best when installed in the golden image, rather than in a package.
Applications That Work Best in the Golden Image
Applications that should be available to the OS in the event that a package or writable volume is not present should remain in the golden image and not in an App Volumes virtual disk. These types of applications include antivirus, Windows updates, and OS and product activations, among others. Applications that should be available to the OS when no user is logged in should also be placed in the golden image.
Similarly, applications that integrate tightly with the OS should not be virtualized in a package. If these apps are removed from the OS in real time, they can cause issues with the OS. Again, if the application needs to be present when the user is logged out, it must be in the golden image and not in a package. Applications that start at boot time or need to perform an action before a user is completely logged in, such as firewalls, antivirus, and Microsoft Internet Explorer, fall into this category.
Applications that use the user profile as part of the application installation should not be virtualized in an App Stack. App Volumes does not capture the user profile space C:\users\<username>. If, as part of its installation process, an application places components into this space, those components will not be recorded as part of the packaging process. If this happens, undesired consequences or failure of the application might result when the application is captured in a package.
Applications Whose Components Are Not Well Understood
In the rare event that an issue with an application does present itself, it is important to have a thorough understanding of how the application functions. Understanding the processes that are spawned, file and registry interactions, and where files are created and stored is useful troubleshooting information.
App Volumes is a delivery mechanism for applications. It is important to understand that App Volumes does an intelligent recording of an installation during the packaging process and then delivers that installation. If the installation is not accurate or is configured incorrectly, the delivery of that application will also be incorrect (“garbage in, garbage out”). It is important to verify and test the installation process to ensure a consistent and reliable App Volumes delivery.
App Volumes Agent Altitude and Interaction with Other Mini-Filter Drivers
The App Volumes Agent is a mini-filter driver. Microsoft applies altitudes to filter drivers. The concept is that the larger the number, the “higher” the altitude. Mini-filter drivers can see only the other filter drivers that are at a higher altitude. The actions at a lower altitude are not seen by filter drivers operating at a higher altitude.
The lower-altitude mini-filter drivers are the first to interact with a request from the OS or other applications. Generally speaking, the requests are then given to the next mini-filter driver in the stack (next highest number) after the first driver finishes processing the request. However, this is not always the case because some mini-filter drivers might not release the request and instead “close” it out to the OS or application.
In the case where a request is closed, the subsequent mini-filter drivers will never see the request at all. If this happens with an application running at a lower altitude than App Volumes, the App Volumes mini-filter driver will never get a chance to process the request, and so will not be able to virtualize the I/O as expected.
This is the primary reason that certain applications that use a mini-filter driver should be disabled or removed from the OS while you install applications with App Volumes. There might be additional scenarios where App Volumes Agent should be disabled, allowing other applications to install correctly in the base OS.
Other Special Considerations
The following guidelines will also help you determine whether an application requires special handling during the virtualization process or whether virtualization in a package is even possible:
- Additional application virtualization technologies – Other application virtualization technologies (Microsoft App-V, ThinApp, and others) should be disabled during packaging because the filter drivers could potentially conflict and cause inconsistent results in the packaging process.
- Mixing of 32- and 64-bit OS types – The OS type (32- or 64-bit) of the machine that the package is attached to should match the OS type that applications were packaged on. Mixing of application types in App Volumes environments follows the same rules as Windows application types—that is, if a 32-bit application is certified to run in a 64-bit environment, then App Volumes supports that configuration also.
- Exceptional applications – Some software apps just do not work when installed on an App Volumes package. There is no list of such applications, but an administrator might discover an issue where an application simply does not work with App Volumes.
- In summary, most applications work well with App Volumes, with little to no additional interaction needed. However, you can save time and effort by identifying potential problems early, by looking at the application type and use case before deciding to create a package.
Writable volumes can be used to persist a variety of data as users roam between nonpersistent desktop sessions. As is described in App Volumes 2.14 Technical What’s New Overview, Outlook OST and Windows Search Index files are automatically redirected to writable volumes, improving search times for customers using these technologies. See Working with Writable Volumes for information on creating and managing writable volumes.
Writable volumes are often complemented by VMware Dynamic Environment Manager™ to provide a comprehensive profile management solution. For technical details on using App Volumes with Dynamic Environment Manager, see the VMware blog post VMware User Environment Manager with VMware App Volumes.
Note the key differences between packages and writable volumes:
- Package VMDKs are mounted as read-only and can be shared among all desktop VMs within the data center.
- Writable volumes are dedicated to individual users and are mounted as the user authenticates to the desktop. Writable volumes are user-centric and roam with the user for nonpersistent desktops.
Writable Volume Templates
Several writable volume templates are available to suit different use cases. See Configuring Storage for options.
The UIA (user-installed applications)-only template provides persistence for user-installed applications. After a writable volume with the UIA-only template is created and assigned to a user, that user can install and configure applications as they normally would. The installation is automatically redirected to the writable volume, and persisted between desktop sessions.
Note: For this functionality to work properly, users require account permissions in Windows that allow application installation. You may also use Dynamic Environment Manager Privilege Elevation to complement UIA-only writable volumes.
Table 8: Implementation Strategy for App Volumes Writable Volumes
Writable volumes were created for and assigned to end users who required the ability to install their own applications.
The UIA-only writable volume template was used.
Writable volumes provide added flexibility for end users who are permitted to install software outside of the IT-delivered set of applications.
The UIA-only template ensures application installation and configuration data is stored, while profile data is managed using other technologies.
If a writable volume becomes corrupt, applications can be reinstalled without the risk of data loss.
Performance Testing for Writable Volumes
Writable volumes are read-write. Storage utilization patterns are largely influenced by user behavior with regard to desktop logins and logouts, user-installed applications, and changes to local user profiles. Group each set of similar users into use cases, and evaluate performance based on peak average use.
Additional Writable Volumes Operations
See the section Additional Configuration Options for Writable Volumes of App Volumes Configuration.
Multi-site Design Using Separate Databases
VMware recommends designing multi-site implementations using a separate-databases, or multi-instance model. This option uses a separate SQL Server database at each site, is simple to implement and allows for easy scaling if you have more than two sites. Additionally, latency or bandwidth restrictions between sites have little impact on the design.
In this model, each site works independently, with its own set of App Volumes Managers and its own database instance. During an outage, the remaining site can provide access to packages with no intervention required. For detailed information on the failover steps required and the order in which they need to be performed, refer to Failover.
Figure 9: App Volumes Multi-site Separate-Databases Option
This strategy makes use of the following components:
- App Volumes Managers – At least two App Volumes Manager servers are used in each site for local redundancy and scalability.
- Load balancers – Each site has its own namespace for the local App Volumes Manager servers. This is generally a local load balancer virtual IP that targets the individual managers.
Note: The App Volumes Agent, which is installed in virtual desktops and RDSH servers, must be configured to use the appropriate local namespace.
- Separate databases – A separate database is used for each site; that is, you have a separate Windows Server Failover Clustering (WSFC) cluster and an SQL Server Always On availability group listener for each site, to achieve automatic failover within a site.
- vCenter Server machine managers – The App Volumes Manager servers at each site point to the local database instance and have machine managers registered only for the vCenter Servers from their own site.
- Storage groups – Storage groups containing a common, non-attachable datastore can be used to automatically replicate packages from one site to the other. This common datastore must be visible to at least one vSphere host from each site.
Note: In some environments, network design might prevent the use of storage group replication between sites. See Copying an AppStack to another App Volumes Manager instance for more information about manually copying AppStacks and packages.
- Entitlement replication – To make user-based entitlements for packages available between sites, you can reproduce entitlements at each site. You can either manually reproduce the entitlements at each site or use the App Volumes Entitlements Sync tool, provided on the VMware Flings site. For instruction, see the App Volumes Configuration chapter. Manually reproducing entitlements can be somewhat streamlined by entitling packages to groups and OUs, rather than individuals.
Table 9: Strategy for Deploying App Volumes in Multiple Sites
App Volumes was set up in the second site used for Horizon (on-premises).
A separate database and App Volumes instance deployment option was used.
An NFS datastore was used as a common datastore among the storage groups to facilitate cross-site package replication.
This strategy provides App Volumes capabilities in the second site.
The separate-databases option is the most resilient, provides true redundancy, and can also scale to more than two sites.
With packages replicated between sites, the packages are available for use at both locations.
When installing and configuring the App Volumes Managers in a setup like this, each site uses a standard SQL Server installation.
- Install the first App Volumes Manager in Site 1. If using Always On availability groups to provide a local highly available database, use the local availability group listener for Site 1 when configuring the ODBC connection.
Important: For step-by-step instructions on this process, see App Volumes Configuration.
- Complete the App Volumes Manager wizard and add the vCenter Servers for Site 1 as machine managers, including mapping their corresponding datastores.
- Continue with installing the subsequent App Volumes Managers for Site 1. Add them as targets to the local load balancer virtual IP.
- Repeat steps 1–3 for Site 2 so that the App Volumes Managers in Site 2 point to the local availability group listener for Site 2, and register the local vCenter Servers for Site 2 as machine managers.
- For details on setting up storage groups for replicating packages from site to site, see the Recovery Service Integration section of Service Integration.
- Replicate package entitlements between sites, as described in the Replicate Application Packages and Reconstruct Relationships and Entitlements section of App Volumes Configuration.
With this design, the following is achieved:
- Packages are made available in both sites.
- Packages are replicated from site to site through storage groups defined in App Volumes Manager and through the use of a common datastore that is configured as non-attachable.
- User-based entitlements for packages are replicated between sites.
- A writable volume is normally active in one site for a given user.
- Writable volumes can be replicated from site to site using processes such as array-based replication. An import operation might be required on the opposite site. The order and details for these steps are outlined in Failover.
- Entitlements for writable volumes are available between sites.
In this model, each site works independently, with its own set of App Volumes Managers and its own database instance. During an outage, the remaining site can provide access to packages with no intervention required.
- The packages have previously been copied between sites using non-attachable datastores that are members of both sites’ storage groups.
- The entitlements to the packages have previously been reproduced, either manually or through an automated process.
In use cases where writable volumes are being used, there are a few additional steps:
- Mount the replicated datastore that contains the writable volumes.
- Perform a rescan of that datastore. If the datastore was the default writable volume location, App Volumes Manager automatically picks up the user entitlements after the old assignment information has been cleaned up.
- (Optional) If the datastore is not the default writable volume location, perform an Import Writable Volumes operation from the App Volumes Manager at Site 2.
All assignments to writable volumes are successfully added, but to the new valid location.
Recommended Practices for Production Environments
The following recommendations pertain to golden image optimization, setting up virtual desktops, and security best practices.
Golden Image Optimization
Golden images should be optimized for VDI or RDSH to ensure the best performance possible in a virtualized environment. Consider using the instructions in Creating an Optimized Windows Image for a VMware Horizon Virtual Desktop when building your golden images. The VMware OS Optimization Tool is referenced, and helps optimize Windows desktop and server operating systems for use with VMware Horizon.
The OS Optimization Tool includes customizable templates to enable or disable Windows system services and features, per VMware recommendations and best practices, across multiple systems. Because most Windows system services are enabled by default, the OS Optimization Tool can be used to easily disable unnecessary services and features to improve performance.
Desktop Pool Recommendations
When setting up client endpoint devices, consider the following best practices:
- When reverting a desktop VM that is running the App Volumes Agent to a previous snapshot, make sure that the VM is gracefully shut down, to avoid synchronization issues. This is primarily relevant to the packaging desktop and golden image VMs for Horizon linked- and instant-clone desktop pools.
- If you are using a Horizon desktop pool, the App Volumes Agent should be installed on the golden image VM for linked- and instant-clone pools, or on the VM template for full-clone pools, for ease of distribution.
- If using a Horizon linked-clone pool, make sure the Delete or Refresh machine on logoff policy in Desktop Pool Settings is set to Refresh Immediately. This policy ensures that the VMs stay consistent across logins.
App Volumes Security Recommendations
To support a large production environment, there are some important security configurations that administrators should put in place:
- Open only essential, required firewall ports on App Volumes Manager and SQL. Consider using an advanced firewall solution, such as VMware NSX®, to dynamically assign virtual machine firewall policies based on server role.
- Replace the default self-signed TLS/SSL certificate with a certificate for App Volumes Manager signed by a reliable certificate authority. See Replacing the Self-Signed Certificate in VMware App Volumes 2.12.
- Verify that App Volumes Manager can accept the vCenter Server certificate. App Volumes Manager communicates with vCenter Server over TLS/SSL. The App Volumes Manager server must trust the vCenter Server certificate.
- Note: It is possible to configure App Volumes Manager to accept an unverifiable (self-signed) certificate. Navigate to Configuration > Machine Managers in the management console. Each machine manager (vCenter Server) has a Certificate option that shows the current status of the certificate and allows an administrator to explicitly accept an unverifiable certificate.
- Consider using ThinApp to package applications, to take advantage of the security benefits of isolation modes, when required. Each ThinApp package can be isolated from the host system, and any changes, deletions, or additions made by the application to the file system or registry are recorded in the ThinApp sandbox instead of in the desktop operating system.
- For writable volumes, determine which end users require ongoing administrator privileges. Writable volumes with user-installed applications require that each desktop user be assigned local computer administrator privileges to allow the installation and configuration of applications.
- Some use cases could benefit from temporary, request-based elevated privileges to allow incidental application installation for a specific user or user group. Carefully consider the security risks associated with granting users these elevated privileges.
- Create and use an AD user service account specifically for App Volumes. This is good security hygiene and a forensics best practice. It is never a good idea to use a blanket or general-purpose AD user account for multiple purposes within AD.
- Consider creating an administrative role in vCenter Server to apply to the App Volumes service account.
Installation and Initial Configuration
Installation prerequisites are covered in more detail in the System Requirements section of the VMware App Volumes Installation Guide. The following table lists the versions used in this reference architecture.
Table 10: App Volumes Components and Version
VMware vSphere 7.0 Update 1
VMware vCenter 7.0 Update 1
App Volumes Manager
Windows Server 2019
2016 Functional Level
SQL Server 2016
OS for App Volumes Agent
Windows 10 and Windows Server 2019
Refer to the VMware App Volumes Installation Guide for installation procedures. This document outlines the initial setup and configuration process.
After installation is complete, you must perform the following tasks to start using App Volumes:
- Complete the App Volumes Initial Configuration Wizard (https://avmanager).
- Install the App Volumes Agent on one or more clients and point the agent to the App Volumes Manager address (load-balanced address).
- Select a clean packaging system and create an application package. See Working with Applications and Working with Packages in the VMware App Volumes Administration Guide for instructions.
- Assign the package to a test user and verify it is connecting properly.
- Assign a writable volume to a test user and verify it is connecting properly.
Now that you have come to the end of this chapter, you can return to the landing page and search or scroll to select your next chapter in one of the following sections:
- Overview chapters provide understanding of business drivers, use cases, and service definitions.
- Architecture chapters explore the products you are interested in including in your platform, including Workspace ONE UEM, Workspace ONE Access, Workspace ONE Assist, Workspace ONE Intelligence, Horizon, App Volumes Dynamic Environment Manager, and Unified Access Gateway.
- Integration chapters cover the integration of components and services you need to create the platform capable of delivering what you want.
- Configuration chapters provide reference for specific tasks as you build your platform, such as installation, deployment, and configuration processes for Horizon, App Volumes, Dynamic Environment Management, and more.