Track anything using ESP

I’ve talked in the past about how the Enrollment Status Page in Windows Autopilot can be extended using additional providers. The basic structure is described in the EnrollmentStatusTracking CSP (using the old name for ESP, which would have been EST instead of ESP, not that ESP is any better than EST). Using this, other software can hook into the process to extend the default MDM-based capabilities so that tracking and blocking can look at “other stuff.” That’s used for Intune’s Win32 apps (delivered using Intune Management Extensions, a.k.a. Sidecar) and for Configuration Manager’s task sequence integration (described in my previous blog posts here and here). Each additional provider can tell ESP about other “apps” that should be tracked. (Only apps are supported, not policies, certs, or anything else ESP would normally look at, but that doesn’t really matter since you can call anything an app, e.g. a task sequence.)

The policy settings that define these providers are sent to the client as part of the enrollment process, as part of the WAP payload. I covered that in some of my previous presentations on the Windows Autopilot flow:

In the image above, you can see one provider, BabyMDMProvider (a fake one I created as part of a custom MDM service I used for testing) in the “EnrollmentStatusTracking” node. Typically you’d see “Sidecar” and “ConfigMgr” in that list (where “ConfigMgr” is only added when you’ve created a “Co-management settings” object that says to install the ConfigMgr agent — without that it would just be “Sidecar”).

My assumption at the time was that the only possibility for registering a new provided was by having the MDM service add another one to the list it sends in the enrollment payload, which would put you at the mercy of the MDM service itself — it’s not like I can make a change to the Intune MDM service to have it add things to this list. But what would happen if something that Intune delivers, e.g. an MSI, adds another provider to the list? To find out, I created a simple PowerShell script that created a new provider using the WMI bridge (even though the WMI bridge documentation doesn’t admit that such a capability exists):

$policyProviderClass = Get-CimClass -Namespace rootcimv2mdmdmmap -ClassName MDM_EnrollmentStatusTracking_PolicyProviders02_01
$pp = New-CimInstance -CimClass $policyProviderClass -Property @{
  InstanceID = "WaitFor"; ParentID="./Vendor/MSFT/EnrollmentStatusTracking/DevicePreparation/PolicyProviders" }

If I run that before the enrollment happens, I can see that ESP is indeed waiting for that provider to install and tell it what to track. Since I haven’t written that part, it will eventually time out, but it at least proves out that the enrollment process won’t overwrite what I added.

The next test is to try to add that policy provider after the enrollment has completed, e.g. when ESP is waiting for Sidecar and ConfigMgr to finish installing and telling ESP what to track. What would happen if I add a new policy provider during that time? Using the same script, I could verify that yes, ESP will notice it and start tracking it even after ESP has started. (I don’t know if that is clever coding or lazy coding — it’s polling the list every time it updates.) So as long as I can get my provider into the list before ESP stops waiting, e.g. before Sidecar and ConfigMgr finish their installation and notify ESP that they are good to go, I can add a new provider from an MSI delivered by Intune. This MSI would typically install before Sidecar has figured out what needs to be tracked (and it would certainly finish well before the ConfigMgr agent is installed). Yes, it’s a bit of a race, but it should work.

So, package up an appropriate PowerShell script into an MSI, deliver that via Intune (not as a Win32 app, that would likely get installed too late), and then have another script (delivered via Intune, ConfigMgr, or something else, doesn’t really matter) later notify ESP that it is done should work fine.

To make this work in a “real world” scenario wasn’t quite as straightforward. See the “Gotchas” section below. But in the process, I switched from using WMI bridge classes to the LocalMDM PowerShell module instead. This sends the needed OMA-URI commands to Windows.

So what are the pieces? Here’s an overview:

• WaitFor.msi. This packages up two scripts, Initialize.ps1 that is executed by the MSI itself when it installs, and a second Done.ps1 that is executed by some task to indicate that the task is complete and ESP can stop waiting.
• An Intune MSI LOB app that installs the WaitFor.msi, executing Initialize.ps1 to create the policy provider settings, including a single (dummy) app that ESP should track.
• Something to run Done.ps1 to signal that the app is complete. In my case, I added an app to Tanium Deploy to run the script, then added that app to the end of a software bundle (a list of apps to install in order).

The scripts and related source files can be found on GitHub, as can the latest MSI.

Let’s go through each of those in more detail.

WaitFor.msi

As far as MSIs go, this one is pretty simple: It installs the two script files, Initialize.ps1 and Done.ps1, into “C:Program FilesOofhours WaitFor”, and then runs a custom action that uses PowerShell.exe to execute Initialize.ps1 (running elevated as LocalSystem). It doesn’t even have a UI — if you run the MSI manually, all you see is a progress dialog as it does its thing.

This leverages Wix v4, available from https://wixtoolset.org/, as well as the related Heatwave product from https://www.firegiant.com/docs/heatwave/ to support using Visual Studio to manage and build the MSI.

When setting this up in Intune, you just need to create a “Windows MSI line-of-business” app and deploy it to a group containing your Autopilot devices. The setup is simple:

Just make sure it runs in the device context (it *should* be impossible to specify otherwise).

Initialize.ps1

This script, executed by the MSI, uses the EnrollmentStatusTracking CSP to register the CSP, create an app to be tracked, and tell ESP that it’s ready to go. As mentioned previously, it uses the LocalMDM PowerShell module to do its work.

Every policy provider has a name (e.g. “Sidecar”, “ConfigMgr”). For this one, a random name is generated that starts with “WaitFor” (more on “why random” below).

Done.ps1

This script does the eventual notification to ESP that the (dummy) app that it is tracking is done installing. This is designed to be executed by another management tool or other external process. It uses LocalMDM to set the status of the app to “installed.”

In my case, I set this up as a Tanium Deploy app. It requires no source files, just a PowerShell command to run the script with appropriate command line options:

I can then add it to a software bundle that specifies a list of apps to install (in order), where the above app is at the end of the list:

So, after Tanium installs the previous eight apps, it will notify ESP that the dummy app is done (so this “single app” is really eight apps — make sure your ESP timeout is set appropriately).

The whole experience

So what does this look like end-to-end? Here’s a video, with all the delays edited out so that you can see it in a reasonable amount of time. This is a Windows Autopilot user-driven AAD deployment, where Intune deploys the WaitFor MSI and the Tanium client as a Win32 app; once the Tanium client initializes, it installs the eight apps listed above and then executes the Done.ps1 script to tell ESP that everything is good to go. The total time before editing was just under an hour.

It’s not terribly interesting since it looks like a standard Autopilot device ESP display, and it is — but that’s the point, we’re just adding one more app (which is actually a list of apps) into the tracking list, letting the Tanium client take care of those and then notify ESP when it is done. You can see a sign of one of those apps, the Chrome icon on the desktop, but all the others are there too (manually pinned, just to easily show the apps):

(If you were paying attention, I said 8 apps, but there are only six above. I had one app, Visual Studio Code, in the list twice, and another, Adobe Acrobat DC, only said to update, not to install. I should probably fix those.)

In a perfect world, the policy provider would let you specify how many apps you wanted to create/track, and the “done” script would let you signal each one of them, but I’ll leave that as an exercise for a later date.

Not surprisingly, Tanium applied Windows updates during that time too and notified me to reboot, so there were other things going on in the background too:

Gotchas

There was a surprising amount of pain putting this together, and that pain wasn’t directly related to the actual policy provider work. The first problem I ran into was with uploading new versions of the WaitFor.msi for testing. Intune would report that they were uploaded fine, but the client would continue to run the old version. It appears that the Intune portal session was quietly timing out, so even though it said the LOB app change was made and the new version uploaded, it actually wasn’t. To work around that issue, I just made sure I refreshed the Intune portal before making any changes to the MSI app. If it asked me to sign in again after refreshing the page, then I know the previous session timed out (and wouldn’t have worked); if it doesn’t ask me to sign in again, then the previous session was still good. Either way, I would then have 100% success with adding new versions (which I checked in the client registry, HKLMSoftwareMicrosoftWindowsCurrentVersionUninstall, until I was confident it was working reliably).

The next problem was even worse: Everything would start working, and then stop. What do I mean by that? I would watch the ESP diagnostics registry key (HKLMSoftwareMicrosoftWindowsAutopilotEnrollmentStatusTracking) and everything would progress fine: I would see my policy provider show up with an InstallationState = 1 for “install pending”, then it would switch to InstallationState = 3 for “installed”, then the app to track would be created and ESP would be notified. Usually that would then result in ESP showing three apps to install (Tanium client, WaitFor MSI, and my custom dummy app) — for a short period of time. Then, the number would change; instead of seeing “0 of 3, 1 of 3, 2 of 3, 3 of 3” I would typically see “0 of 3, 1 of 3, 2 of 3, 2 of 2”. My dummy app, and really my entire policy provider, would disappear.

After a lot of time troubleshooting, I could see (using Oliver Kieselbach’s SyncML Viewer) that the device was receiving a policy from Intune that was changing the state of my WaitFor policy provider back to InstallationState = 1. But why? Intune should not even be aware of this policy, since it was set via LocalMDM. It appears that is a result of a different CSP, NodeCache. Intune uses this to notice when policies drift; if it sees a change it will change it back. NodeCache saw my initial value (InstallationState=1) and reported that to Intune when it asked; Intune was later told by NodeCache that the value changed to 3, so Intune sent a new policy down to set it back to 1. How rude.

Weirdly, reverting the VM back to a pre-enrolled state and going through the process again always had the same result: I’d set the value to 3, Intune would change it back to 1. To test, I changed the name of my policy provider from WaitFor to XXXX and repeating the process (rolling back, re-enrolling, monitoring the value) worked fine — but Intune still tried to reset the previous WaitFor policy provider InstallationState value back to 1, even though there was now no such thing. So Intune remembered the old value, likely tied to the Intune device object that is being reused through each iteration I went through.

To work around this mess, I just changed to a random policy provider name (“WaitFor” + a random number between 1 and 99999). Now everything works fine. With way too much time spent troubleshooting.

There was also some fun figuring out how to run an elevated PowerShell script from a Wix v4-created installer. It worked differently from the previous Wix v3 samples found on the web. At some point I’ll do a blog post on those details just to save others some time. It’s not that bad, it’s just different than it used to be.

One other random gotcha to add to the list: Sometimes Intune re-runs the WaitFor.msi installer. It doesn’t happen very often, but it has happened more than once, no idea why. Each time, the Initialization.ps1 PowerShell script runs. That wasn’t expected, so it would end up generating a new random provider name and then the Done.ps1 notification would use the wrong one. I had to add logic to check to see if we already did this once, and if so, don’t do it again. It’s not worth any further investigation, given the rarity of the issue.