Doyensec Researchers Run Code Inside Windows SmartScreen Enclave; Microsoft Patched Flaw Sept. 8

Researchers at the security firm Doyensec ran their own code inside a part of Windows designed to stay sealed off even if the rest of the machine is compromised. Microsoft patched the underlying flaw on September 8, 2026. Doyensec published its research on Tuesday, Oct. 13, more than a month later.

The work was done in a lab as a proof of concept. It has not been seen in real attacks.

What was broken

The target was a VBS enclave, an isolated slice of memory Windows uses to guard secrets. An enclave is a signed library that a normal program loads but cannot read or change while it runs.

According to Microsoft, the Windows hypervisor keeps that memory in a more trusted world, called VTL1 or Isolated User Mode. Ordinary code in the normal world, VTL0, cannot reach it. The design is supposed to hold even if an attacker has already taken over the normal kernel or is running as SYSTEM.

That is why running attacker-chosen code inside it is notable.

The specific enclave Doyensec attacked is SFAPE.dll. It powers Enhanced Phishing Protection in Microsoft Defender SmartScreen. The feature watches text a user types or pastes and warns them if they enter their Windows sign-in password somewhere unsafe, such as Notepad or a phishing page.

To check that without exposing the password, Windows intercepts it at login and stores a hashed form inside the enclave. Later inputs are compared against it. On Windows 11, the program that loads this enclave is lsass.exe, the process that handles sign-in credentials and runs as SYSTEM.

How the attack worked

Doyensec started with a small mistake and widened it. Calling the enclave's Initialize function a second time makes it write a fixed four-byte error value to an address the calling program controls. On that second call, the enclave skips the address check the first call performs.

Doyensec does not oversell this. The firm wrote that calling it an arbitrary write "would be a huge overstatement."

Aimed at the right spot, though, that single write corrupts the value the enclave uses to tell its own memory from the host's. The safety check then stops working. Doyensec then combined that with a timing bug to get code running inside the enclave.

Microsoft's position and the limits of the attack

Microsoft tracks the flaw as CVE-2026-83498. It rates the issue an elevation-of-privilege vulnerability, Critical, with a CVSS score of 7.8.

The company says the flaw was not exploited and was not publicly known before the fix. It also says it found the same issue internally. Doyensec's research does not claim otherwise.

The attack does not work from a distance. It requires a program already running on the machine that loads and calls the enclave. For SFAPE, that program is LSASS.

Microsoft's own rating says the same thing: the attack vector is local, and the attacker needs low-level access but no help from the user. This is a route for code that already has a foothold to reach into the more trusted world beside it. It is not a way in from outside.

What it means

VBS enclaves exist for one scenario: the normal operating system is already lost, and secrets still need to stay protected. A bug that lets code cross into VTL1 undercuts that promise, even if the practical path to it is narrow.

The fix has been available since September 8. Doyensec held its write-up until after the patch shipped, and Microsoft says it saw no exploitation in the meantime.

What is still unclear is how many Windows 11 machines have not yet installed the September update. Those are the only systems where the lab technique remains relevant.