Phidiax Tech Blog

After installing KB5005076 on Windows Server 2012 R2 (or any of the cumulative security updates that follow it as they include this update), IIS application pools which are set to allow 32 bit mode begin to crash, writing BEX errors (Buffer Overflow Exception) to the event log, referencing either w3wp or regsvr32 as the faulting process.

In the beginning of troubleshooting, we used uninstalling this KB/security patch (or running all IIS pools in 64 bit mode) to fix the issue. 

Faulting application name: w3wp.exe, version: 8.5.9600.16384, time stamp:

Faulting module name: ntdll.dll, version: 6.3.9600.20144, time stamp: Dx60eg12D8

Exception code: 0xc000005

Fault offset: 0x0002a8c0

Faulting application path: C:\Windows\SYSWOW64\inetsrv\w3wp.exe

Faulting module path: C:\Windows\SYSTEM32\ntdll.dll

UPDATE:

In this case, the root cause turned out to be a negative interaction between two separate detection/anti-malware utilities running on the servers. Their negative interaction only became apparent following this Windows Update, and was repaired by adding exceptions to each tool to avoid interference. The crash report generated by Windows allowed us to determine what libraries were being loaded by the crashing process. Evaluating this list led us to determine that both of these anti-malware components were using native API hooks and thus were both loaded by w3wp.exe. 

We performed "process of elimination" debugging by disabling each anti-malware tool individually, the 32 bit applications hosted as normal when one or the other was running. Only when both were enabled did the 32 bit crashes begin again. 

Sample subset of Crash Report (specific anti-malware DLL names are redacted):

EventType=BEX

WOW64=1

NsAppName=w3wp.exe

Response.type=4

Sig[0].Name=Application Name

Sig[0].Value=w3wp.exe

Sig[1].Name=Application Version

Sig[1].Value=8.5.9600.16384

Sig[2].Name=Application Timestamp

Sig[2].Value=52157ba0

Sig[3].Name=Fault Module Name

Sig[3].Value=ntdll.dll

Sig[4].Name=Fault Module Version

Sig[4].Value=6.3.9600.20144

Sig[5].Name=Fault Module Timestamp

Sig[5].Value=615be385

Sig[6].Name=Exception Offset

Sig[6].Value=0002a8c0

Sig[7].Name=Exception Code

Sig[7].Value=c0000005

Sig[8].Name=Exception Data

Sig[8].Value=00000008

DynamicSig[1].Name=OS Version

DynamicSig[1].Value=6.3.9600.2.0.0.272.7

DynamicSig[2].Name=Locale ID

DynamicSig[2].Value=1033

UI[2]=C:\Windows\SysWOW64\inetsrv\w3wp.exe

UI[5]=Check online for a solution (recommended)

UI[6]=Check for a solution later (recommended)

UI[7]=Close

UI[8]=IIS Worker Process stopped working and was closed

UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available.

UI[10]=&Close

LoadedModule[0]=C:\Windows\SysWOW64\inetsrv\w3wp.exe

LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll

LoadedModule[5]=C:\Windows\SYSTEM32\antimalware2.dll

LoadedModule[6]=C:\Windows\SYSTEM32\KERNEL32.dll

LoadedModule[7]=C:\Windows\SYSTEM32\KERNELBASE.dll

LoadedModule[8]=C:\Windows\SYSTEM32\antimalware1.dll

LoadedModule[9]=C:\Windows\SYSTEM32\ADVAPI32.dll

LoadedModule[10]=C:\Windows\SYSTEM32\msvcrt.dll

LoadedModule[11]=C:\Windows\SYSTEM32\combase.dll

LoadedModule[12]=C:\Windows\SYSTEM32\inetsrv\iisutil.dll

LoadedModule[13]=C:\Windows\SYSTEM32\sechost.dll

LoadedModule[14]=C:\Windows\SYSTEM32\RPCRT4.dll

LoadedModule[15]=C:\Windows\SYSTEM32\WS2_32.dll

LoadedModule[16]=C:\Windows\SYSTEM32\SspiCli.dll

LoadedModule[17]=C:\Windows\SYSTEM32\pcwum.DLL

LoadedModule[18]=C:\Windows\SYSTEM32\NSI.dll

LoadedModule[19]=C:\Windows\SYSTEM32\CRYPTBASE.dll

LoadedModule[20]=C:\Windows\SYSTEM32\bcryptPrimitives.dll

FriendlyEventName=Stopped working

ConsentKey=BEX

AppName=IIS Worker Process

AppPath=C:\Windows\SysWOW64\inetsrv\w3wp.exe