windows · powershell · windbg
My script is hanging. What is it waiting for?
For this one, assume your script runs a command and captures the output. Normally it takes a second. This time the last progress message stays on screen for ages. The CPU looks quiet, and nothing comes back.
To take a look, we’ll attach to PowerShell and find the call that hasn’t returned.
Get set up
You’ll need Windows 11 x64 and WinDbg. Use Windows PowerShell 5.1, powershell.exe, for the commands below. It runs on .NET Framework; PowerShell 7 uses a different runtime.
Extract the lab ZIP to C:\Lab\ScriptHangLab. It includes the native source, executable, matching PDB, and PowerShell scripts. The README covers rebuilding and downloaded-script policy. No administrator access is needed. The helper exits after 15 minutes if left running.
Open two PowerShell windows in that directory. In the first:
Set-Location C:\Lab\ScriptHangLab
.\Run-Lab.ps1 -Mode inherited-writer -RunDirectory C:\Lab\ScriptHangLab\runs\broken
Use a new or empty directory just to be safe. In the second window, also run Set-Location C:\Lab\ScriptHangLab; we’ll use it while the first is blocked.
The demo reaches this point (with setup lines omitted):
Wrapper PID: 24864
Collecting command output...
The script prints its PID so we can attach to the right powershell.exe.
Where has PowerShell stopped?
The wrapper starts a System.Diagnostics.Process with UseShellExecute=false and RedirectStandardOutput=true. Later it does this:
$output = $process.StandardOutput.ReadToEnd()
$process.WaitForExit()
If ReadToEnd hasn’t returned, we haven’t reached WaitForExit. Let’s check.
Attach WinDbg to the wrapper PID: File → Attach to process. Then run:
.symfix C:\Lab\Symbols
.reload
.loadby sos clr
~*e !clrstack
SOS shows the managed frames that an ordinary native stack can miss. .loadby sos clr loads the extension beside the target’s Framework CLR. Some threads won’t have a managed stack; look for the one in ReadToEnd.
In this capture it was OS thread 6184. The relevant managed calls were these, with addresses and argument signatures omitted:
Microsoft.Win32.Win32Native.ReadFile
System.IO.FileStream.ReadFileNative
System.IO.FileStream.ReadCore
System.IO.FileStream.Read
System.IO.StreamReader.ReadBuffer
System.IO.StreamReader.ReadToEnd
Select that thread and include its arguments:
~~[6184]s
!clrstack -a
The selected thread was stopped in ntdll!NtReadFile+0x14. ReadToEnd was waiting for another read to complete. Its this argument gave us the reader object:
System.IO.StreamReader.ReadToEnd()
PARAMETERS:
this (<CLR reg>) = 0x0000022065e63320
Use the OS thread ID and object addresses from your own output. Debugger thread indexes can change between attaches, and optimized arguments may show <no data>. Here SOS recovered the references we needed.
What are we reading?
Inspect the reader with SOS’s !do, short for DumpObj:
!do 0000022065e63320
Its stream field pointed to 0000022065e632b8, a System.IO.FileStream:
!do 0000022065e632b8
The useful fields were:
| Field | Value |
|---|---|
_isPipe |
1 |
_canRead |
1 |
_canWrite |
0 |
_handle |
0000022065e62b18 |
So this FileStream wraps a pipe reader. Follow _handle to the SafeFileHandle object:
!do 0000022065e62b18
!handle 748 f
The object’s handle field was 748, the native handle value. !handle reported a File object with read access. But! Pipes can appear under that object type and it doesn’t automatically imply a file on disk. These private fields belong to this Framework version, so follow their names and values rather than assuming fixed offsets.
When .NET redirects stdout, it gives the command the write end of a pipe and keeps the read end. The Framework Process source shows that setup. StreamReader keeps asking for data; FileStream eventually calls native ReadFile.
An empty pipe isn’t necessarily finished. Windows waits until all writer handles have closed before the reader can reach its end. After buffered bytes have been consumed, ReadFile reports ERROR_BROKEN_PIPE; Framework treats that as EOF and lets ReadToEnd return.
The command can therefore exit while some other process still owns a writer. Adam Sitnik describes this exact child/grandchild inheritance problem in the .NET team’s Process API article. We’re following it through a Framework process here.
Which command did the script launch?
We have a reader, but its handle doesn’t name the process it was created for. We can look for the object retaining that reader:
!gcroot -nostacks 0000022065e63320
!gcroot follows references from garbage-collection roots to an object. -nostacks leaves out stack roots. Most of this chain was PowerShell plumbing. Look for the Process just before the reader:
-> 0000022065e19b68 System.Management.Automation.PSVariable
-> 0000022065e170d8 System.Management.Automation.PSObject
-> 0000022065e16f28 System.Diagnostics.Process
-> 0000022065e63320 System.IO.StreamReader
Inspect that Process:
!do 0000022065e16f28
Its standardOutput field matched 0000022065e63320, our blocked reader. Nice. Its processId was 24368. That identifies the command associated with this stream, independently of the lab’s metadata.
Detach with qd or Debug → Detach Debuggee. In the second PowerShell window, check the recovered PID and look for processes it created:
Get-CimInstance Win32_Process -Filter 'ProcessId=24368'
Get-CimInstance Win32_Process -Filter 'ParentProcessId=24368' |
Select-Object ProcessId, ParentProcessId, CreationDate,
ExecutablePath, CommandLine |
Format-List
PID 24368 was no longer listed. The parent query returned one process, 20176, running the lab executable with helper inherited-writer in its command line. Its creation time fit this launch.
ParentProcessId gives us candidates, not proof of a pipe connection. PIDs can be reused, so we have to check creation times and paths too. On a busy machine there may be several descendants to investigate, including another generation below the direct child.
Inspect the candidate’s stdout
Attach WinDbg to PID 20176. We can find its current standard-output handle using Windows symbols, without the program’s private PDB:
.symfix C:\Lab\Symbols
.reload /f ntdll.dll
dt ntdll!_PEB @$peb ProcessParameters
That reported:
+0x020 ProcessParameters : 0x000001d1`96839d20 _RTL_USER_PROCESS_PARAMETERS
Use that address to inspect the process parameters:
dt ntdll!_RTL_USER_PROCESS_PARAMETERS 000001d196839d20 StandardOutput
That gave us the stdout handle:
+0x028 StandardOutput : 0x00000000`00000a54 Void
Inspect it with !handle a54 f. The shortened output showed write access:
Handle a54
Type File
Attributes 0
GrantedAccess 0x120196:
ReadControl,Synch
Write/Add,Append/SubDir/CreatePipe,WriteEA,ReadAttr,WriteAttr
No Object Specific Information available
The PEB and process-parameter layouts are Windows internals; these commands use the types resolved for this build. StandardOutput is also only the current slot. A program may have saved or duplicated an earlier handle elsewhere.
Neither !handle inspection supplied a useful pipe name. The access flags alone don’t prove that this writer connects to our reader. Matching handle numbers across processes wouldn’t prove it either: each process has its own handle table.
Close the writer and leave the helper running
Detach from the helper first. Then use the lab’s control endpoint from the second PowerShell window:
.\Control-Lab.ps1 -RunDirectory C:\Lab\ScriptHangLab\runs\broken -Action CloseWriter
.\Control-Lab.ps1 -RunDirectory C:\Lab\ScriptHangLab\runs\broken -Action Status
The control script uses the run directory’s metadata to reach the helper. It asks the helper to close its stdout writer and waits for an acknowledgement, leaving the helper running.
The blocked wrapper returned:
Output collection completed.
The native command wrote one line and exited.
Native exit code: 0
The status check still showed the same helper:
Native child PID: 24368 (exited)
Helper PID: 20176 (running)
Ready=true
WriterClosed=true
StopRequested=false
There’s the hang. The command exited, but its helper kept stdout open. We closed that writer and PowerShell returned, while the same helper, PID 20176, stayed alive.
CloseOutputWriter in the native source clears the standard-output slot with SetStdHandle and releases the actual handle with CloseHandle. Changing the slot alone wouldn’t close an existing handle. The helper then returns to waiting for a stop request.
We’re done with this helper now:
.\Control-Lab.ps1 -RunDirectory C:\Lab\ScriptHangLab\runs\broken -Action Stop
Fix the launch when you control the code
The sample’s LaunchHelper explicitly passes its existing stdout to the helper. Here’s the relevant setup, with validation and other startup fields omitted:
gOutputWriter = GetStdHandle(STD_OUTPUT_HANDLE);
if (!SetHandleInformation(gOutputWriter,
HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT))
Fail("SetHandleInformation(stdout)");
startup.dwFlags = STARTF_USESTDHANDLES;
startup.hStdOutput = gOutputWriter;
It then calls CreateProcessW with handle inheritance enabled. The helper gets a handle to the same write end. It doesn’t write anything, but it still owns the handle.
The fixed branch doesn’t supply those standard handles and passes FALSE for bInheritHandles. This helper needs no standard I/O; it opens its control events by name. Both branches use DETACHED_PROCESS, so detaching from the console wasn’t enough to prevent the broken branch from receiving the explicitly supplied writer.
The fixed run collected the same line and returned native exit code 0 while its helper stayed running. That helper never received the captured-output writer. The README has the replay commands and the handle-list option for helpers that do need inherited handles.
What if it’s a vendor executable?
Check the vendor’s supported logging and wait options first. Honestly, the debugger is hard work - often a tool may offer a log file, or a switch that keeps the original process around until the operation finishes. Use that if it exists and save yourself a lot of time.
If the wrapper only needs the direct process’s exit and a record of its output, give it real files for stdout and stderr. There is no pipe reader waiting for EOF.
Windows PowerShell 5.1 can do this with Start-Process. Use -PassThru to get its Process object and make a finite wait on that object. Leave off -Wait here: that switch waits for descendants too.
Try the same inherited-writer mode that hung the original wrapper:
$id = [guid]::NewGuid().ToString('D')
$logs = 'C:\Lab\ScriptHangLab\file-run-' + $id
$null = New-Item -ItemType Directory -Path $logs -ErrorAction Stop
$arguments = 'child inherited-writer ' + $id + ' "' + $logs + '"'
$p = Start-Process `
-FilePath .\bin\ScriptHangLab.exe `
-ArgumentList $arguments `
-RedirectStandardOutput (Join-Path $logs 'stdout.log') `
-RedirectStandardError (Join-Path $logs 'stderr.log') `
-WindowStyle Hidden -PassThru
try {
$null = $p.Handle
if (-not $p.WaitForExit(10000)) {
throw "Wait timed out for PID $($p.Id). Logs: $logs"
}
$exitCode = $p.ExitCode
if ($null -eq $exitCode) { throw 'No exit code was retained.' }
"Exit code: $exitCode"
Get-Content -LiteralPath (Join-Path $logs 'stdout.log')
} finally { $p.Dispose() }
Opening $p.Handle keeps a handle to that process; the finally releases it. Disposing the object doesn’t terminate the process. The ten-second limit applies to the wait after launch. If it expires, the example throws and leaves the process running.
This returned exit code 0 and the same 47 bytes of output while helper 26580 stayed alive. Its writer was still open, but the handle’s file type was FILE_TYPE_DISK. The program was still passing stdout on to the helper.
Stop this lab helper when you’ve checked the logs, using the $id from the example:
& .\bin\ScriptHangLab.exe control $id stop
For your command, substitute its executable, arguments, and working directory. Keep the vendor’s exit-code contract too. A missing exit code should be an error, never an assumed zero.
The download also includes a reusable file-log runner. Its API calls, parameters and timeout behaviour are in the README.
The exit code is for the process we launched. Its helpers can still be working and writing to the logs. If you’re running an installer, check how the vendor tells you the installation is finished.
Closing the helper’s stdout handle let our original script finish while the helper kept running.
Captured on Windows 11 Business x64 26300.9711, Windows PowerShell 5.1.26100.9711, CLR 4.8.10008.0, and SDK CDB 10.0.26100.6584. CDB uses the same debugger commands as WinDbg. The native lab uses Visual Studio 2022 C++ tools with optimisations disabled (/Od) and private symbols (/Zi); the discovery above used Windows symbols and SOS without loading the lab PDB.