Before you begin
Why this matters
Haskell programs often behave differently in GHCi, on a screen, and when another program reads their output: a prompt that appears in one and not another, or a last line missing from a file. All of it follows from one setting on each output: when it sends the text it holds. Once you can ask for that setting, you can predict the output.
What this helps you reason about
- Predict whether written text is visible yet.
- Explain why two outputs writing to one place can produce lines in a different order from the code.
- Spot an output that must be closed before the program ends.
When you call putStrLn, you usually see the line at once, and for most small programs that model is correct. In Haskell, every Handle carries its own buffering mode, and that mode depends on where the handle points and how the program is run. This lesson follows text through a handle’s buffer and shows when it arrives late, out of order, or not at all.
Already know this? The short version
- Each Handle has a buffering mode, NoBuffering, LineBuffering, or BlockBuffering, and hGetBuffering reports it.
- A compiled GHC program gives stdout LineBuffering on a terminal and BlockBuffering on a pipe or file, and gives stderr NoBuffering; GHCi, which the stepper uses, sets stdout to NoBuffering.
- hFlush, a newline under LineBuffering, a full buffer, and hClose flush; when a compiled program ends, GHC flushes stdout and stderr, even after exitWith, but not other handles, and finalizers are no substitute for hClose.
- A putStr prompt needs hFlush stdout; flush stdout before writing to stderr when both go to one file; hFlush is not a disk sync.
Words in this lesson (8)
- operating system
- The software, such as Linux, macOS, or Windows, that runs programs and controls the screen, files, and other devices. Every write a program makes is a request to it.
- stream
- A destination that receives bytes in order. A program writes to streams; it does not draw on the screen itself.
- Handle
- Haskell’s value for an open stream or file. stdout and stderr are Handles, and each Handle has its own buffer and buffering mode.
- standard output and standard error
- The two output streams every program starts with. Standard output is for normal results; standard error is for messages about problems.
- terminal and pipe
- A terminal is the window where you type commands. A pipe is a connection that carries one program’s output into another program’s input.
- buffer
- An area of memory where written text waits inside the program before it is sent on to the operating system.
- buffering mode
- How a Handle’s buffer behaves: NoBuffering sends text at once, LineBuffering at each newline, and BlockBuffering when the buffer is full.
- flush
- To send the text waiting in a buffer on to the operating system now. A flush is not a disk write.
Examples are focused snippets; imports, enclosing functions, or classes may be omitted. C examples use C11; Haskell examples use Haskell 2010. Expand the notes for language-specific depth.
01 / A write call
What does a print call promise?
Here is a useful model: putStrLn writes to the screen immediately. For most small programs, it predicts exactly what you see.
Underneath, a program does not draw on the screen itself. It asks the operating system to write bytes to a stream, and the operating system delivers them.
Every program starts with two output streams. Standard output is for the program’s normal results; putStrLn writes there. Standard error is for messages about problems; hPutStrLn stderr writes there.
The program does not choose where these streams go. Whoever starts the program chooses. A stream may go to a terminal, to a file, or through a pipe into another program.
In Haskell, an open stream is a Handle. stdout and stderr are the handles for the two standard streams.
import System.IO
main :: IO ()
main = do
mode <- hGetBuffering stdout
putStr "loading... "
print mode
putStrLn "ready"A write passes text to a stream. The program does not choose where that stream goes.
Precisely: what the stepper changes
The stepper runs Haskell in GHCi, the interactive GHC interpreter, and GHCi sets stdout to NoBuffering, so every write is sent at once. A compiled program starts with other modes (see Terminals and pipes). Changing stdout’s buffering in the stepper stops the trace, because GHCi writes its own prompt through the same handle and the stepper waits for that prompt.
Experiment
Watch when each write reaches the output panel
The stepper runs Haskell in GHCi, and GHCi sets stdout to NoBuffering, so every write is sent at once. A file opened with openFile gets block buffering by default; a compiled program writing stdout to a pipe gets block buffering too.
Why it matters A Handle carries its own buffering mode. Ask hGetBuffering instead of assuming; the answer depends on where the handle points and on how the program is run.
Predict which buffering mode the program reports for stdout in the stepper. Then add notes <- openFile "notes.txt" WriteMode as the first line of the do block, ask hGetBuffering notes instead of hGetBuffering stdout, and predict again.
import System.IO
main :: IO ()
main = do
mode <- hGetBuffering stdout
putStr "loading... "
print mode
putStrLn "ready"
What to look for
Watch the output panel at each stop. loading... appears as soon as putStr runs, and the mode printed after it is NoBuffering. GHCi stops where actions run, so the output follows the do block in order.
Expected output
loading... NoBuffering
ready02 / Buffers
Where does written text wait?
Asking the operating system to write takes time, much more than building a string. So a Handle has a buffer: an area of memory where written text waits until there is enough to send at once. hSetBuffering chooses how the buffer behaves: NoBuffering, LineBuffering, or BlockBuffering, which holds text until the buffer is full.
So the model from section 01, that a print writes to the screen immediately, needs one addition: written text can still be inside the program. A different handle writing to the same output can arrive first. The experiment makes a second handle to standard output with hDuplicate, from GHC.IO.Handle in base, and gives it block buffering.
out <- hDuplicate stdout
hSetBuffering out (BlockBuffering Nothing)
hPutStrLn out "1 queued" -- waits in out
putStrLn "2 direct" -- stdout: no buffer in GHCi
hFlush outA buffer holds written text inside the program. Until it is sent on, nothing outside the program can see it.
Precisely: why a duplicate handle
hDuplicate stdout returns a new Handle for the same output with its own buffer. The experiment needs it because stdout must stay unbuffered in the stepper. In a compiled program you can call hSetBuffering stdout (BlockBuffering Nothing) directly.
Experiment
Make a direct write arrive before a buffered one
hDuplicate stdout makes a second handle to the same output with its own buffer, and hSetBuffering gives that buffer block mode. putStrLn writes through stdout, which GHCi leaves unbuffered, so it does not wait for out.
Why it matters When text reaches one output by two paths and only one path is buffered, lines can come out in a different order from the code. Flush the buffered path before writing on the other one.
Predict the order of the three lines. Then move the first hFlush out above putStrLn "2 direct" and predict again.
import System.IO
import GHC.IO.Handle (hDuplicate)
main :: IO ()
main = do
out <- hDuplicate stdout
hSetBuffering out (BlockBuffering Nothing)
hPutStrLn out "1 queued"
putStrLn "2 direct"
hFlush out
hPutStrLn out "3 queued"
hFlush out
What to look for
Watch the output panel at each stop. 2 direct appears first. 1 queued appears only after hFlush out runs. After the edit, 1 queued appears first. out shows as GHC.IO.Handle.Types.FileHandle with unevaluated parts, and some stops do not list it: GHCi cannot show what is inside a handle, so the output panel is the place to look. Do not change stdout’s own buffering in the stepper: GHCi writes its prompt through stdout, and the stepper waits for that prompt.
Expected output
2 direct
1 queued
3 queued03 / Flushing
What moves text out of a buffer?
To flush a buffer is to send the text waiting in it on to the operating system now. Buffers are flushed in four common ways. The program asks for it: an explicit flush. A newline is written: line buffering. The buffer is full and needs room. And some buffers, in some languages, are flushed when the program exits normally. Which of them apply depends on the language and on the buffer.
In Haskell, hFlush sends a handle’s buffer now. LineBuffering also flushes at each newline, and BlockBuffering when the buffer is full. hClose flushes before closing. When a compiled program ends, GHC flushes stdout and stderr, even after exitWith or an uncaught exception; other handles are flushed only if you close them.
import System.IO
main :: IO ()
main = do
putStr "Name: "
hFlush stdout -- a prompt has no newline
name <- getLine
putStrLn ("Hello, " ++ name)Flushing moves text from a buffer to the operating system. A partial line needs hFlush unless the handle is unbuffered.
Precisely: what the Haskell Report says
The Haskell 2010 Report defines the three modes and leaves the default mode to the implementation, noting that files are normally block-buffered and terminals line-buffered. hFlush hands bytes to the operating system; it does not ask for them to be written to the disk.
04 / Terminals and pipes
Why can the same program print differently in a terminal?
Many buffers change their behavior depending on where output goes. A person at a terminal wants to see each line as soon as it is printed. A program reading from a pipe, or a file being written, does better with fewer, larger writes. So a common choice is line buffering for a terminal and full buffering for a pipe or a file. A fully buffered stream is flushed only when its buffer fills up or when the program asks.
GHC chooses the standard handles’ modes when a compiled program starts. stdout gets LineBuffering on a terminal and BlockBuffering when it is a pipe or a file. stderr gets NoBuffering in both cases. So putStr "Name: " before a slow step shows nothing in either case until hFlush stdout.
import System.IO
main :: IO ()
main = do
terminal <- hIsTerminalDevice stdout
mode <- hGetBuffering stdout
print (terminal, mode)The same putStr is line-buffered in a terminal, block-buffered in a pipe, and unbuffered in GHCi. Ask hGetBuffering instead of guessing.
Coming from Python: Python makes the same choice for print: line-buffered on a terminal, block-buffered through a pipe.
Precisely: three environments
Observed with a compiled GHC program: through a pipe it printed (False,BlockBuffering Nothing); in a terminal, (True,LineBuffering). In the stepper’s GHCi (GHC 9.0), stdout reports NoBuffering. Nothing in BlockBuffering Nothing means GHC chooses the buffer size.
05 / Two streams
Why do errors and results come out of order?
Standard output and standard error are separate streams, so they can go to different places. Whoever starts a program can send its standard output to a file while standard error still appears in the terminal. That is why results go to standard output and problems to standard error: a program that reads the results never sees the warnings.
The two streams can also go to the same file. Then the order of lines in that file depends on buffering. If standard output is buffered and standard error is not, a warning can appear in the file before results that the program wrote earlier.
In a compiled program writing to a file, stdout is block-buffered and stderr is not. When both go to one file, a warning written with hPutStrLn stderr can appear before stdout lines printed earlier. hFlush stdout before writing to stderr keeps the order.
report :: Either String Int -> IO ()
report (Right celsius) = print celsius
report (Left problem) = do
hFlush stdout -- earlier results first
hPutStrLn stderr ("warning: " ++ problem)Results go to standard output, problems to standard error. Flush stdout before writing to stderr when both may share a file.
Precisely: redirecting in a terminal, and the output panel
In a terminal, tool > results.txt runs a program named tool, and > sends its standard output to the file results.txt. tool > run.log 2>&1 sends both streams to run.log. Read 2>&1 one piece at a time: 2 is the number of standard error, > sends it somewhere, and &1 means stream 1, standard output, not a file named 1. So 2>&1 means “send standard error where standard output goes”. The stepper shows only standard output, so the buffers experiment uses stdout, unbuffered in GHCi, as the direct path and a duplicate handle as the buffered one. hPutStrLn stderr is unbuffered in the same way.
06 / Lost output
What happens to waiting text when a program stops early?
Text in a buffer is still inside the program. If the program stops before the buffer is flushed, that text is gone. This happens when a program crashes, when the operating system stops it, and when it ends through a call that skips the normal cleanup.
In Haskell, GHC flushes stdout and stderr when a compiled program ends, but not other handles you opened and left open: text still in their buffers is lost. A program killed by the operating system loses everything that is buffered, including stdout’s.
import System.IO
import GHC.IO.Handle (hDuplicate)
main :: IO ()
main = do
out <- hDuplicate stdout
hSetBuffering out (BlockBuffering Nothing)
hPutStrLn out "saved"
hFlush out
hPutStrLn out "lost" -- out is never closedClose or flush every handle you open. Only stdout and stderr are flushed for you.
Coming from C: C’s exit flushes every open FILE stream; GHC flushes only stdout and stderr, so close the other handles yourself.
Precisely: finalizers are not a flush
GHC attaches a finalizer to each Handle that flushes and closes it once the garbage collector finds it unreachable. Finalizers are not guaranteed to run before the program exits, so they cannot be relied on. Use hClose, or withFile, which closes the handle when its action finishes.
Experiment
Exit while text is still in a buffer
When a compiled GHC program ends, it flushes stdout and stderr, but not other handles you left open. In the stepper’s GHCi, the run ends with the text still in the buffer. hClose flushes the buffer before closing the handle.
Why it matters Close or flush every handle you open before the program ends. Only stdout and stderr are flushed for you.
Predict whether lost is printed. Then add hClose out as the last line of the do block and predict again.
import System.IO
import GHC.IO.Handle (hDuplicate)
main :: IO ()
main = do
out <- hDuplicate stdout
hSetBuffering out (BlockBuffering Nothing)
hPutStrLn out "saved"
hFlush out
hPutStrLn out "lost"
What to look for
Watch the output panel at each stop. saved appears after hFlush out. lost never appears: main finishes with it still in out’s buffer.
Expected output
saved07 / Try it
Can you predict what reaches the output?
Predict the output before revealing the answer. The program is compiled and its output goes to a pipe.
import System.IO
main :: IO ()
main = do
out <- hGetBuffering stdout
err <- hGetBuffering stderr
putStrLn ("stdout: " ++ show out)
putStrLn ("stderr: " ++ show err)
What does this print through a pipe, and what would it print in a terminal and in the stepper?
Reveal the reasoning
stdout: BlockBuffering Nothing, then stderr: NoBuffering. GHC gives stdout block buffering when it is not a terminal, and gives stderr no buffering. Nothing means GHC chooses the buffer size. In a terminal, stdout would report LineBuffering. In the stepper, GHCi reports NoBuffering for stdout, because it changes the mode.
Practice tickets
- IO-101 · Show progress while photos copy → Free
- IO-102 · Keep the move journal after a crash → Sign in · Pro
- IO-103 · Separate results from warnings → Sign in · Pro