I/O, buffering, and external state

Before you begin

Why this matters

Most of the time output appears when you print it, so the times it does not are confusing. A script that stops early can lose its last lines. Messages can come out in a different order from the code that wrote them. Another program’s output can arrive all at once. Each of these comes from text waiting somewhere. Once you know where text waits, you can predict these cases and fix them.

What this helps you reason about

Predict whether a message is on the screen before a slow step runs.
Explain why two messages can appear in a different order from the code that wrote them.
Spot text that will be lost if the program stops early, and send it out in time.

When you call console.log, you see the text at once, and in Node.js that model is correct: Node.js does not keep your text waiting, as C’s printf can. Text waits when your own code or a library collects it to write later. This lesson is about that waiting text: when it leaves, and when it is lost.

Already know this? The short version

console.log and process.stdout.write keep no text waiting: Node.js starts sending the text to the operating system during the call.
Node.js documents these writes as synchronous for files and for terminals on Linux and macOS, and asynchronous for pipes on Linux and macOS.
Late, reordered, or missing output in Node.js comes from buffers that your code or a library adds, such as a logger that writes in batches or a corked stream; such a buffer is flushed only when that code flushes it.
process.exit() loses whatever such a buffer still holds, and can stop an unfinished asynchronous write to a pipe; set process.exitCode and let the program end instead.
Flush buffered results before writing to console.error when order matters; a flush hands bytes to the operating system, and fs.fsyncSync asks for a disk write.

Go to the final check· Go to the practice tickets

Words in this lesson (7)

operating system: The software, such as Linux, macOS, or Windows, that runs programs and controls the screen, files, and other devices. Every write a program makes is a request to it.
stream: A destination that receives bytes in order. A program writes to streams; it does not draw on the screen itself.
standard output and standard error: The two output streams every program starts with. Standard output is for normal results; standard error is for messages about problems.
terminal and pipe: A terminal is the window where you type commands. A pipe is a connection that carries one program’s output into another program’s input.
buffer: An area of memory where written text waits inside the program before it is sent on to the operating system.
flush: To send the text waiting in a buffer on to the operating system now. A flush is not a disk write.
process.exit(): A Node.js function that ends the program at once. It does not wait for your own buffers or for unfinished asynchronous writes.

Examples are focused snippets; imports, enclosing functions, or classes may be omitted. C examples use C11; Haskell examples use Haskell 2010. Expand the notes for language-specific depth.

01 / A write call

What does a print call promise?

Here is a useful model: console.log writes to the screen immediately. For most small programs, it predicts exactly what you see.

Underneath, a program does not draw on the screen itself. It asks the operating system to write bytes to a stream, and the operating system delivers them.

Every program starts with two output streams. Standard output is for the program’s normal results; console.log writes there. Standard error is for messages about problems; console.error writes there.

The program does not choose where these streams go. Whoever starts the program chooses. A stream may go to a terminal, to a file, or through a pipe into another program.

main.ts

process.stdout.write("loading... "); // no newline
let total = 0;
for (let i = 1; i <= 3; i++) {
  total += i;
}
console.log("ready", total);

A write passes text to a stream. The program does not choose where that stream goes.

Coming from the browser: In the browser, console.log sends text to the developer tools, not to a stream; standard output, standard error, and pipes exist when Node.js runs your code.

Precisely: what Node.js promises

Node.js documents when a write to process.stdout is synchronous, which means finished before the call returns. Files: synchronous on every system. Terminals: synchronous on Linux and macOS, asynchronous on Windows. Pipes: asynchronous on Linux and macOS, synchronous on Windows. Asynchronous means Node.js may hand part of the text to the operating system later, after the call returns. Observed in the stepper (Node.js 22 on Linux, output to a pipe): every short write had reached the pipe by the next stop.

Experiment

Watch when each write reaches the output panel

Node.js does not keep a short write waiting for a newline or a flush. In the stepper, output goes to a pipe, and each write reached it before the next stop. console.log is process.stdout.write with a newline added.

Why it matters Before you assume "it prints immediately", ask what sits between your print call and the screen. When nothing does, output appears in statement order.

Predict whether loading... appears before the loop runs, even though it does not end with a newline. Then replace process.stdout.write("loading... ") with console.log("loading...") and compare.

// Describes Node.js’s process object to TypeScript; runs no code.
declare const process: { stdout: { write(text: string): boolean } };

process.stdout.write("loading... ");
let total = 0;
for (let i = 1; i <= 3; i++) {
  total += i;
}
console.log("ready", total);

What to look for

Step past line 4 and look at the output panel: loading... is already there, before the loop starts. The panel shows standard output as the stepper received it at each stop. Node can stop on the same line two or three times; that is normal for this debugger. After the edit, loading appears at the same stop, on its own line.

Expected output

loading... ready 6

02 / Buffers

Where does written text wait?

Asking the operating system to write takes time, much more than adding text to a string. So many programs and libraries collect text in a buffer first: an area of memory where written text waits until there is enough to send at once.

So the model from section 01, that a print writes to the screen immediately, needs one addition: written text can still be inside the program. Anything that does not go through the buffer, such as a direct write to the same output, can arrive first.

In TypeScript programs, buffers come from your code or a library: a logger that collects lines and writes them in groups, or a stream that queues data. The queued array below is the simplest buffer.

main.ts

const queued: string[] = [];

function flush(): void {
  if (queued.length > 0) console.log(queued.join("\n"));
  queued.length = 0;
}

queued.push("1 queued");
console.log("2 direct"); // does not wait
flush();

A buffer holds written text inside the program. Until it is sent on, nothing outside the program can see it.

Coming from React: The browser and React use the same idea. The page is not redrawn while a long synchronous loop runs, and React collects several state updates and applies them together later, which React calls batching; its flushSync function applies them at once.

Precisely: cork and uncork

Node.js streams have a built-in buffer you can switch on. writable.cork() holds all following writes in memory, and writable.uncork() sends them. process.stdout is a writable stream, so code that corks it puts a buffer between your writes and standard output.

Experiment

Make a direct write arrive before a buffered one

queued is a buffer the program keeps itself: an array that collects lines until flush writes them out. console.log does not look at that array, so 2 direct reaches the output while 1 queued is still waiting.

Why it matters When text reaches one output by two paths and only one path is buffered, lines can come out in a different order from the code. Flush the buffered path before writing on the other one.

Predict the order of the three lines. Then move the first flush() call above console.log("2 direct") and predict again.

const queued: string[] = [];

function flush(): void {
  if (queued.length > 0) console.log(queued.join("\n"));
  queued.length = 0;
}

queued.push("1 queued");
console.log("2 direct");
flush();
queued.push("3 queued");
flush();

What to look for

Watch the output panel at each stop. 2 direct appears first, at the stop after it is written. 1 queued appears only at the stop after the flush, even though it was written first. After the edit, 1 queued appears first, and the lines come out in code order. Every frame also lists flush as a variable, shown as function flush(): that is how the panel shows a function, not a copy of it.

Expected output

2 direct
1 queued
3 queued

03 / Flushing

What moves text out of a buffer?

To flush a buffer is to send the text waiting in it on to the operating system now. Buffers are flushed in four common ways. The program asks for it: an explicit flush. A newline is written: line buffering. The buffer is full and needs room. And some buffers, in some languages, are flushed when the program exits normally. Which of them apply depends on the language and on the buffer.

Each buffer has its own rules. A buffer you write yourself is flushed only when your code flushes it: never at a newline and never at exit unless you arrange it. A library’s buffer follows the rules in its documentation.

main.ts

const queued: string[] = [];

function write(line: string): void {
  queued.push(line);
  if (queued.length === 100) flush(); // flush when full
}

function flush(): void {
  if (queued.length > 0) console.log(queued.join("\n"));
  queued.length = 0;
}

write("first");
flush(); // flush when asked

Flushing moves text from a buffer to the operating system. Know which rules your buffer follows.

Precisely: a flush is not a disk write

A flush hands bytes to the operating system. For a file, the system may keep them in memory for a while before writing them to the disk. That is enough for other programs to read them and for them to survive a crash of your program, but not a power failure. fs.fsyncSync asks the system to finish writing a file to the disk.

04 / Terminals and pipes

Why can the same program print differently in a terminal?

Node.js does not change process.stdout for a terminal, a file, or a pipe. Other programs do, and you meet them from Node.js, so this section explains their choice.

Many buffers change their behavior depending on where output goes. A person at a terminal wants to see each line as soon as it is printed. A program reading from a pipe, or a file being written, does better with fewer, larger writes. So a common choice is line buffering for a terminal and full buffering for a pipe or a file. A fully buffered stream is flushed only when its buffer fills up or when the program asks.

child_process is the Node.js module that starts other programs, and a program started with it sends its output to yours through a pipe. A Python script, for example, then fully buffers its output, so its lines can arrive late and all at once. Many command-line tools also change their output when it is not a terminal, for example by leaving out colors. To check which case your own program is in, read process.stdout.isTTY: it is true when standard output is a terminal (TTY is an old name for a terminal) and undefined otherwise.

main.ts

if (process.stdout.isTTY) {
  console.log("standard output is a terminal");
} else {
  console.log("standard output is a pipe or a file");
}

Where output goes can change when it appears. Do not assume a terminal.

Coming from Python: Python’s print is line-buffered on a terminal and block-buffered through a pipe, where it needs flush=True; console.log starts sending its text to the operating system during the call in both cases.

Precisely: the stepper is not a terminal

The stepper runs your program with standard output connected to a pipe, so process.stdout.isTTY is undefined there. In a terminal it is true. Behavior that depends on a terminal, such as colors in some libraries, is therefore not what the stepper shows.

05 / Two streams

Why do errors and results come out of order?

Standard output and standard error are separate streams, so they can go to different places. Whoever starts a program can send its standard output to a file while standard error still appears in the terminal. That is why results go to standard output and problems to standard error: a program that reads the results never sees the warnings.

The two streams can also go to the same file. Then the order of lines in that file depends on buffering. If standard output is buffered and standard error is not, a warning can appear in the file before results that the program wrote earlier.

In Node.js, console.log and console.error keep no text waiting, so their lines keep their order. Once you buffer results yourself, flush them before writing a warning. In the snippet below, values, isValid, convert, results, and flushResults stand for your own data and functions.

main.ts

for (const value of values) {
  if (isValid(value)) {
    results.push(convert(value)); // buffered results
  } else {
    flushResults(); // earlier results first
    console.error("warning: skipped " + value);
  }
}
flushResults();

Results go to standard output, problems to standard error. Flush buffered results before writing to the other stream.

Precisely: redirecting in a terminal, and the output panel

In a terminal, tool > results.txt runs a program named tool, and > sends its standard output to the file results.txt. tool > run.log 2>&1 sends both streams to run.log. Read 2>&1 one piece at a time: 2 is the number of standard error, > sends it somewhere, and &1 means stream 1, standard output, not a file named 1. So 2>&1 means “send standard error where standard output goes”. The stepper collects standard error separately and does not show it in the output panel. To make order visible, the buffers experiment uses console.log as the unbuffered path; console.error behaves the same way.

06 / Lost output

What happens to waiting text when a program stops early?

Text in a buffer is still inside the program. If the program stops before the buffer is flushed, that text is gone. This happens when a program crashes, when the operating system stops it, and when it ends through a call that skips the normal cleanup.

In Node.js, process.exit() ends the program at once. Text already passed to process.stdout has started on its way out. A buffer your code keeps is lost unless you flush it first, and reaching the end of the program does not flush it either. This is how a script loses its last log lines: a logger that collects lines still holds them when the script calls process.exit().

main.ts

const queued: string[] = [];

function flush(): void {
  process.stdout.write(queued.join(""));
  queued.length = 0;
}

queued.push("saved\n");
flush();
queued.push("lost\n"); // never flushed
process.exit(0);

Buffered text survives only if something flushes it. Flush what matters before every point where the program can stop.

Coming from Python: Python flushes sys.stdout when the program ends, even after sys.exit(); Node.js has no such step for a buffer your own code keeps.

Precisely: process.exit and queued writes

The Node.js documentation warns about process.exit() and pipes. On Linux and macOS, a write to a pipe is asynchronous, so part of a long write may still be waiting when the call returns. process.exit() ends the program before that part is written. Setting process.exitCode and letting the program end on its own gives those writes time to finish.

Experiment

Exit while text is still in a buffer

process.exit ends the program at once. Text already written with process.stdout.write had reached the output in the stepper, but the queued array belongs to the program: nothing writes it out unless the program calls flush. Deleting process.exit does not help, because no code flushes it at the end either.

Why it matters A buffer your own code keeps is flushed only by your own code. Flush it before every point where the program can stop.

Predict whether lost is printed. Then delete process.exit(0) and predict again: does reaching the end of the program print it?

// Describes Node.js’s process object to TypeScript; runs no code.
declare const process: { stdout: { write(text: string): boolean }; exit(code: number): never };

const queued: string[] = [];

function flush(): void {
  process.stdout.write(queued.join(""));
  queued.length = 0;
}

queued.push("saved\n");
flush();
queued.push("lost\n");
process.exit(0);

What to look for

Watch the output panel. saved appears at the stop after the flush. lost never appears, not even after the run has ended. After the edit, lost still never appears.

Expected output

saved

07 / Try it

Can you predict what reaches the output?

Predict the output before revealing the answer. This buffer flushes itself when it holds two lines.

main.ts

const pending: string[] = [];

function flush(): void {
  if (pending.length > 0) console.log(pending.join("\n"));
  pending.length = 0;
}

function write(line: string): void {
  pending.push(line);
  if (pending.length === 2) flush();
}

write("one");
console.log("two");
write("three");
write("four");

What does this print? Does every line appear?

Reveal the reasoning

two, then one, then three. write("one") only adds to the buffer. console.log("two") prints at once, so two comes first. write("three") makes two waiting lines, which fills this buffer, so one and three are printed together. write("four") waits in the buffer, and nothing flushes it before the program ends, so four never appears.

I/O, buffering, and external state

Why this matters

What this helps you reason about

What does a print call promise?

Watch when each write reaches the output panel

Where does written text wait?

Make a direct write arrive before a buffered one

What moves text out of a buffer?

Why can the same program print differently in a terminal?

Why do errors and results come out of order?

What happens to waiting text when a program stops early?

Exit while text is still in a buffer

Can you predict what reaches the output?

Practice tickets

Keep exploring

Primary references