primerian.dev
← Fundamentals
Guided lesson 6 questions · 5 languages

Composition and data transformation

Small steps joined into one: each result becomes the next input.

Composing two functions makes one function that passes the first function’s result to the second. A pipeline is a chain of such steps. Because each step receives the previous step’s result, the types must fit, the order decides the result, and a step must leave the value it receives unchanged.

Before you begin

Why this matters

Much code that prepares data is a chain of small steps: clean the text, drop what is not needed, sort, format. If you can follow a value through such a chain, you can find the step that gives a wrong result, and you can tell whether two steps may swap places.

What this helps you reason about

  • Follow one value through a chain of functions, one frame at a time.
  • Predict how swapping two steps changes the result.
  • Spot a step that changes the array it receives and so breaks a later step.

A radio show takes song requests from listeners. Before a request reaches the studio screen, the show removes extra spaces, makes the text lower case, drops repeated songs, and joins the rest into one line. Each of these jobs is a small function. This lesson joins small functions into one, and compares the result with one loop that does every job at once.

Already know this? The short version
  • pipe(f, g) returns (x) => g(f(x)): f runs first, and its result is g’s argument. Nested calls such as screenLine(dedupe(cleanAll(requests))) are the same pipeline, read from the inside out.
  • A pipeline of small named steps replaces one loop that changes several variables; each step can be tested, moved, or reused alone, and the stepper shows one frame per step.
  • Each step’s result type must be the next step’s parameter type, or tsc reports Argument of type 'string' is not assignable to parameter of type 'string[]'.
  • Order matters when two steps change the same thing: trim, then cut to six characters, is not cut, then trim. sort, reverse, splice, and arr[i] = … change the array a step receives; map, filter, and slice return a new one.

Go to the final check· Go to the practice tickets

Words in this lesson (7)
function composition
Joining two functions into one: the result of the first becomes the argument of the second. Composing f, then g, gives a function that computes g(f(x)).
higher-order function
A function that takes a function as an argument, returns a function, or both.
type parameter
A name for a type chosen by each call, written in angle brackets, such as T in function farthest<T extends Vehicle>(a: T, b: T): T. extends Vehicle is its constraint: T must be a type that fits Vehicle.
arrow function
A function written as a value, without a name, such as () => "stall " + i. The part before => lists its parameters, and the part after it is the value it returns.
pipeline
A chain of steps in which each step’s result is the input of the next step.
mutation
Changing the inside of an object, as in counter.value = 5. Every name that refers to that object sees the change. Reassignment (count = 5) is different: it changes only which value one name holds.
pure function
A function whose result depends only on its arguments and which has no side effects. Called again with the same arguments, it gives the same result.

Examples are focused snippets; imports, enclosing functions, or classes may be omitted. C examples use C11; Haskell examples use Haskell 2010. Expand the notes for language-specific depth.

01 / Compose

What does it mean to compose two functions?

The studio screen shows six characters. trimSpaces removes the spaces at both ends of a text, and shorten keeps its first six characters. A function’s result can be passed straight to another function. In shorten(trimSpaces(request)), trimSpaces runs first and gives "Jolene". That result becomes the argument of shorten. Joining two functions this way is called function composition: the result of the first becomes the argument of the second.

A small higher-order function can build the joined function once, so that it can be named and passed around. pipe below takes two functions and returns a new arrow function that calls first, then second. Its type parameters say how the types connect: first turns an A into a B, second turns that B into a C, so the new function turns an A into a C.

main.ts
function pipe<A, B, C>(
  first: (a: A) => B,
  second: (b: B) => C,
): (a: A) => C {
  return (a) => second(first(a));
}

const display = pipe(trimSpaces, shorten);
display("   Jolene"); // "Jolene"
shorten(trimSpaces("   Jolene")); // the same

Composing two functions gives one function that passes the first function’s result to the second.

Coming from Java: Java’s f.andThen(g) is pipe(f, g). A TypeScript function has no andThen method, so a small helper such as pipe does the job.

Precisely: pipe and compose

Mathematics writes composition the other way round: g ∘ f means “f first, then g”. Libraries often call the left-to-right version pipe or flow and the right-to-left version compose, so read a helper’s definition before you trust its name. JavaScript has no built-in pipe; a |> operator has been proposed but is not part of the language.

Experiment

Follow one request through two steps

The composed function calls the first step with the request and passes its result to the second step. trimSpaces removes the three spaces, so shorten keeps "Jolene", all six characters. In the other order, shorten keeps the first six characters, " Jol", and trimSpaces removes the spaces from those.

Why it matters A composed function runs its steps one after the other, and each step sees only the result of the step before it. Swap two steps and you change what each one sees.

Predict what is printed between the brackets, and which step’s frame appears first. Then change pipe(trimSpaces, shorten) to pipe(shorten, trimSpaces), and predict again.

function pipe<A, B, C>(
  first: (a: A) => B,
  second: (b: B) => C,
): (a: A) => C {
  return (a) => second(first(a));
}

function trimSpaces(text: string): string {
  return text.trim();
}

function shorten(text: string): string {
  return text.slice(0, 6);
}

const display = pipe(trimSpaces, shorten);
console.log("[" + display("   Jolene") + "]");
What to look for

Step to the last line. The composed function gets a frame named arrow function (line 5) on top of <top level>, with a holding " Jolene". On top of it, trimSpaces gets a frame with text " Jolene"; when it returns, shorten gets a frame with text "Jolene". The run prints [Jolene]. After the edit, shorten’s frame comes first, with text " Jolene", then trimSpaces gets " Jol", and the run prints [Jol]. <top level> is the stepper’s name for the code outside every function, and Node sometimes stops on one line two or three times.

Expected output

[Jolene]

02 / Pipeline

How does a pipeline differ from a loop that changes variables?

A pipeline is a chain of steps in which each step’s result is the input of the next step. The radio show’s screen line is a pipeline of three steps: cleanAll trims and lower-cases every request, dedupe drops repeated titles, and screenLine joins the titles with " | ". Each step is a small function with one job. Written as nested calls, the pipeline is read from the inside out.

The same work fits in one loop. The loop below does all three jobs in every pass, and it changes two variables as it goes, seen and line. To check one job, such as dropping repeats, you have to follow all three. In the pipeline, each step can be called and tested alone, and the stepper shows one frame for each step, with that step’s input in its parameter.

The pipeline builds a new array at each step, so it does a little more work. For lists of a few thousand items this rarely matters.

main.ts
const requests = ["Jolene ", "jolene", "Hallelujah"];

// One loop, three jobs, two changing variables.
// includes is true when the array already holds the value.
const seen: string[] = [];
let line = "";
for (const r of requests) {
  const title = r.trim().toLowerCase();
  if (!seen.includes(title)) {
    seen.push(title);
    line = line === "" ? title : line + " | " + title;
  }
}

// A pipeline of three named steps.
const piped = screenLine(dedupe(cleanAll(requests)));
// line and piped: "jolene | hallelujah"

In a pipeline each step does one job and gives a new value; one loop that does every job changes several variables in each pass.

Precisely: method chains

Array methods form pipelines too: requests.map(clean).filter(isLong) passes each method’s result to the next method. A chain works only while every step is a method of the value before it; pipe and nested calls work with any functions.

Experiment

Trace a pipeline one step at a time

Each step runs once, on the whole list, and gives a new list to the next step. Removing repeats compares texts exactly, so it finds the repeat only after cleaning has turned "Jolene " and "jolene" into the same text.

Why it matters Trace a pipeline one step at a time: each step’s input is the previous step’s result, which the stepper shows in the step’s parameter.

Predict the printed line, and the order in which the step frames appear. Then change screenLine(dedupe(cleanAll(requests))) to screenLine(cleanAll(dedupe(requests))), and predict again.

function cleanAll(requests: string[]): string[] {
  return requests.map((r) => r.trim().toLowerCase());
}

function dedupe(titles: string[]): string[] {
  // A Set keeps each value once, in the order first seen.
  return Array.from(new Set(titles));
}

function screenLine(titles: string[]): string {
  return titles.join(" | ");
}

const requests = ["Jolene ", "jolene", "Hallelujah"];
console.log(screenLine(dedupe(cleanAll(requests))));
What to look for

Three frames appear on top of <top level>, one at a time. cleanAll comes first, with requests ["Jolene ", "jolene", "Hallelujah"]; its arrow function frame appears once per request. dedupe comes next, with titles ["jolene", "jolene", "hallelujah"]. screenLine comes last, with titles ["jolene", "hallelujah"]. The run prints jolene | hallelujah. After the edit, dedupe comes first and receives the raw requests, which all differ, and the run prints jolene | jolene | hallelujah.

Expected output

jolene | hallelujah

03 / Types

Why must each step’s result fit the next step’s input?

Each step receives the previous step’s result, so the result type of each step must be the parameter type of the step after it. cleanAll and dedupe both take and give string[], so they fit together in either order. screenLine gives a string, so it can only come last: no other step takes a string.

The compiler checks every join. Moving screenLine before dedupe gives Argument of type 'string' is not assignable to parameter of type 'string[]'. The same check happens inside pipe: in pipe(screenLine, dedupe), B would have to be string and string[] at once, and the call does not compile.

main.ts
// cleanAll(requests: string[]): string[]
// dedupe(titles: string[]): string[]
// screenLine(titles: string[]): string

dedupe(screenLine(cleanAll(requests))); // does not compile

Each step’s result type must be the next step’s parameter type; the compiler reports the join where they differ.

04 / Order

Does the order of the steps matter?

Two steps whose types fit can often run in either order, and the result can differ. Trimming, then cutting to six characters, turns " Jolene" into "Jolene". Cutting first keeps " Jol", and trimming that gives "Jol". The pipeline experiment shows a second case: cleaning first makes "Jolene " and "jolene" the same text, so removing repeats finds the repeat. Removing repeats first compares the raw texts, which differ, so the song stays twice.

For each pair of steps, ask whether the second step needs something the first one produces. Removing repeats needs clean text, so cleaning comes first. Some pairs do not care: trimming and lower-casing give the same text in either order.

main.ts
const cleanFirst = dedupe(cleanAll(requests));
// ["jolene", "hallelujah"]
const dedupeFirst = cleanAll(dedupe(requests));
// ["jolene", "jolene", "hallelujah"]

Put a step after the steps whose results it needs. When two steps both change the same thing, their order changes the result.

05 / Mutation

What goes wrong when a step changes its input?

A step should give a new value and leave its input alone. A step that changes its input breaks a pipeline quietly: every later step that reads the same array, and the caller, see the change. cleanAll below writes each cleaned text back into the array it received. The show also logs each request as it was typed, and that log now shows the cleaned text.

An array parameter refers to the caller’s array, as Arguments, values, and references showed, so requests[i] = … inside cleanAll changes the caller’s array. Some array methods also change the array they are called on: sort, reverse, splice, and push. map, filter, and slice return a new array. A step built from those is a pure function: the same input gives the same result, and nothing else changes.

main.ts
function cleanAll(requests: string[]): string[] {
  for (let i = 0; i < requests.length; i++) {
    requests[i] = requests[i].trim().toLowerCase();
  }
  return requests; // the same array
}

const requests = ["  Jolene", "VOGUE"];
const cleaned = cleanAll(requests);
console.log(requests); // [ 'jolene', 'vogue' ]

A step that changes its input changes it for the caller and for every later step; build a new value instead.

Coming from Python: Python has list.sort(), which changes the list, and sorted(list), which returns a new one. A TypeScript array’s sort always changes the array it is called on.

Precisely: sorting a copy

slice() with no arguments copies an array, so titles.slice().sort() sorts the copy. Node.js 20 and later also have toSorted and toReversed, which return a new array. These copies are shallow: objects inside the array are shared, as Shallow and deep copying explains.

Experiment

Watch a step change the list it receives

cleanAll receives a reference to the same array that requests holds, writes each cleaned text into it, and returns that same array. cleaned and requests are two names for one array. map instead builds a new array and leaves the old one alone.

Why it matters A step that changes its input changes it for the caller and for every later step. Give back a new list, and leave the one you received alone.

Predict both printed lines. Then replace the body of cleanAll with return requests.map((r) => r.trim().toLowerCase());, and predict again.

function cleanAll(requests: string[]): string[] {
  for (let i = 0; i < requests.length; i++) {
    requests[i] = requests[i].trim().toLowerCase();
  }
  return requests;
}

const requests = ["  Jolene", "VOGUE"];
const cleaned = cleanAll(requests);
console.log("cleaned: " + cleaned.join(","));
console.log("as typed: " + requests.join(","));
What to look for

Step into cleanAll. Its requests and the requests of <top level> are the same array: select <top level> after each pass and its requests shows the cleaned text appearing, ["jolene", "VOGUE"], then ["jolene", "vogue"]. The run prints cleaned: jolene,vogue, then as typed: jolene,vogue. After the edit, <top level>’s requests stays [" Jolene", "VOGUE"], and the second line is as typed: Jolene,VOGUE.

Expected output

cleaned: jolene,vogue
as typed: jolene,vogue

06 / Try it

Can you explain a fresh example?

The show adds a 30-second spoken intro before each song and shows song lengths in whole minutes, dropping any remaining seconds. Two steps, two orders.

main.ts
function pipe<A, B, C>(
  first: (a: A) => B,
  second: (b: B) => C,
): (a: A) => C {
  return (a) => second(first(a));
}

const addIntro = (seconds: number) => seconds + 30;
const toMinutes = (seconds: number) => Math.floor(seconds / 60);

const oneWay = pipe(addIntro, toMinutes);
const otherWay = pipe(toMinutes, addIntro);
console.log(oneWay(200), otherWay(200));

What is printed, and why does the compiler accept both orders?

Reveal the reasoning

3 33. oneWay adds the intro first: 200 + 30 is 230 seconds, which is 3 whole minutes. otherWay turns 200 seconds into 3 minutes first, then adds 30 to that 3, as if it were seconds, and gives 33. Both steps take a number and give a number, so the types fit in either order, and only the result shows the mistake.

Practice tickets

  1. COMPOSE-101 · The practice screen shows fewer than three tunes → Free
  2. COMPOSE-102 · The poster names the wrong lead actor → Sign in · Pro
  3. COMPOSE-103 · Effects on the sound desk run in the wrong order → Sign in · Pro

Keep exploring

Immutable updates and persistent data →

Primary references