Before you begin
Why this matters
Names, messages, and labels often have a length limit. When you know what .length counts, you can explain why a short name is refused, why a cut label ends in �, and why two names that look the same are not ===.
What this helps you reason about
- Predict .length for text with accents and emoji.
- Count code points instead of code units, and cut text without splitting a surrogate pair.
- Spot two strings that look the same but are stored differently.
The name “Zoë” has three characters, and "Zoë".length is 3. But "🏆".length is 2. This lesson shows how text is stored as numbers, what .length counts, and how to count and cut text without breaking a character.
Already know this? The short version
- Unicode gives each character a code point (ë is U+00EB); UTF-8 stores it in 1 to 4 bytes, UTF-16 in one or two 16-bit code units.
- A string is UTF-16 code units: .length, s[i], slice, and charCodeAt count and cut code units, and a code point above U+FFFF is a surrogate pair of two.
- for...of, [...s], and codePointAt work with code points; new TextEncoder().encode(s) gives UTF-8 bytes.
- ë may be one code point or e + U+0308, which are not ===; s.normalize("NFC") makes them equal, and Intl.Segmenter counts grapheme clusters.
Words in this lesson (7)
- Unicode
- The standard that gives a number to every character of every writing system, and to symbols and emoji. Programs in every language use it for text.
- code point
- The number Unicode gives a character, written U+ and hexadecimal digits: ë is U+00EB, which is 235. The first 128 code points are the ASCII character codes.
- encoding
- A rule for storing code points as smaller fixed-size pieces, such as bytes. UTF-8 and UTF-16 are encodings.
- code unit
- One fixed-size piece of an encoding: a byte in UTF-8, a 16-bit number in UTF-16. One code point takes one or more code units.
- UTF-8
- An encoding that stores each code point in one to four bytes. Letters without accents take one byte; ë takes two.
- UTF-16
- An encoding that stores each code point in one or two 16-bit code units. A code point above U+FFFF, such as most emoji, takes two, called a surrogate pair.
- grapheme cluster
- What a reader sees as one character. It can be several code points, such as e followed by a combining mark, or the two code points of a flag.
Examples are focused snippets; imports, enclosing functions, or classes may be omitted. C examples use C11; Haskell examples use Haskell 2010. Expand the notes for language-specific depth.
01 / Code points
What number does each character get?
In Bits, bytes, and binary, a program stored each character as a number, its character code: the character 7 is 55. Those codes come from ASCII, an old table of 128 characters. ASCII has no ë, no é, and no emoji.
Unicode is the table that programs use today. It gives a number to every character of every writing system, and to symbols and emoji. That number is the character’s code point. It is written U+ followed by hexadecimal digits: Z is U+005A, which is 90; ë is U+00EB, which is 235; and the trophy 🏆 is U+1F3C6, which is 127942. The first 128 code points are the ASCII codes, so the character 7 is still 55.
In TypeScript, text.codePointAt(i) gives the code point that starts at position i. A string literal can name a code point with an escape: "\u00eb" is ë, and "\u{1F3C6}" is 🏆.
const player = "Zoë";
console.log(player.codePointAt(2)); // 235
console.log(player === "Zo\u00eb"); // true
console.log("\u{1F3C6}"); // 🏆Unicode gives every character a number, its code point, such as U+00EB for ë.
Experiment
Print the code point of each character
for (const ch of player) runs the block once for each code point of player. ch.codePointAt(0) gives the code point of ch as a number. Each character has one code point. Z and o are below 128, so their code points are the same as their ASCII codes, 90 and 111. ë is U+00EB, 235, and 🏆 is U+1F3C6, 127942.
Why it matters A code point belongs to the character, not to the language. The same name gives the same code points in every language that reads it by code point.
Predict the three numbers. Then change "Zoë" to "Zoë🏆" and predict what the trophy adds.
const player = "Zoë";
for (const ch of player) {
console.log(ch, ch.codePointAt(0));
}
What to look for
Step through the loop and watch ch. Between passes ch shows “not initialized yet”, because a new ch is made for each pass; the debugger may also stop on one line several times. The run prints Z 90, o 111, and ë 235. After the edit, the loop runs one more time and prints 🏆 127942: for...of gives the whole trophy as one ch.
Expected output
Z 90
o 111
ë 23502 / Encodings
How is a code point stored?
A code point can be as large as U+10FFFF, more than a million, so it does not fit in one byte. An encoding is a rule for storing code points in smaller fixed-size pieces, called code units. Two encodings matter for text in programs.
UTF-8 uses bytes. A code point below 128 takes one byte, the same byte as in ASCII. A larger code point takes two, three, or four bytes. Z is the byte 5a; ë is the two bytes c3 ab; the football ⚽ (U+26BD) is e2 9a bd; and 🏆 is f0 9f 8f 86. Every byte after the first byte of a character has the bit pattern 10xxxxxx, and no first byte has it. So a program can always tell where a character starts. Files and web pages are almost always UTF-8.
UTF-16 uses 16-bit code units. A code point up to U+FFFF takes one code unit with the same value: ë is 00eb. A larger code point takes two code units from a range that Unicode keeps for this purpose, called a surrogate pair: 🏆 is d83c dfc6. Neither half is a character on its own.
A TypeScript string is a sequence of UTF-16 code units. When text leaves the program, for example into a file, it is usually written as UTF-8. new TextEncoder().encode(text) gives those bytes as a Uint8Array.
const bytes = new TextEncoder().encode("Zoë");
console.log(bytes); // Uint8Array(4) [ 90, 111, 195, 171 ]
console.log((195).toString(16)); // c3UTF-8 stores a code point in one to four bytes; UTF-16 stores it in one or two 16-bit code units.
Coming from Python: A Python 3 str is a sequence of code points; a JavaScript string is a sequence of UTF-16 code units.
Precisely: the UTF-8 bit patterns
UTF-8 writes the bits of the code point into fixed patterns. Up to U+007F: 0xxxxxxx. Up to U+07FF: 110xxxxx 10xxxxxx. Up to U+FFFF: 1110xxxx 10xxxxxx 10xxxxxx. Up to U+10FFFF: 11110xxx and three bytes of 10xxxxxx. ë is 235, binary 00011 101011; the two groups go into 110xxxxx and 10xxxxxx and give 11000011 10101011, which is c3 ab. The number of 1s at the start of the first byte is the number of bytes in the character.
03 / Length
What does a length count?
In TypeScript, text.length counts UTF-16 code units. For "Zoë" that is 3, one per letter. For "Foxes 🏆" it is 8, because the trophy is a surrogate pair. text[i] and text.charCodeAt(i) also work with code units: "🏆"[0] is half a character.
To count code points, take the string apart by code points. [...text] makes an array with one element for each code point; the spread ... works like a for...of loop. So [...text].length counts code points: 7 for "Foxes 🏆".
const team = "Foxes 🏆";
console.log(team.length); // 8
console.log([...team].length); // 7
console.log(team.charCodeAt(6).toString(16)); // d83c.length counts UTF-16 code units; [...text].length counts code points.
Coming from Python: len("🏆") is 1 in Python, which counts code points; "🏆".length is 2.
Precisely: why JavaScript uses UTF-16
JavaScript strings were defined when every Unicode character fitted in 16 bits. Code points above U+FFFF were added later, as surrogate pairs, and .length kept its old meaning. A string can even hold half of a pair on its own; the specification calls a string a sequence of 16-bit values, not of characters.
Experiment
Count the same names in two units
.length counts UTF-16 code units. The trophy is a surrogate pair, so “Foxes 🏆” is 8 code units but 7 code points. [...text] makes an array with one element per code point, so its length counts code points. The typed e and its mark are two code points, each one code unit.
Why it matters Know which unit your length counts before you compare it with a limit. Even a count of code points can differ for two names that look the same.
Predict both printed lines. Then change "Zoë" to "Zoe\u0308", the same name typed as e followed by U+0308, a mark drawn on the letter before it, and predict again.
const player = "Zoë";
const team = "Foxes 🏆";
console.log(player.length, team.length);
console.log([...player].length, [...team].length);
What to look for
Step past each line and watch the output. The run prints 3 8, then 3 7: .length counts the trophy as 2, and the spread counts it as 1. After the edit, player still shows as Zoë, but the run prints 4 8, then 4 7: the mark is one more code unit and one more code point.
Expected output
3 8
3 704 / Graphemes
What does a reader count as one character?
A reader does not see code points. A grapheme cluster is what a reader sees as one character. Most grapheme clusters are one code point, but not all of them.
The letter ë can be stored in two ways. Composed, it is one code point, U+00EB. Decomposed, it is the letter e followed by U+0308, a combining mark: a code point that is drawn on the character before it. Both forms look the same, and a keyboard, a phone, or a file can send either. They are different strings: their lengths differ by one, and === compares their code units and says they are not equal. Equality and comparison shows how strings are compared.
A flag is another example. 🇫🇷 is two code points, the regional letters F and R, shown together as one flag.
Node.js has the rules for grapheme clusters built in: Intl.Segmenter with granularity "grapheme" splits text into grapheme clusters. To compare text that may come in either form, convert both strings to the same form first. text.normalize("NFC") composes every letter and mark that Unicode has a single code point for.
const composed: string = "Zo\u00eb";
const decomposed: string = "Zoe\u0308";
console.log(composed.length, decomposed.length); // 3 4
console.log(composed === decomposed); // false
console.log(composed === decomposed.normalize("NFC")); // true
const split = new Intl.Segmenter("en", { granularity: "grapheme" });
console.log([...split.segment(decomposed)].length); // 3A reader counts grapheme clusters. One of them can be several code points, and the same text can be composed or decomposed.
Precisely: literal types and normal forms
The type string on composed and decomposed is needed here. Without it, each const has the literal type of its exact text, and TypeScript refuses composed === decomposed with This comparison appears to be unintentional because the types '"Zoë"' and '"Zoë"' have no overlap: the two types print the same but differ. NFD is the opposite form of NFC: it splits every letter that has a mark into the letter and its marks.
05 / Cutting text
What happens when text is cut at a fixed count?
Text is often cut to fit: a screen with room for 12 characters, a field that holds 20 bytes. A cut at a fixed count of bytes or code units can end in the middle of a character. What is left is not valid text. When it is printed, the output usually shows the replacement character � (U+FFFD) in its place.
In TypeScript, slice and substring cut at code unit positions. A cut between the two halves of a surrogate pair keeps half a character. To keep the first n code points, cut the array of code points: [...text].slice(0, n).join(""). join("") puts the pieces back into one string. A cut by code points can still separate a letter from a combining mark; to cut by grapheme clusters, use Intl.Segmenter.
const team = "Foxes 🏆";
console.log(team.slice(0, 7)); // Foxes �
console.log([...team].slice(0, 7).join("")); // Foxes 🏆Cut by code points, never at a fixed code unit position; a surrogate pair must stay together.
Experiment
Keep the first 7 code units of a team name
team.slice(0, 7) keeps the first 7 UTF-16 code units. The trophy is code units 6 and 7, so the cut keeps only its first half. [...team] splits the string into code points, slice(0, 7) keeps the first 7 of them, and join("") puts them back together into one string.
Why it matters Cut text at the edge of a character, never at a fixed number of bytes or code units. Even then, a cut by code points can separate a letter from its mark.
Predict what is printed. Then change the second line to const short = [...team].slice(0, 7).join(""); and predict again.
const team = "Foxes 🏆";
const short = team.slice(0, 7);
console.log(short, short.length);
What to look for
Step past line 2 and look at short in the panel: "Foxes \ud83c", the first half of the trophy’s surrogate pair, with no character to show. The run prints Foxes � 7: the output cannot hold half a pair, so the � mark (U+FFFD) stands in for it. After the edit, short is "Foxes 🏆" and the run prints Foxes 🏆 8.
Expected output
Foxes � 706 / Try it
Can you explain a fresh example?
Work out each count for the label “Rémi 🏆”: é takes 2 bytes in UTF-8 and the trophy 4. Conditions and truthiness comes next: it asks what a condition such as a length check may contain.
const label = "Rémi 🏆";
const bytes = new TextEncoder().encode(label);
console.log(label.length, [...label].length, bytes.length);
What is printed?
Reveal the reasoning
7 6 10. .length counts UTF-16 code units: R, é, m, i, the space, and two for the trophy, which is a surrogate pair. [...label] has one element per code point, 6. The UTF-8 bytes are 1 + 2 + 1 + 1 + 1 + 4 = 10.
Practice tickets
- TEXT-101 · Riders with short names are refused as too long → Free
- TEXT-102 · Some long names come out broken on the board → Sign in · Pro
- TEXT-103 · The same name costs more when typed on some phones → Sign in · Pro