Character (Typography & Computing)

Understanding how characters operate requires tracing their historical evolution across digital computing: 1. 7-Bit ASCII (1963): The American Standard Code for Information Interchange defined 128 values: • 33 Non-printable control characters (like Null 0x00, Line Feed 0x0A, Bell 0x07). • 95 Printable characters (English letters A-Z, a-z, numbers 0-9, and standard punctuation). Because it used only 7 bits, it could not represent accented European letters (like 'é', 'ü', or 'ñ'), Cyrillic, Greek, or Asian languages. 2. Extended ASCII and Code Pages (The Chaos of the 1980s): Different computer manufacturers utilized the 8th bit (values 128–255) to create competing 'Code Pages' (like Windows-1252 or ISO-8859-1). Opening a file created on one machine frequently rendered unreadable gibberish (Mojibake) on another machine. 3. The Unicode Revolution: Founded in 1991, the Unicode Consortium assigned a unique numerical identity (a Code Point written as `U+XXXX`) to every glyph in human history. 4. UTF-8 (The Global Web Standard): Invented by Ken Thompson and Rob Pike in 1992, UTF-8 encodes Unicode code points using a variable-length byte scheme: • Standard English ASCII (0–127): 1 Byte • Latin accents, Greek, Arabic, Hebrew: 2 Bytes • CJK (Chinese, Japanese, Korean) ideographs: 3 Bytes • Emojis and rare historic scripts: 4 Bytes Today, UTF-8 is used on over 98% of all websites worldwide.

In modern programming, failing to distinguish between bytes, code points, and grapheme clusters causes severe string truncation and indexing bugs:

Standard JavaScript `string.length` counts 16-bit code units, NOT human characters! To count true visible characters (graphemes), use the modern `Intl.Segmenter` API:

export function countVisualCharacters(text: string): {
  visualCharacters: number;
  utf8Bytes: number;
  codePoints: number;
} {
  if (!text) return { visualCharacters: 0, utf8Bytes: 0, codePoints: 0 };

  // 1. Calculate UTF-8 Byte Weight
  const utf8Bytes = new TextEncoder().encode(text).length;

  // 2. Count Unicode Code Points
  const codePoints = Array.from(text).length;

  // 3. Count True Human Graphemes using Intl.Segmenter (UAX #29)
  const segmenter = new Intl.Segmenter(undefined, { granularity: 'grapheme' });
  const segments = [...segmenter.segment(text)];
  const visualCharacters = segments.length;

  return { visualCharacters, utf8Bytes, codePoints };
}

While long-form writing relies on word count, digital systems enforce strict character count boundaries: • Google SERP Title Tags: Truncates at ~60 characters (~580px). • Google SERP Meta Descriptions: Truncates at ~160 characters (~960px). • X / Twitter Posts: Strict 280-character ceiling. • SMS Text Messages: 160 characters per standard 7-bit GSM message segment. Sending 161 characters bills as two messages! • Database Schemas: SQL columns defined as `VARCHAR(255)` reject strings exceeding 255 characters.