AST Fundamentals (Parse, Transform, Generate)
Every source-to-source tool follows the same pipeline — parse code into a structured Abstract Syntax Tree, transform it, generate code back out.
Purpose
An Abstract Syntax Tree (AST) represents source code as a structured tree of typed nodes — every operator, identifier, and literal is a distinct node, not a matched substring. Every linter, codemod, bundler, and transpiler follows the same three-stage pipeline built on this representation.
The pipeline
const code = 'const x = 1 + 2;';
const ast = parse(code);
// { type: 'VariableDeclaration', declarations: [{
// id: { type: 'Identifier', name: 'x' },
// init: { type: 'BinaryExpression', operator: '+',
// left: { type: 'Literal', value: 1 }, right: { type: 'Literal', value: 2 } }
// }] }- Parse: source text → AST.
- Transform: AST → a modified AST (a codemod’s actual work happens here).
- Generate: (modified) AST → source text again.
Behavior notes
- AST-based transformation is more reliable than text-based find-and-replace because the AST distinguishes code structure — it can’t accidentally match text inside a string literal or comment, since those are distinctly-typed nodes.
- A traversal visits nodes in the tree (usually depth-first), commonly using a visitor pattern — register a callback for each node type of interest, and the traversal calls it whenever it encounters one.
- Different tools use different concrete AST formats (Babel’s, ESTree/Acorn’s, TypeScript’s own) — broadly similar in concept, but not directly interchangeable without conversion.
Common mistakes
- Attempting a non-trivial code transformation with text-based regex matching instead of parsing to an AST — fragile against any formatting variation the regex didn’t anticipate.
- Assuming all JavaScript parsers produce an identical AST shape — check which AST format a specific tool expects before writing a transform against it.
Runtime notes
A tool-building technique — implemented by libraries (Babel, Acorn, TypeScript’s compiler API) that run in Node or a browser, independent of what runtime the parsed code itself targets.