Preprocessors work on source text before it is broken down into its individual parts. This is an excellent place to clean up bad characters or to extract portions for later processing that the parser may otherwise choke on.
Preprocessors are run after the text is broken into lines.
Each preprocessor implements a
run method that takes a pointer to a
list of lines of the document, modifies it as necessary and returns
either the same pointer or a pointer to a new list.
Preprocessors must extend
Normalize whitespace for consistent parsing.