right number of newlines between tokens when needed. This reduces the
delta of the gcc -E output from 12198 differences to 6764. It still needs
to emit filenames on #line directives, track filename switches, and track
entry/exit of include files.
llvm-svn: 38559
Now, instead of keeping a pointer to the start of the token in memory, we keep the
start of the token as a SourceLocation node. This means that each LexerToken knows
the full include stack it was created with, and means that the LexerToken isn't
reliant on a "CurLexer" member to be around (lexer tokens would previously go out of
scope when their lexers were deallocated).
This simplifies several things, and forces good cleanup elsewhere. Now the
Preprocessor is the one that knows how to dump tokens/macros and is the one that
knows how to get the spelling of a token (it has all the context).
llvm-svn: 38551
