This restores the exact old structure for Java, and only injects the
SIMD acceleration between incrementing pos and checking if pos reached
endPos.
This makes going back to SIMD after a character reference significantly
simpler. The downside is that wholly-non-BMP text (as opposed to
isolated non-BMP emoji or Hanzi) ends up uselessly bouncing to the SIMD
code without benefiting from it when loading from network and counting
column numbers as Unicode scalar values.
If we want to avoid this failure mode, we should change column numbers
to count UTF-16 code units instead of scalars. Either way, the column
is "wrong" in some cases.
Differential Revision: https://phabricator.services.mozilla.com/D270673
Other tokenizer states and the serializer are potential follow-ups.
The code movement from nsHtml5Tokenizer.cpp to nsHtml5Tokeniner.h is
for enabling the eventual non-unified build of nsHtml5TokenizerSIMD.cpp
once the LLVM bug has been fixed.
Differential Revision: https://phabricator.services.mozilla.com/D227317
Other tokenizer states and the serializer are potential follow-ups.
The code movement from nsHtml5Tokenizer.cpp to nsHtml5Tokeniner.h is
for enabling the eventual non-unified build of nsHtml5TokenizerSIMD.cpp
once the LLVM bug has been fixed.
Differential Revision: https://phabricator.services.mozilla.com/D227317
Other tokenizer states and the serializer are potential follow-ups.
The code movement from nsHtml5Tokenizer.cpp to nsHtml5Tokeniner.h is
for enabling the eventual non-unified build of nsHtml5TokenizerSIMD.cpp
once the LLVM bug has been fixed.
Differential Revision: https://phabricator.services.mozilla.com/D227317
Henri Sivonen