Commit Graph
130 Commits
Author SHA1 Message Date
Henri Sivonen 7bc354567b Bug 1747388. r=smaug
Differential Revision: https://phabricator.services.mozilla.com/D145571
2022-05-10 07:05:09 +00:00
Henri Sivonen a8ed2707bb Bug 1701828 - meta charset rewrite. r=smaug
Implements https://github.com/whatwg/html/issues/6962 . Improves performance
when <meta charset> occurs in head but after the first kilobyte and aligns
behavior better with WebKit and Blink.

The main change is to avoid reloads when meta appears within head but
after the first kilobyte. Prior to this change, Gecko reloaded in that
case (in compliance with the spec!) even though WebKit and Blink did not.

Differences from WebKit and Blink:

* WebKit and Blink honor <meta charset> in <noscript>. This implementation
  does not.
* WebKit and Blink look for meta as if the tree builder was unaware of
  foreign content. This implementation is foreign content-aware. This
  makes a difference for CDATA sections that contain a > before the meta
  as well as style and script elements within foreign content. This could
  happen if the CDATA section that has mysteriously been introduced around
  a what looks like a meta tag also contains another prior tag-looking
  run of text.
* This implementation processes rel=preload and speculative loads that are
  seen before <meta charset> has been seen. WebKit and Blink instead first
  look for the meta and rewind before starting speculative parsing.
* Unlike WebKit, if there is neither an honored meta nor syntax resembling
  an XML declaration, detection from content takes place (as in Blink).
* Unlike Blink, if there is neither an honored meta nor syntax resembling
  an XML declaration, the detection from content is not dependent of network
  buffer boundaries.
* Unlike Blink, detection from content can trigger a reload at the end of
  the stream if the guess made at that point differs from the first guess.
  (See below for the definition of the input to the first guess.)

Differences from the old spec and Gecko previously:

* Meta inside script and RCDATA elements is no longer honored.
* Late meta is now ignored and no longer triggers a reload.
* Later meta counts as early enough meta: In addition to the previous
  meta within the first 1024 bytes, now a meta that started within the first
  1024 bytes counts as early enough. Additionally, if by then there hasn't
  been a template start tag and head hasn't ended, meta occurring before the
  earlier of the end of the head or a template start tag counts as early
  enough.
* Meta now counts as not-late even if the encoding label has numeric
  character reference escapes.
* Syntax resembling an XML declaration longer than a kilobyte is honored if
  there is no honored meta.
* If there is neither an honored meta nor syntax resembling an XML declaration,
  the initial chardetng scan is potentially longer than before: the first 1024
  bytes, the token spanning the 1024-byte boundary if there is such a token,
  and, if by then head hasn't ended and there hasn't been a template start tag
  until the end of the template start tag or the end of the token that causes
  head to end, ever comes first. However, if the token implying the end of the
  head is a text token, bytes only to the end of the previous non-text token is
  considered. (This definition avoids depending on network buffer boundaries.)
* XML View Source now uses the code for syntax resembling an XML declaration
  instead of expat for extracting the internal encoding label.

Reftest are added as both WPT and Gecko reftests in order to test both http:
and file: URL scenarios. The Gecko tests retain the WPT <link> tags in order
to use the exact same bytes.

An encoding declaration has been added to a number of old tests that didn't
intend to test the new speculation behavior especially in the context of
https://bugzilla.mozilla.org/show_bug.cgi?id=1727750 .

Differential Revision: https://phabricator.services.mozilla.com/D125808
2021-12-08 11:34:20 +00:00
Norisz Fay e844dde8a6 Backed out changeset 3dfd3c94a105 (bug 1701828) for causing mochitest failures on browser_hsts_host.js CLOSED TREE 2021-12-07 12:05:44 +02:00
Henri Sivonen 37fade70f0 Bug 1701828 - meta charset rewrite. r=smaug
Implements https://github.com/whatwg/html/issues/6962 . Improves performance
when <meta charset> occurs in head but after the first kilobyte and aligns
behavior better with WebKit and Blink.

The main change is to avoid reloads when meta appears within head but
after the first kilobyte. Prior to this change, Gecko reloaded in that
case (in compliance with the spec!) even though WebKit and Blink did not.

Differences from WebKit and Blink:

* WebKit and Blink honor <meta charset> in <noscript>. This implementation
  does not.
* WebKit and Blink look for meta as if the tree builder was unaware of
  foreign content. This implementation is foreign content-aware. This
  makes a difference for CDATA sections that contain a > before the meta
  as well as style and script elements within foreign content. This could
  happen if the CDATA section that has mysteriously been introduced around
  a what looks like a meta tag also contains another prior tag-looking
  run of text.
* This implementation processes rel=preload and speculative loads that are
  seen before <meta charset> has been seen. WebKit and Blink instead first
  look for the meta and rewind before starting speculative parsing.
* Unlike WebKit, if there is neither an honored meta nor syntax resembling
  an XML declaration, detection from content takes place (as in Blink).
* Unlike Blink, if there is neither an honored meta nor syntax resembling
  an XML declaration, the detection from content is not dependent of network
  buffer boundaries.
* Unlike Blink, detection from content can trigger a reload at the end of
  the stream if the guess made at that point differs from the first guess.
  (See below for the definition of the input to the first guess.)

Differences from the old spec and Gecko previously:

* Meta inside script and RCDATA elements is no longer honored.
* Late meta is now ignored and no longer triggers a reload.
* Later meta counts as early enough meta: In addition to the previous
  meta within the first 1024 bytes, now a meta that started within the first
  1024 bytes counts as early enough. Additionally, if by then there hasn't
  been a template start tag and head hasn't ended, meta occurring before the
  earlier of the end of the head or a template start tag counts as early
  enough.
* Meta now counts as not-late even if the encoding label has numeric
  character reference escapes.
* Syntax resembling an XML declaration longer than a kilobyte is honored if
  there is no honored meta.
* If there is neither an honored meta nor syntax resembling an XML declaration,
  the initial chardetng scan is potentially longer than before: the first 1024
  bytes, the token spanning the 1024-byte boundary if there is such a token,
  and, if by then head hasn't ended and there hasn't been a template start tag
  until the end of the template start tag or the end of the token that causes
  head to end, ever comes first. However, if the token implying the end of the
  head is a text token, bytes only to the end of the previous non-text token is
  considered. (This definition avoids depending on network buffer boundaries.)
* XML View Source now uses the code for syntax resembling an XML declaration
  instead of expat for extracting the internal encoding label.

Reftest are added as both WPT and Gecko reftests in order to test both http:
and file: URL scenarios. The Gecko tests retain the WPT <link> tags in order
to use the exact same bytes.

An encoding declaration has been added to a number of old tests that didn't
intend to test the new speculation behavior especially in the context of
https://bugzilla.mozilla.org/show_bug.cgi?id=1727750 .

Differential Revision: https://phabricator.services.mozilla.com/D125808
2021-12-07 07:35:32 +00:00
Henri Sivonen 44da2f95ed Bug 1153920 - Conform ampersand error reporting to HTML spec. r=smaug
Created by inlining the `AMBIGUOUS_AMPERSAND` state in
https://github.com/validator/htmlparser/pull/30 back into the states
that transitioned to `AMBIGUOUS_AMPERSAND` in that PR by
Michael(tm) Smith.

Differential Revision: https://phabricator.services.mozilla.com/D81992
2021-09-02 11:13:37 +00:00
Michael[tm] Smith f8016b5f48 Bug 1541822 - Ensure doctype name is set to null when missing r=smaug
This change ensures that the tokenizer sets the doctype name to null
when the doctype name is missing in the input source.

Otherwise, without this change, the doctype name is set to the empty
string — which doesn’t conform to the requirements in the HTML spec, and
which causes us to fail 9 tests in the html5lib-tests suite.

Relates to https://github.com/validator/htmlparser/issues/35

Differential Revision: https://phabricator.services.mozilla.com/D122936
2021-08-19 10:02:57 +00:00
Michael[tm] Smith e82ccf150a Bug 1725946 - Conform tokenizer-only U+0000 NUL handling to spec r=smaug
This change brings the tokenizer’s handling of U+0000 NUL characters in
the DATA state and the CDATA section state into conformance with the
requirements in the HTML spec — for the case where only tokenization is
being performed, without tree construction; that is, the case where the
tokenizer() method is called, rather than parse() or parseFragment().

Specifically, the tokenization steps defined in the spec require that
when a U+0000 NUL is consumed in the DATA state or in the CDATA section
state, the parser must then emit a U+0000 NUL. But when performing tree
construction, the spec requires that when a U+0000 NUL is consumed, the
parser must instead emit a U+FFFD REPLACEMENT CHARACTER.

Without this change, the parser always emits a U+FFFD REPLACEMENT
CHARACTER — even when only tokenization is being performed. That causes
us to fail a number of tests in html5lib-tests suite.

For more background on the relevant behavior, see the following:

* https://www.w3.org/Bugs/Public/show_bug.cgi?id=9659
* https://github.com/whatwg/html/commit/d98f83e
* https://github.com/validator/htmlparser/commit/9b9c263

Relates to https://github.com/validator/htmlparser/issues/35

Differential Revision: https://phabricator.services.mozilla.com/D122721
2021-08-17 10:09:10 +00:00
Michael[tm] Smith 5de2048fa5 Bug 1319410 fixup - Stay in the COMMENT_LESSTHAN state, annotate fall-throughs. r=smaug
Differential Revision: https://phabricator.services.mozilla.com/D82163
2021-08-16 05:01:10 +00:00
Michael[tm] Smith f9ed6f998f Bug 1319410 - Make consecutive hyphens in comments a non-error r=smaug
Also allow `<!-->` at (IE conditional) comment end

See https://github.com/whatwg/html/pull/1356
See https://github.com/whatwg/html/pull/1456

Differential Revision: https://phabricator.services.mozilla.com/D82000
2021-08-16 05:01:09 +00:00
Henri Sivonen 61d5df5ae7 Bug 1724920 - Sort headers in the generated parts of the HTML parser. r=smaug
Differential Revision: https://phabricator.services.mozilla.com/D122225
2021-08-11 08:08:52 +00:00
Henri Sivonen e8bf7114c7 Bug 786797 - Check for integer overflow when computing new buffer sizes. r=smaug
Differential Revision: https://phabricator.services.mozilla.com/D102592
2021-01-21 17:02:58 +00:00
Michael[tm] Smith 31b329245a Bug 1650090 - Stop reporting HTML4-specific parse errors. DONTBUILD NPOTB r=alchen
Differential Revision: https://phabricator.services.mozilla.com/D82021
2020-08-03 11:03:02 +00:00
Henri Sivonen 759bddad47 Bug 1619230 - Use .get() in P::completedNamedCharacterReference(mViewSource). r=erahm
Differential Revision: https://phabricator.services.mozilla.com/D64927
2020-03-02 17:45:31 +00:00
Henri Sivonen abbf967ecd Bug 1618488 - Add .get() to mViewSource in calls to P::transition in the tokenizer. r=erahm
Differential Revision: https://phabricator.services.mozilla.com/D64552
2020-02-27 19:28:46 +00:00
Henri Sivonen 63a38ce016 Bug 1603919 - Use [[fallthrough]] instead of MOZ_FALLTHROUGH in generated HTML parser code. r=cpeterson
Differential Revision: https://phabricator.services.mozilla.com/D57451
2019-12-18 00:00:06 +00:00
Henri Sivonen aefec1e86d Bug 1584216 - Adjust tokenization of U+0000. r=alchen
Differential Revision: https://phabricator.services.mozilla.com/D47594
2019-10-01 12:26:55 +00:00
Henri Sivonen 85b708503b Bug 1562033. r=alchen
Differential Revision: https://phabricator.services.mozilla.com/D36361
2019-07-08 07:33:59 +00:00
Henri Sivonen c9bd88713b Bug 1555523. r=alchen
Differential Revision: https://phabricator.services.mozilla.com/D33639
2019-06-15 12:45:57 +00:00
Henri Sivonen f110425f3c Bug 1540675 - Handle line breaks correctly in comment end bang state. r=alchen.
Differential Revision: https://phabricator.services.mozilla.com/D25594
2019-04-15 13:40:14 +03:00
Sylvestre Ledru 1ad0cac654 Bug 1519636 - clang-format-8: Reformat recent changes to the Google coding style r=Ehsan
clang-format-8 upstream had some improvements wrt macros
See: https://reviews.llvm.org/D33440
This is why the diff is bigger than usual

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D26098
2019-04-05 21:41:42 +00:00
Csoregi Natalia 2ded4ab36d Backed out changeset 4ad80127f89f (bug 1519636) for bustage on MarkupMap.h and nsAccessibilityService.cpp. CLOSED TREE 2019-04-05 09:48:19 +03:00
Sylvestre Ledru ebe9c62d30 Bug 1519636 - clang-format-8: Reformat recent changes to the Google coding style r=Ehsan
clang-format-8 upstream had some improvements wrt macros
See: https://reviews.llvm.org/D33440
This is why the diff is bigger than usual

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D26098
2019-04-04 21:36:16 +00:00
Narcis Beleuzu 6b9ce2a82b Backed out changeset 389b6bbd76db (bug 1519636) for bustages on MarkupMap.h . CLOSED TREE 2019-04-05 00:27:56 +03:00
Sylvestre Ledru 194039a8d9 Bug 1519636 - clang-format-8: Reformat recent changes to the Google coding style r=Ehsan
clang-format-8 upstream had some improvements wrt macros
See: https://reviews.llvm.org/D33440
This is why the diff is bigger than usual

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D26098
2019-04-04 20:12:23 +00:00
Sylvestre Ledru e5a134f73a Bug 1511181 - Reformat everything to the Google coding style r=ehsan a=clang-format
# ignore-this-changeset
2018-11-30 11:46:48 +01:00
Emilio Cobos Álvarez 4d343d7da2 Bug 1506133 - More HTML parser cleanup post bug 1392185. r=hsivonen
Differential Revision: https://phabricator.services.mozilla.com/D11480
2018-11-12 14:37:09 +00:00
Jonathan Kingston 2d23795780 Bug 1466449 - Update Tokenizer.java and StackNode.java to initialize properties. r=hsivonen 2018-11-09 14:20:19 +01:00
Andi-Bogdan Postelnicu 318baca59d Bug 1453795 - Parser/HTML - Initialize member fields in classes/ structures. r=hsivonen 2018-06-14 11:46:13 +03:00
Sebastian Hengst d91e9954eb Backed out 4 changesets (bug 525063) on request from Andi. a=backout
Backed out changeset 516c4fb1e4b8 (bug 525063)
Backed out changeset 6ff8aaef2866 (bug 525063)
Backed out changeset bf13e4103150 (bug 525063)
Backed out changeset d7d2f08e051c (bug 525063)
2018-04-13 16:01:28 +03:00
Tristan Bourvon 6095241db8 Bug 525063 - Initialize uninitialized class attributes in m-c. r=ehsan 2018-04-10 21:11:02 +02:00
Chris Peterson 8e5d7c26d6 Bug 1424548 - Part 11: Regenerate HTML parser code with MOZ_FALLTHROUGH annotations. r=hsivonen
And stop suppressing -Wimplicit-fallthrough warnings. We no longer need to suppress these clang warnings because the generated parser code now includes MOZ_FALLTHROUGH annotations.
2017-12-01 01:05:26 -08:00
Chris Peterson f3d0ca9013 Bug 1424548 - Part 9a: clang-format generated HTML parser code. r=hsivonen 2017-12-21 00:02:43 -08:00
Olli Pettay 9ca325753f Bug 1418002 - Remove HTMLContentElement, r=jessica 2017-11-30 10:57:15 +02:00
Alphan Chen ab20180bfb Bug 500617 - Remove always-zero offsets from HTML5 parser. r=hsivonen
MozReview-Commit-ID: DT8oWxbbYNy
2017-10-23 14:42:21 +03:00
Nicholas Nethercote 7dbfdaf890 Bug 1400460 - Rename nsIAtom as nsAtom. r=hiro.
(Path is actually r=froydnj.)

Bug 1400459 devirtualized nsIAtom so that it is no longer a subclass of
nsISupports. This means that nsAtom is now a better name for it than nsIAtom.

MozReview-Commit-ID: 91U22X2NydP
2017-10-03 09:05:19 +11:00
Henri Sivonen fa792cb56c Bug 1375701 - Atomize class attribute value in the parser in the innerHTML case. r=Ehsan
MozReview-Commit-ID: CKyGlzYS15e
2017-08-11 09:22:57 +03:00
Sebastian Hengst 7e433bbe75 Backed out changeset fabf345eec6e (bug 1375701) for bustage at parser/html/nsHtml5String.h:143:3: bad implicit conversion constructor for 'nsHtml5String'. r=backout on a CLOSED TREE 2017-08-15 16:28:10 +02:00
Henri Sivonen 8d12eac1f2 Bug 1375701 - Atomize class attribute value in the parser in the innerHTML case. r=Ehsan
MozReview-Commit-ID: CKyGlzYS15e
2017-08-11 09:22:57 +03:00
Henri Sivonen 3b532a7391 Bug 483155 - Put content creator function pointers onto nsHtml5ElementName. r=smaug
MozReview-Commit-ID: E2AAx7Zz2UF
2017-07-04 11:00:03 +03:00
Sebastian Hengst b20f04d43a Backed out changeset 5b706cd241cb (bug 483155) for Valgrind failure on Linux x64 opt. r=backout 2017-08-11 10:48:10 +02:00
Henri Sivonen b2625c0d7e Bug 483155 - Put content creator function pointers onto nsHtml5ElementName. r=smaug
MozReview-Commit-ID: E2AAx7Zz2UF
2017-07-04 11:00:03 +03:00
Sylvestre Ledru 576cac5ec7 Bug 1381253 - Remove redundant control flow declarations rs=ehsan
MozReview-Commit-ID: FFxP4aMCbOL
2017-07-15 19:03:04 +02:00
Henri Sivonen 43720a8e05 Bug 1269490 - Merge nsHtml5Atoms into nsGkAtoms. r=wchen
MozReview-Commit-ID: EcfnXkFuJy3
2017-04-20 11:47:02 +03:00
Henri Sivonen 3b050334d4 Bug 1358037 - Inline the methods of nsHtml5ElementName and nsHtml5AttributeName. r=wchen
Moving methods that refer to constant to the header that defines the
constants makes it infeasible to #define the constants at the end
of the header. To avoid introducing a new pass over the Java AST,
this patch turns the primitive constants that were
previously #defined into C++ contants, which, according to
Stroustrup 3rd ed., are handled at compile-time and don't consume
storage.

MozReview-Commit-ID: JnaDgW2OM7t
2017-04-20 10:34:32 +03:00
Henri Sivonen 759c770926 Bug 1355479 - Flatten attribute storage in the HTML parser to AutoTArray to avoid malloc. r=wchen
MozReview-Commit-ID: 77Hqo24F2nB
2017-04-13 11:14:48 +03:00
Henri Sivonen 89dc7f5ddb Bug 1355779 - Skip interned nsHtml5ElementName lookup for Custom Elements (ones with hyphen). r=wchen
MozReview-Commit-ID: 4xQlhExyo7d
2017-04-12 14:28:49 +03:00
Henri Sivonen 6274301b73 Bug 1355769 - Avoid malloc for nsHtml5ElementName when processing a non-interned element name. r=wchen
MozReview-Commit-ID: 4In5wo0flOB
2017-04-12 13:21:03 +03:00
Henri Sivonen cb263a268d Bug 1347737 - Introduce a new non-heap-allocated type for holding nsStringBuffer* in the HTML parser. r=wchen.
MozReview-Commit-ID: Gn9fXroxQhY
2017-03-20 14:45:15 +02:00
Henri Sivonen f5eeedfb21 Bug 1309195 - Mark strBuf as empty after the contents have been used or ignored. r=wchen.
MozReview-Commit-ID: 5KU62o2IMPs
2016-10-19 10:38:20 +03:00
Henri Sivonen 4eb7f7145d Bug 1286911. r=wchen.
MozReview-Commit-ID: hzAu0jKAxt
2016-09-27 13:47:53 +03:00