mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-10-19 08:15:31 +00:00
933b3522b8
`ply`, [by design](https://github.com/dabeaz/ply/issues/79), does not produce reproducible table files; hence bug 1633156. (Note that this was *always* true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to [3.7](https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights).)
In any other circumstance I would consider submitting a patch to `ply` to fix this, but as of the [in-progress version 4.0 of the library](https://github.com/dabeaz/ply/blob/master/CHANGES), it doesn't even emit this cached data any more, and indeed the [latest version of the code](1fac9fed64/ply) doesn't even call `open()` at all except to do logging or to read the text data to be parsed from `stdin`. So if we were going to pin our future on `ply` and upgrade to later versions of the library in the future, we would have to live in a world where `ply` doesn't generate cached table files for us anyway.
Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the `CHANGES` file from that repository:
```
PLY no longer writes cached table files. Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
```
In practice, I have found this to be true; namely, `./mach build pre-export export` takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to `yacc()` takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the `export` tier now. This isn't *nothing*, but in my opinion it's also not nearly long enough where it would be a concern given how long `export` already takes.
That `CHANGES` file also stresses that if caching this data is important, we have the option of doing so via `pickle`. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.
Differential Revision: https://phabricator.services.mozilla.com/D73484
37 lines
1.1 KiB
Makefile
37 lines
1.1 KiB
Makefile
# This Source Code Form is subject to the terms of the Mozilla Public
|
|
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
|
GARBAGE_DIRS += _ipdlheaders
|
|
GARBAGE += $(wildcard *.pyc $(srcdir)/ipdl/*.pyc $(srcdir)/ipdl/cxx/*.pyc) ipdl.track
|
|
|
|
ifdef COMPILE_ENVIRONMENT
|
|
|
|
# This file is generated by the moz.build backend.
|
|
include ipdlsrcs.mk
|
|
|
|
include $(topsrcdir)/config/rules.mk
|
|
|
|
ipdl_py_deps := \
|
|
$(wildcard $(srcdir)/*.py) \
|
|
$(wildcard $(srcdir)/ipdl/*.py) \
|
|
$(wildcard $(srcdir)/ipdl/cxx/*.py) \
|
|
$(wildcard $(topsrcdir)/other-licenses/ply/ply/*.py) \
|
|
$(NULL)
|
|
|
|
# NB: the IPDL compiler manages .ipdl-->.h/.cpp dependencies itself,
|
|
# which is why we don't have explicit .h/.cpp targets here
|
|
ipdl.track: $(ALL_IPDLSRCS) $(srcdir)/sync-messages.ini $(srcdir)/message-metadata.ini $(ipdl_py_deps)
|
|
$(PYTHON3) $(srcdir)/ipdl.py \
|
|
--sync-msg-list=$(srcdir)/sync-messages.ini \
|
|
--msg-metadata=$(srcdir)/message-metadata.ini \
|
|
--outheaders-dir=_ipdlheaders \
|
|
--outcpp-dir=. \
|
|
$(IPDLDIRS:%=-I%) \
|
|
$(ALL_IPDLSRCS)
|
|
touch $@
|
|
|
|
export:: ipdl.track
|
|
endif
|
|
|