find_paths_and_tags() can instantiate a BuildReader to read moz.build
metadata. We switch `mach test` to call this to avoid some redundant
code.
To facilitate `mach try`'s use case, we add test flavors to the
result of find_paths_and_metadata() and change the function name to
reflect its more generic purpose.
MozReview-Commit-ID: Lcz05W8g2dt
--HG--
extra : rebase_source : 17d6f23275f4eed1bd544a9b2cd991214625dabc
This will make the return value more easily extensible and will help
consumers know what the data structures represent.
MozReview-Commit-ID: DaeYsqfMW37
--HG--
extra : rebase_source : 440d16f0604b21f0ec885e1bca5066c32b18f641
It's easy to mess up the scoping so that (a) the label is pushed and then
immediately popped, and/or (b) the string doesn't live long enough. It's also
easy to do a utf16-to-utf8 conversion unnecessarily when the profiler is
inactive. This patch splits that macro into three new ones that are harder to
mess up.
- AUTO_PROFILER_LABEL_DYNAMIC_CSTR: same as current.
- AUTO_PROFILER_LABEL_DYNAMIC_NSCSTRING: for nsCStrings.
- AUTO_PROFILER_LABEL_DYNAMIC_LOSSY_NSSTRING: for nsStrings.
--HG--
extra : rebase_source : 3e2bbec4737b696e1c86579ae54be4cb3186c100
Most cases where the pointer is stored into an already-declared variable can
trivially be changed to MakeNotNull<T*>, as the NotNull raw pointer will end
up in a smart pointer.
In RAII cases, the target type can be specified (e.g.:
`MakeNotNull<RefPtr<imgFrame>>)`), in which case the variable type may just be
`auto`, similar to the common use of MakeUnique.
Except when the target type is a base pointer, in which case it must be
specified in the declaration.
MozReview-Commit-ID: BYaSsvMhiDi
--HG--
extra : rebase_source : 8fe6f2aeaff5f515b7af2276c439004fa3a1f3ab
They are equivalent -- both infallible, both requiring measuring the length of
the string -- but moz_xstrdup is much more readable. (One place deals with
16-bit strings and so uses NS_strdup instead, which is also infallible.)
The patch also removes some failure-path code that will never execute due to
the infallibility.
--HG--
extra : rebase_source : 115574cf55db90b60e6bee59e5dc6ee409374159
Currently the choices of SingleLineStyle and MultiLineStyle are sub-optimal,
which makes debugging of profiles difficult.
- A large chunk of the profile is put on a single line, which is very hard to
read in a text editor.
- Other short and repetitive constructs are unnecessarily spread across
multiple lines.
This patch makes things more readable.
This adds the magic CLOSED TREE string to the commit message for both
try syntax and the fuzzy frontend.
MozReview-Commit-ID: 492UME3VcJN
--HG--
extra : rebase_source : 425a83d6370609e1e162379213d547458c8e8cf2
The browser JUnit 3 test suite has never run in automation, and no
test has been added or updated since 2016. Half of the existing
"tests" are broken on my local device. This allows to clean several
Makefile's related to this undeveloped testing approach, which won't
be pursued post-Gradle.
MozReview-Commit-ID: 1EbGMLntuyR
--HG--
extra : rebase_source : b4751f96851a8df61c0333a3e0ade3805b280dfe
This allows a bunch of additional stuff to be removed: ContextStateTracker,
ContextStateTrackerOGL, and GPUMarkerPayload.
--HG--
extra : rebase_source : 879045a9f9ac31ca0beb596964c6c3ef30283a53
Because it just doesn't control any behaviour within the profiler, and it just
duplicates gfxPrefs::LayersDumpTexture().
With this gone, PROFILER_FEATURE_ACTIVE can also be removed.
--HG--
extra : rebase_source : d6718894b8a9332cf73729ea6b7bd2de348817bd
This adds all the common arguments to |mach try empty|, including preset
arguments which will just be ignored if specified. The commit message can be
changed with:
./mach try empty -m DONTBUILD
MozReview-Commit-ID: 5RYfLBOIX1g
--HG--
extra : rebase_source : 412cbc50c2901a97893779a73a7f70d5ee6e2d72
It's easy to mess up the scoping so that (a) the label is pushed and then
immediately popped, and/or (b) the string doesn't live long enough. It's also
easy to do a utf16-to-utf8 conversion unnecessarily when the profiler is
inactive.
This patch splits that macro into three new ones that are harder to mess up.
- AUTO_PROFILER_LABEL_DYNAMIC_CSTR: same as current.
- AUTO_PROFILER_LABEL_DYNAMIC_NSCSTRING: for nsCStrings.
- AUTO_PROFILER_LABEL_DYNAMIC_LOSSY_NSSTRING: for nsStrings.
--HG--
extra : rebase_source : 53c8b43b6a1be06d00618a133e28bf95c46a3ba3
It's easy to mess up the scoping so that (a) the label is pushed and then
immediately popped, and/or (b) the string doesn't live long enough. It's also
easy to do a utf16-to-utf8 conversion unnecessarily when the profiler is
inactive.
This patch splits that macro into three new ones that are harder to mess up.
- AUTO_PROFILER_LABEL_DYNAMIC_CSTR: same as current.
- AUTO_PROFILER_LABEL_DYNAMIC_NSCSTRING: for nsCStrings.
- AUTO_PROFILER_LABEL_DYNAMIC_LOSSY_NSSTRING: for nsStrings.
--HG--
extra : rebase_source : 59f77df0124249bfd11fee3585420a17b4201d37
Currently the Gecko Profiler defines a moderate amount of stuff when
MOZ_GECKO_PROFILER is undefined. It also #includes various headers, including
JS ones. This is making it difficult to separate Gecko's media stack for
inclusion in Servo.
This patch greatly simplifies how things are exposed. The starting point is:
- GeckoProfiler.h can be #included unconditionally;
- everything else from the profiler must be guarded by MOZ_GECKO_PROFILER.
In practice this introduces way too many #ifdefs, so the patch loosens it by
adding no-op macros for a number of the most common operations.
The net result is that #ifdefs and macros are used a bit more, but almost
nothing is exposed in non-MOZ_GECKO_PROFILER builds (including
ProfilerMarkerPayload.h and GeckoProfiler.h), and understanding what is exposed
is much simpler than before.
Note also that in BHR, ThreadStackHelper is now entirely absent in
non-MOZ_GECKO_PROFILER builds.
This patch does the following.
- Makes the TracingKind argument non-optional.
- Puts the UniqueProfilerBacktrace argument last in the second variant.
- Reorders AutoProfilerTracing to match the order of the profiler_tracing()
declarations.
--HG--
extra : rebase_source : 8e9acdaf777c642cd854570771a3f96da6d524d1
(Path is actually r=froydnj.)
Bug 1400459 devirtualized nsIAtom so that it is no longer a subclass of
nsISupports. This means that nsAtom is now a better name for it than nsIAtom.
MozReview-Commit-ID: 91U22X2NydP
--HG--
rename : xpcom/ds/nsIAtom.h => xpcom/ds/nsAtom.h
extra : rebase_source : ac3e904a21b8b48e74534fff964f1623ee937c67
It is broken on the version of perl on the legacy buildbots, which is the only
place that this branch of the code runs.
MozReview-Commit-ID: 1Urw6mNOEHf
--HG--
extra : rebase_source : d599484b0f2062b7e3178e761c80b620624852a3
Vendor new dependency for compare-locales, python-fluent.
This is the 0.4.2 release of python-fluent.
Also, make mach command a bare minimum wrapper.
The compare-locales mach command used to own a couple of defaults
and opinions.
Now that those opinions and defaults are in the Makefiles, this
command can be much simpler. As a side effect, this should
make the thunderbird port easier, where none of the mach defaults
worked.
Update l10n.mk for compare-locales 2.x:
The command line interface is different now, with positional arguments.
Also, the l10n config file isn't a default anymore (that never worked
for thunderbird).
And the merge dir is now the parent of the locale dir, much like
we already anticipated in the code.
MozReview-Commit-ID: DxVVKyVSt5y
--HG--
extra : rebase_source : 7c591fea8c7d9ecef615fb56f9359d4f1bd4b340
This was preventing us from running ./mach try fuzzy from a subdirectory of the
topsrcdir. This also fixes taskgraph/utils/verify.py to find the docs directory
based off of topsrcdir instead of cwd. This was needed as cwd was being set at
import time.
MozReview-Commit-ID: CgQqD6bQ5q4
--HG--
extra : rebase_source : 4d863d442b0e715b3fe386c43b4077054c10dc70
Running |./mach try -j <job>| may fail if there are changes in the local
working copy because the command will attempt to provide test paths based
on those changes and subsequently require platforms to be specified on the
basis of those paths. This commit makes this auto-detection only run when
a particular option is passed so this doesn't interfere with the common
case of simply running a selected job on try.
MozReview-Commit-ID: F3RBgDAYi27
--HG--
extra : rebase_source : 81e7774d7b1af31ee188b76afdac5062db1ab811
This will allow us to inspect Try pushes to see how many are using
`mach try`.
The commit message for `mach try fuzzy` already advertised its usage.
We tweak the syntax a little so it isn't redundant.
MozReview-Commit-ID: 6kmYRKp0ffF
--HG--
extra : rebase_source : 9dd7caa5404ba235e6a4071802000b8bb740a6b7
The excluded directories aren't being properly handled in the py2/py3 compat
linters. In order for FileFinder to apply the exclusions properly they need
to either be relative to or contained by the base.
This means that currently the following will work:
./mach lint -l py2 <topsrcdir>
./mach lint -l py2 testing/mochitest
But this is broken:
./mach lint -l py2 testing
This change fixes the compat linters so exclude paths will be made relative
to the FileFinder base before passing them in. Any exclude not contained by
the base is simply discarded as it won't be relevant to that FileFinder
instance anyway.
MozReview-Commit-ID: LJx97TvKlSa
--HG--
extra : rebase_source : b8f0cb20ec5a88a33c26ace699ed9216b070f443
check_compat.py was adapted from gps' check-py3-compat.py in mercurial:
https://www.mercurial-scm.org/repo/hg/file/tip/contrib/check-py3-compat.py
The py3 linter simply runs ast.parse(f) for each file being linted. Any syntax errors
are formatted as mozlint results and dumped to stdout as json. I looked into also
importing the file (using 3.5+'s importlib.util.spec_from_file_location), but there
were too many problems:
1. Lots of false positives (e.g module not found)
2. Some files seemed to run indefinitely on import
I decided to punt on importing for now, we can always investigate in a follow-up.
The py2 linter runs ast.parse(f), and also checks that the file has:
from __future__ import absolute_import, print_function
Initially every python file in the tree is excluded from the py2 check, though
at least this makes it easy to find+fix, and new files in un-excluded
directories will automatically be linted.
MozReview-Commit-ID: ABtq9dnPo9T
--HG--
extra : rebase_source : 60762937284d498514cd020b90cbfd2ba23f0b70
This will allow us keep python related linting files in the same place.
MozReview-Commit-ID: ABtq9dnPo9T
--HG--
rename : tools/lint/flake8_/__init__.py => tools/lint/python/__init__.py
rename : tools/lint/flake8_/__init__.py => tools/lint/python/flake8.py
rename : tools/lint/flake8_/flake8_requirements.txt => tools/lint/python/flake8_requirements.txt
extra : rebase_source : 2568bc0bf8f4adbf8e0be73a54d5da068a8d81b0
This adds a new morph template for modifying a task's env and the corresponding glue
to specify it from |mach try fuzzy|. It can be used like:
./mach try fuzzy --env FOO=1 --env BAR=baz
This will simply set those environment variables in *all* tasks. We could add the
ability to only specify it for a subset of tasks in the future, but that seems like
a hard problem that probably isn't worth it.
MozReview-Commit-ID: C4sokv886PU
--HG--
extra : rebase_source : 64c9105848e6becd33fae63aeabc3423cfda3bdf
To use this, create a ~/.mozbuild/machrc file and add:
[try]
default = fuzzy
Now, running |mach try| without a subcommand will default to the fuzzy selector. The syntax selector can still be
run with |mach try syntax|.
MozReview-Commit-ID: Ai6f8cW3Swq
--HG--
extra : rebase_source : 6a107478f7fbd7d8deb4882f0892963fed755b33
This allows .flake8 files to override one another, and fixes a pretty bad known
bug with our flake8 implementation. For example, say we have a .flake8 file at:
/foo/.flake8
Before this patch, if we ran |mach lint foo/bar|, the configuration defined in
that .flake8 file wouldn't get picked up. It would only work if running the
specific directory that contains it, e.g |mach lint foo|.
This change additionally allows multiple .flake8 files to be used. So if
there's one defined at both:
/.flake8
/foo/.flake8
Then running |mach lint foo/bar| will first apply the root .flake8, then the
one under /foo (overriding earlier configuration).
This bug still doesn't make flake8 configuration perfect though. Any directory
containing a .flake8 file still needs to be explicitly listed in the "include"
section of /tools/lint/flake8.yml. Otherwise in the example above, if running
|mach lint /|, it wouldn't be able to find /foo/.flake8. This is a hard problem
and is likely best solved by fixing flake8's upstream configuration handling.
Unfortunately this means we still can't switch from a whitelist to a blacklist.
MozReview-Commit-ID: 3DZAi1QHYYo
--HG--
extra : rebase_source : 51298c5847f6c2792581d9b312c87b70fa716ee1
This fixes a regression from bug 1384593. I was going to add a test for this, but it would be
a better use of time to fix bug 1185599 and add a test there. We already have vcs tests in other
parts of the tree so consolidating them into a single module is likely easier than standing up
another one-off vcs tester.
MozReview-Commit-ID: E51Tb1qC9Wb
--HG--
extra : rebase_source : f957b1857d162fd229dcef0e87668dd270512cf7
This reduces the time taken to upload the Firefox docs from ~30s to
~5s (per invocation).
MozReview-Commit-ID: DxOrvxvVn42
--HG--
extra : rebase_source : b170fb728267c43484573e0a57ebc96512eb0e7c
Previously, we uploaded the main Firefox tree docs to /.
In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.
In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:
* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)
For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.
While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.
MozReview-Commit-ID: 8Gl6l2m6uU4
--HG--
extra : rebase_source : e56885092c12eb8cc76e5e7300f938be566e3e5a
extra : intermediate-source : 8509af1e135177a93460270b27f263c10a62d996
extra : source : 71b4f32caf209fe9dffc340c0b8ccb51ac79c7de
We now have an --upload flag to control whether upload is performed.
We don't inline it because we want to maintain a "firewall" between
regular docs and all the extra packages and imports needed for S3.
MozReview-Commit-ID: DVKhsS545gp
--HG--
extra : rebase_source : 7fee832145189be882db0f0ca057eda6158e0492
Instead of doing the file finding inside s3_upload(), the function now
takes the output of distribution_files().
The new code is much simpler.
MozReview-Commit-ID: 43i2Alvyu5i
--HG--
extra : rebase_source : 29d840c65acec794749edfda0e47193806f387f4
extra : source : 5a758defcf2f1a17de851e703388469300789ac5
We try to keep mach_commands.py files as minimal as possible.
MozReview-Commit-ID: I4kvZtDjqGd
--HG--
extra : rebase_source : 7b0af45661fc2a8b0b906a569aa95df7d3e0513b
By using mozpack, we get deterministic archives. This also makes the
task command simpler.
MozReview-Commit-ID: EPI7tuGQuso
--HG--
extra : rebase_source : babcbf3d5607480321106b40eaa4598c0faee188
An upcoming commit will want to put something outside the
format-specific output directory.
MozReview-Commit-ID: 2kYDREddpN
--HG--
extra : rebase_source : 28454f71b3abbd094048edb568b4f5e8d50881d8
I didn't fix all violations in this file. Something is better than
nothing.
MozReview-Commit-ID: BDdWhJfkPVk
--HG--
extra : rebase_source : caa0a5908c2cb37b8cbb1c5ee004d43e3a0b883a
For regular builds, we build the mar from an unpackaged exe on windows.
For repacks, we just built that from our l10n-stage directory,
don't unpack again.
MozReview-Commit-ID: 8gQ9G23RgzB
--HG--
extra : rebase_source : 3d9289cee054737c136b304682563d59aeaecb6a
This also prints the last known query used in the commit message. This won't be accurate as
users can delete + enter multiple queries in a single session, but will give users a slightly
better idea of what was scheduled by glancing at the commit message.
MozReview-Commit-ID: 93FbEmMvd9t
--HG--
extra : rebase_source : fc09dd145a5e88e5bb77e4e483552b579e90f3ca
We should not be declaring forward declarations for nsString classes directly,
instead we should use nsStringFwd.h. This will make changing the underlying
types easier.
--HG--
extra : rebase_source : b2c7554e8632f078167ff2f609392e63a136c299
The main motivation behind this change is to make testing easier, so e.g:
./mach try fuzzy --no-push
and
./mach try syntax --no-push
both work the same way.
MozReview-Commit-ID: LmjA3Kq7xKN
--HG--
extra : rebase_source : 49beb7e1ae0f7502966ebadc4d9c37cae4357df4
This will give us a good way to share arguments between subcommands.
MozReview-Commit-ID: KmXRj8TBvYK
--HG--
extra : rebase_source : 9ff275af226cfe65697b980d1b19f9ca9e1dad5f
This will remove the need to sniff class types. The 1 in-tree
consumer doing this has been converted.
MozReview-Commit-ID: I8cUa8J54VE
--HG--
extra : rebase_source : 4c24adaf7eb9d62678ac78604e819a7376d4073b
There's a lot of overhead to loading scripts from <script> tags in
browser.xul. They're loaded asynchronously, and can't begin loading until
after we begin loading browser.xul. They're loaded using off-main-thread
compilation, which means we need to create and merge a new JS zone for each
script we load that way. They don't benefit from the startup cache at all.
If we load those scripts using the subscript loader, they're loaded from the
preloader cache, and begin loading as early as possible. And they're all
loaded into a single off-thread zone, which means there are less zones to
create and merge. From a quick test, this seems to save about 20-40ms on
Windows ts_paint talos runs.
The main downside is that subscript loader scripts don't benefit from lazy
parsing, so we might wind up increasing memory usage if most of the functions
in those scripts aren't used very often. But we should hopefully be able to
fix that by migrating more browser JS to lazy loading.
MozReview-Commit-ID: Lozb0d0QweC
--HG--
extra : rebase_source : 069a003dfd1d6d66540da814a1fd19742c4fca26
extra : amend_source : b93e0ef7b320c99f4a27ddf6e8af6a681b6dba0f
This makes configuration files for yamllint work a bit better. It's still not perfect, but it's an improvement
on the current situation.
MozReview-Commit-ID: IKxgQm1a7bP
--HG--
extra : rebase_source : 051fafe21337f0557ee39ec71c90e74fd61d3da7
Suppress these warnings intead of fixing them because libfuzzer is a third-party llvm library:
tools/fuzzing/libfuzzer/FuzzerDriver.cpp:450:10: warning: 'return' will never be executed [-Wunreachable-code-return]
tools/fuzzing/libfuzzer/FuzzerDriver.cpp:663:12: warning: 'return' will never be executed [-Wunreachable-code-return]
MozReview-Commit-ID: 9mWEuc5wCn9
--HG--
extra : rebase_source : e880a5da2dac2f8c79139646713b02feb4ddc9b7
This uses the new templating mechanism in taskgraph to schedule artifact builds when using
|mach try fuzzy|. Like |mach try syntax|, this will automatically be enabled if
--enable-artifact-builds is set in the mozconfig. The --artifact/--no-artifact arguments
can be used to override the default.
MozReview-Commit-ID: J8HVZzOt4mX
--HG--
extra : rebase_source : 130fa85ed24ef8f0fe5a713f254bbffd1a83b5a7
