The name 'relativedir' is ambiguous - it is unclear whether it is a
relative srcdir or objdir. Rename it to 'relsrcdir' in the
ContextDerived() object to match the naming used in Context() so it is
obvious that it is a relative srcdir.
Most of these are a straight text replacement from relativedir to
relsrcdir, except for tup.py:_get_backend_file(), which was supposed to
be using an objdir in the first place.
MozReview-Commit-ID: 9eFHOCMofq5
--HG--
extra : rebase_source : bcd981953fcd72a8c049de02eca9cf31c04dff3c
exe_7z_archive.py runs during the MinGW build L10N check step, and
hardcodes 7z as the 7zip executable. This works on Windows, but not
Linux. We need to pass it the correct executable for 7zip, which is
located during configure.
However, repacks (repackage-winXX-nightly) don't do configure, and
don't have the tools to do configure. So we leave the default
command in the python script if one is not supplied.
MozReview-Commit-ID: B7GEKRYEJTD
--HG--
extra : rebase_source : 10ec7e688d53341625217306e88f2e647195ce8d
The script behind GENERATED_FILES currently opens output files in text mode,
which means that they wind up with CRLF line endings on Windows. With
switching updater.ini to use LOCALIZED_GENERATED_FILES, this means that it
will wind up with different line endings than it currently has.
Changing this to use binary format means that we'll have LF line endings
everywhere, which shouldn't harm anything on Windows as most of our generated
files are source files anyway.
MozReview-Commit-ID: 7rTUDtVGL82
--HG--
extra : rebase_source : 53a604c225477ad02e439b7b9ace587aefd0785a
The moz.build files that specify USE_YASM = True will get the value of
AS_DASH_C_FLAG and AS overwritten in PassthruVariables. We can save
those in the BackendTupfile and use them in place of the configure or
default settings as appropriate. This enables compilation of those .s
files that are built with yasm.
MozReview-Commit-ID: J66q8nKQ0an
--HG--
extra : rebase_source : 0ee97ef7a2122b42f1d21c473539a2e6352bd9ab
This accounts for LOCAL_INCLUDES in the moz.build files, as well as the
default INCLUDES specified in config.mk that are used for host
compilation. Since some of the HOST_CFLAGS were also used for linking,
those flags are split off into HOST_C{XX}_LDFLAGS so that
the linker-only flags can be placed in those variables.
MozReview-Commit-ID: J1LxIZVeFJ
--HG--
extra : rebase_source : ed7293604e5428e3124f1ecfb2b706e087436b72
The filenames that these objects generate are passed into the _handle_*
methods instead of with a Sources object, so they need to be added to
the BackendTupfile's list of sources separately.
MozReview-Commit-ID: GoqhiJ3Ismm
--HG--
extra : rebase_source : 60e53c2c28a93c543a99bff0463b2935b2826e09
Both SFLAGS and ASFLAGS are used to compile assembly, but SFLAGS include
DEFINES and LOCAL_INCLUDES whereas ASFLAGS do not. It seems easiest to
just separate them into two different ComputedFlags values so that the
backend can distinguish between the two types.
MozReview-Commit-ID: Bkm3621ImJG
--HG--
extra : rebase_source : 420204e37d591512f700d77b780939d20c2feeb0
In most cases, relobjdir is the same as relativedir. However they are
different for some objects, notably with nss gyp handling. We need to
output files in relobjdir, so use that when getting the BackendTupfile.
This puts generated files like certdata.c in the directory that
moz.build is expecting.
MozReview-Commit-ID: DG29OulOKAz
--HG--
extra : rebase_source : 76dbaeb92fb4112b664768653548caf57242d90d
The srcdir does not necessarily directly correspond to the objdir, so it
doesn't make sense to tie them together in BackendTupfile. Since the
srcdir was only used for the IPDL sources, we can just replace that
usage with a local variable.
MozReview-Commit-ID: By0N30VTKhh
--HG--
extra : rebase_source : 5cc35da4fadcc53132d459173bcc29ac5b0a57bc
This change makes the recursive make backend emit slightly different rules
when handling LOCALIZED_GENERATED_FILES vs. GENERATED_FILES.
First, localized file generation is always done in the libs tier.
Second, inputs are allowed to be locale-dependent, which is determined by
the path starting with `en-US/`. These inputs will be run through MERGE_FILE
to determine the actual file path to pass to the script.
Third, the file_generate action now accepts a `--locale` option, and it
gets passed the value of `$(AB_CD)` when generating localized files. If this
option is passed it is also passed as a keyword argument `locale` to the
generation function.
Fourth, the make rules for localized files include an additional dependency
on FORCE when running l10n repacks, so that the targets will always be
rebuilt in that situation.
MozReview-Commit-ID: BfgR8MxxJXZ
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build
extra : rebase_source : 1d123afbad4f3d949a2f13f1685f30b1e3069e97
LOCALIZED_FILES and LOCALIZED_GENERATED_FILES are analogs of FINAL_TARGET_FILES
and GENERATED_FILES, but they receive special handling in the recursive
make backend so that l10n repacks work properly. To this end, we support
using the output of LOCALIZED_GENERATED_FILES in LOCALIZED_FILES, but not
mixing localized with non-localized targets.
MozReview-Commit-ID: GCJAUfUG8OZ
--HG--
rename : python/mozbuild/mozbuild/test/frontend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/localized-files-from-generated/moz.build
rename : python/mozbuild/mozbuild/test/frontend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/localized-files-not-localized-generated/moz.build
extra : rebase_source : c67f87782a5992734948da79c0cdbe64e23ed437
This change adds LOCALIZED_GENERATED_FILES, which emits GeneratedFile objects
just like GENERATED_FILES. It also adds a `localized` field to GeneratedFile
which will be `True` for objects emitted from LOCALIZED_GENERATED_FILES.
MozReview-Commit-ID: 3iWGLMkbF2C
--HG--
rename : python/mozbuild/mozbuild/test/frontend/data/generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/localized-generated-files/moz.build
extra : rebase_source : 36446da5d367925655e7adfa3e4133be843f99d3
This makes it a bit easier to share with other parts of the tree,
like test and linting.
MozReview-Commit-ID: 8Gzk8uOF5zK
--HG--
extra : rebase_source : 9354614c78481ca4cbe0327501018a95792e9351
This is a new module that will provide a place to store some common
abstractions around the 'blessings' module. The main entrypoint is:
from mozterm import Terminal
term = Terminal()
If blessings is available, this will return a blessings.Terminal()
object. If it isn't available, or something went wrong on import,
this will return a NullTerminal() object, which is a drop-in
replacement that does no formatting.
MozReview-Commit-ID: 6c63svm4tM5
--HG--
extra : rebase_source : 9ab221774d92a418d9b098d79bb2c88f75d937f8
- aarch64 has GNU-free toolchain thus /usr/bin/as is N/A by default
but GNU as is required to build bundled ICU.
- Downstream Python splits sqlite3 C module into a separate package
to reduce optional dependencies but _sqlite3 is used by mozprofile.
MozReview-Commit-ID: Lh2ktcqh16f
--HG--
extra : rebase_source : bbcbe437132ecfce0f3e6e7b2005d0e8eac65836
I believe all backends will need to know which GeneratedFiles are needed
before compilation can start, so we should make that an attribute of the
object. Each backend can then make its own decision about what to do
with the different types of GeneratedFiles.
MozReview-Commit-ID: ByburRx540b
--HG--
extra : rebase_source : ccfee5b569da432cb61882f2f6ea518f1ccbfa07
This makes things slightly more inconvenient (having to set two
environment variables instead of one for the simplest case) until a few
patches down the line, when DMD is statically linked, at which point it
will get down to one environment variable every time.
--HG--
extra : rebase_source : 08dc3c05318b572ae1026227d0369fa8bf21b20f
This makes things slightly more inconvenient (having to set two
environment variables instead of one for the simplest case) until a few
patches down the line, when DMD is statically linked, at which point it
will get down to one environment variable every time.
--HG--
extra : rebase_source : 08dc3c05318b572ae1026227d0369fa8bf21b20f
The Firefox UI tests were packaged wrongly, and as such didn't use
the real path as in tree. This patch fixes that by separating the
packaging logic for the harness, and the tests.
Also it updates the mozharness script to run the Firefox UI tests
command by using the test folder as current working directory. This
will make sure that the relative path to the tests is reported.
It's identical to the location in the tree.
MozReview-Commit-ID: 3YVfCw4RWfV
--HG--
extra : rebase_source : 355ceef605c95c16715733f02fd85fc388ce28b3
Because the default "fast" variant uses fork() on !windows, death tests
are dangerous, as they themselves say. There are race conditions
involving locks that lead to dead locks in the death test process while
disabling the crash reporter (currently, but this could happen for
different code, even the tested code itself).
See https://bugzilla.mozilla.org/show_bug.cgi?id=1419196#c7 for details.
Using the "threadsafe" variant creates new processes for each death
test. This is notably slower, but can't dead-lock because of some random
lock being held by some random other thread at the moment fork occurred.
--HG--
extra : rebase_source : 56bf678bc9a6588751520549d57db7293134e1f8
This combines and expands the list of directories where compilation
currently works in tup, going backwards through the alphabet. Not all of
these directories actually contain compileable code, but this makes it
obvious which top-level directories are not yet enabled. It is likely
that other directories will compile successfully as well - this is
simply a staging point.
MozReview-Commit-ID: Arsk9Oq5XTV
--HG--
extra : rebase_source : 3b6318a41dcdd0db595b4cf0530c4489e46359fe
Flags like -DMOZ_APP_NAME="firefox" need to be sent through the shell as
'-DMOZ_APP_NAME="firefox"', otherwise the double-quotes get eaten and
the string is invalid.
MozReview-Commit-ID: 7QN3VTMAY77
--HG--
extra : rebase_source : 8142586888a7e3b609753d502f7db109bdafe8c4
This patch fixes a bug in the FasterMake backend and adds a new
test_fastermake.py file to add a test for the behavior.
The FasterMake backend didn't handle wildcards in file names present in
FINAL_TARGET_FILES properly. For an entry like:
FINAL_TARGET_FILES.foo += ['*.xyz']
It would wind up trying to install the files to `dist/bin/foo/*.xyz/`, a
path with a literal asterisk in it. The code seems to have been written
assuming that wildcards would only be present in directory components of
the path. This change fixes this specific case, although it's possible that
it still doesn't handle all permutations of wildcards properly.
MozReview-Commit-ID: rk2tSyDyIu
--HG--
extra : rebase_source : c8f0cfdaa6b957df70bb528af9b0b816b387840e
This commit adds new moz.build sandbox symbols that are intended to be used
for localized files: LOCALIZED_FILES and LOCALIZED_PP_FILES. They are currently
just do-nothing subclasses of FinalTarget[Preprocessor]Files, but the next
change in this series will add support for them to the recursive make backend.
Because they subclass FinalTarget[Preprocessor]Files, build backends that are
not concerned about localized builds should be able to handle them as if
they were FINAL_TARGET[_PP]_FILES without any additional code.
MozReview-Commit-ID: K0baBZ0F7av
--HG--
extra : rebase_source : 323e2993638fb0ba44ed89a4e0edd16b27a287e0
extra : source : e3ce81bc209b09b6771d7056d1fb06a65e27dc0d
This commit adds support for handling LOCALIZED_FILES and LOCALIZED_PP_FILES
in the recursive make backend. They get special handling because they have
a few special needs:
* They run during the libs tier, so that repacks work.
* The filenames cannot be determined at build-backend generation time,
since repacks run configure once and then repack multiple locales using
the generated backend files, so they are written with to be expanded with
MERGE_FILE by make so that the file gets picked up from the proper locale dir.
Other build backends that aren't trying to handle localized builds will
silently handle these like FINAL_TARGET_FILES, which is fine until we revamp
our l10n repack story.
MozReview-Commit-ID: 2LZhPZNhQ4S
--HG--
extra : rebase_source : dde0eb2e0ffe0bac5e5a6ab2c5f5724c06154101
extra : source : 5ee033a3c356bed86219699698abfe538370740a
In order to determine if we need to re-run configure, we write
a JSON file representing the evaluated mozconfig. If this JSON
file changes, configure (and config.status for that matter) is
out of data and it is re-executed.
This commit moves the generation of that JSON file to Python.
MozReview-Commit-ID: 636rpSY7gOm
--HG--
extra : rebase_source : ee1defd74decfd64ffb66a45b053dada58de04fb
This is a pretty straightforward port of the logic. But we
even go a step further: we delete the file in the objdir if there is
no source mozconfig!
MozReview-Commit-ID: AHFFzy6mXRY
--HG--
extra : rebase_source : 1b9387bd72f5a8e9bf8274f5764b0db0176fdba2
extra : source : 0cab9a382d817e6fbab9daa37db0f23e7f73e71f
The file is a filtered version of the make file that we previously
started generating for client.mk. Why there is special casing for
UPLOAD_EXTRA_FILES, I'm not sure. This smells fishy and is something
I'd like to take a look at once all code is ported out of client.mk.
The removal of the logic from client.mk meant that we could remove
a bunch of code from client.mk related to loading mozconfig files.
We can now simply include the auto-generated make file directly and
be done with it.
MozReview-Commit-ID: 4M5NElQA7iR
--HG--
extra : rebase_source : 87ed98fa62513007c6fdd2df00eb871a5a29a146
extra : source : 247617a64b7c438528f97d10c86e2f7b8cb72237
We write the file that client.mk is printing from Python. We can
also log it from there pretty easily. So do that.
MozReview-Commit-ID: 7eeugdOJR5b
--HG--
extra : rebase_source : 308826e948fa20684bbc40c806322f802689627e
Upcoming commits need to move more logic from client.mk. It will
be easier if we have a list of lines in the mozconfig as a local
variable.
MozReview-Commit-ID: 1wFZTfWLGP9
--HG--
extra : rebase_source : 5e3c408fdf7f953e9cbac1c4a57fd5fa87f8fbbc
