Commit Graph

687 Commits

Author SHA1 Message Date
Mike Shal
49fc6484f8 Bug 1507334 - Remove '10' argument to profileserver.py; r=firefox-build-system-reviewers,chmanchester#firefox-build-system-reviewers
Support for running profileserver.py multiple times was originally added
in bug 663251 and subsequently removed in bug 672563. Since then we've
been copying around the argument to profileserver.py from mozconfigs in
buildbot-configs, to mozconfigs in mozilla-central, to client.mk, and
ultimately to Makefile.in. It no longer does anything and can be
removed.

Differential Revision: https://phabricator.services.mozilla.com/D15748

--HG--
extra : moz-landing-system : lando
2019-01-10 01:34:39 +00:00
Mike Hommey
e489736f60 Bug 1518703 - Remove R_386_PC32 check. r=froydnj,ted
The test is actually redundant with the TEXTREL check in
check_binary.py (R_386_PC32 relocations will only happen as text
relocations)

Differential Revision: https://phabricator.services.mozilla.com/D16002

--HG--
extra : moz-landing-system : lando
2019-01-10 00:07:53 +00:00
Chris Manchester
b101c5a8f3 Bug 1517532 - Extend artifact builds to find and download artifacts from pgo builds when MOZ_PGO is set. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D15988

--HG--
extra : moz-landing-system : lando
2019-01-08 23:03:27 +00:00
Chris Manchester
25e14c4f13 Bug 1507338 - Add a configure option for MOZ_PROFILE_GENERATE. r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D12999

--HG--
extra : moz-landing-system : lando
2019-01-03 16:16:38 +00:00
Chris Manchester
d69003537d Bug 1498031 - Merge code paths for running configure between Tup and Make based backends. r=firefox-build-system-reviewers,mshal
This addresses a related issue along the way: a build that results in running
configure would not update the value of self.config_environment (and therefore
self.substs) as seen from the build driver, so out of date values would have
been used.

The changes to Makefile.in and client.mk made exploit the assumption that by
he time anything in the Make build is running, config.status is up to date.
Users running "make" without the benefit of "mach" will need to manually run
configure when necessary in order to take this into account.

Differential Revision: https://phabricator.services.mozilla.com/D8473

--HG--
extra : moz-landing-system : lando
2018-10-16 22:21:36 +00:00
Ted Mielczarek
e11506c714 bug 1490325 - write sccache log directly to artifact directory, make logs slightly more verbose. r=mshal
This patch makes it so we write the sccache log directly to the artifact
directory, so that it will be uploaded even if the build fails. It also makes
the log slightly more verbose. Both of these should help with diagnosing
sccache failures in CI.

The sccache log will no longer be explicitly gzip compressed, but some
Taskcluster client implementations will store it as gzip compressed.

Differential Revision: https://phabricator.services.mozilla.com/D6187

--HG--
extra : moz-landing-system : lando
2018-09-18 20:02:17 +00:00
Gabriele Svelto
b48b3e4db0 Bug 1473821 - The MOZ_PGO_INSTRUMENTED environment variable is not used anymore so remove all remaining mentions of it r=gps
Differential Revision: https://phabricator.services.mozilla.com/D3219

--HG--
extra : moz-landing-system : lando
2018-08-17 16:40:10 +00:00
Mike Hommey
f50e1c662a Bug 1483779 - Enable PGO on clang-cl 32-bits builds. r=dmajor
--HG--
extra : source : 5fcbe08fb321a5076f56c380b554c63068efbb00
2018-08-16 16:44:36 +09:00
arthur.iakab
07aa46fd13 Backed out 2 changesets (bug 1479800, bug 1483779)for frequent cgx and arm64 failures
Backed out changeset 5fcbe08fb321 (bug 1483779)
Backed out changeset dccd5299c5ad (bug 1479800)
2018-08-17 09:50:30 +03:00
Mike Hommey
152366b300 Bug 1483779 - Enable PGO on clang-cl 32-bits builds. r=dmajor 2018-08-17 10:19:52 +09:00
Mike Hommey
6d4eb0a92e Bug 1464170 - Support PGO in clang builds on Linux. r=dmajor
This uses the same code path as for clang-cl builds.
2018-08-10 05:54:11 +09:00
David Major
45083c4934 Bug 1341525: Enable PGO in 64-bit clang-cl builds. r=froydnj 2018-08-01 17:59:23 -04:00
Andreea Pavel
11f1c09ffe Backed out changeset 7a3882f27aa3 (bug 1341525) for failing win Sa builds on a CLOSED TREE
--HG--
extra : amend_source : dbf7c208749c67b6bb8674eb5ba55c6514f89a2d
2018-08-02 02:26:13 +03:00
David Major
ebe3264f11 Bug 1341525: Enable PGO in 64-bit clang-cl builds. r=froydnj 2018-08-01 17:59:23 -04:00
Andreea Pavel
250fd9a027 Backed out changeset 2011dae40293 (bug 1341525) for failing win Sa builds 2018-08-02 01:33:30 +03:00
David Major
622dccd28f Bug 1341525: Enable PGO in 64-bit clang-cl builds. r=froydnj
--HG--
extra : rebase_source : 06038cb081f7b2b2789a8b867b03340bf103b05e
2018-08-01 17:59:23 -04:00
Chris AtLee
38cced543a Bug 1466537: Disable full crashsymbols by branch r=ted
Differential Revision: https://phabricator.services.mozilla.com/D1558

--HG--
extra : moz-landing-system : lando
2018-07-30 20:23:14 +00:00
Aaron Klotz
9e27dd8849 Bug 1473423: Allow test symbols to be included in full symbol archive in try builds; r=ted
This patch moves the logic for include/exclude/compress from the makefile into
symbols_archive.py, and adds an additional check to the full-archive case to
allow the inclusion of test symbols in try builds.

--HG--
extra : amend_source : 306419aae90a1935ced49fec89112df05142501c
2018-07-11 17:38:37 -06:00
Nathan Froyd
5c5a455e97 Bug 1444171 - use the order file during re-link with clang-cl; r=glandium
Now that we've generated an order file of the first N functions invoked
during startup, let's tell the linker about said functions so it can
cluster them appropriately.
2018-07-09 18:36:14 -04:00
Nathan Froyd
e8a82fdba9 Bug 1444171: Perform an actual clobber for profiledbuild with clang-cl; r=gps
clang-cl-style PGO is more akin to what we have for GCC rather than
MSVC, so we should do a full clobber for maybe_clobber_profiledbuild.
2018-07-09 18:26:09 -04:00
Chris Manchester
843faaf73a Bug 1470552 - Update Makefile.in to refer to stub file target for buildid.h and source-repo.h generated in backend.mk r=froydnj
MozReview-Commit-ID: CxLkQjX8veW

--HG--
extra : rebase_source : 6db24d50fbb73a7a7ac2e0e771bc0fb9c73a443d
2018-06-22 15:10:39 -07:00
Ciure Andrei
3b1168a508 Backed out 1 changesets (bug 1470552) for build bustages mozbuild/test/backend/test_build.py CLOSED TREE
Backed out changeset 567df8598cdb (bug 1470552)
2018-06-23 00:16:35 +03:00
Chris Manchester
cc743e536a Bug 1470552 - Update Makefile.in to refer to stub file target for buildid.h and source-repo.h generated in backend.mk r=froydnj
MozReview-Commit-ID: JLgCA23U8zg

--HG--
extra : rebase_source : e0365b3b03d40de785838d8ca44148979b39f255
2018-06-22 13:29:00 -07:00
Chris Manchester
636b4fc4c0 Bug 1461836 - Write out complete configure dependencies from configure for consumption by make and non-make backends. r=mshal
MozReview-Commit-ID: 792seCZ2rs1

--HG--
extra : rebase_source : e6450cd2947e2fd55085a2af469535421bc07bfb
2018-05-21 14:01:50 -07:00
Nathan Froyd
5873523189 Bug 1461383 - fix test_build.py for local builds; r=chmanchester,f=mshal
test_build.py fails on local builds with messages like:

make[4]: Entering directory '/tmp/tmp65BjCH'
make[4]: *** No rule to make target 'buildid.h'.  Stop.
make[4]: Leaving directory '/tmp/tmp65BjCH'
/home/froydnj/src/gecko-dev.git/config/faster/rules.mk:76: recipe for target '/tmp/tmp65BjCH/buildid.h' failed
make[3]: *** [/tmp/tmp65BjCH/buildid.h] Error 2
make[3]: *** Waiting for unfinished jobs....
make[4]: Entering directory '/tmp/tmp65BjCH'
make[4]: *** No rule to make target 'source-repo.h'.  Stop.
make[4]: Leaving directory '/tmp/tmp65BjCH'
/home/froydnj/src/gecko-dev.git/config/faster/rules.mk:76: recipe for target '/tmp/tmp65BjCH/source-repo.h' failed
make[3]: *** [/tmp/tmp65BjCH/source-repo.h] Error 2
make[3]: Leaving directory '/tmp/tmp65BjCH/faster'
Makefile:155: recipe for target 'faster' failed

The tests pass in automation, however, because automation always defines
the rules for buildid.h and source-repo.h in the toplevel Makefile.in.
For local builds, however, those rules are not defined to avoid build
churn.  Let's ensure that the necessary rules are defined during testing
as well.
2018-05-15 10:05:23 -04:00
Nathan Froyd
33635829ac Bug 1459721 - part 7 - remove dist_idl install manifest; r=chmanchester
We no longer need to install the IDL files to the objdir for
processing. \o/
2018-05-15 10:05:23 -04:00
Chris Manchester
efa96c69f0 Bug 1454771 - Move tup invocation from Makefile.in to mach. r=mshal
MozReview-Commit-ID: HkhK4oe93Vm

--HG--
extra : rebase_source : cbfc0b05deae9ca5b871952bfb771320f7d51d9e
2018-04-25 14:50:18 -07:00
Chris Manchester
d5e62a783b Bug 1451530 - Pass the current topobjdir to our Tup invocation. r=mshal
MozReview-Commit-ID: D6UNvxwfGTa

--HG--
extra : rebase_source : b3bb9b559862a18ea058c15e2b668afea458752f
2018-04-04 15:13:13 -07:00
Mike Shal
e43c6624d2 Bug 1387098 - enable tup quiet mode; r=froydnj
Tup can have a lot of unnecessary log lines, particularly on a fresh
build when it is scanning directories for the first time. Turning on
--quiet when MOZ_AUTOMATION is set will help trim down the log size
significantly.

MozReview-Commit-ID: AaRppurkXsA

--HG--
extra : rebase_source : dca18a0e3657853eacfc6cdbb935998cadb339ce
2018-02-23 14:27:58 -05:00
Chris Manchester
fcf16fe1e7 Bug 1445398 - Do not re-generate buildid.h for every Tup build. r=mshal
MozReview-Commit-ID: ErkTDOU8lYH

--HG--
extra : rebase_source : 085b0cb43b625fbb2691aa6bc788b62549927a02
2018-03-22 16:03:12 -07:00
Chris Manchester
7f9cffd5ea Bug 1298328 - Fix dependency between buildid.h and libxul, do not re-generate buildid.h for every developer build. r=ted
For automation builds, force buildid.h to be regenerated, and always
#include buildid.h in nsAppRunner.cpp. This will eliminate the mismatch at the
cost of re-linking libxul on every build.

For developer builds, always #include buildid.h in nsAppRunner.cpp, but do not
force buildid.h to be re-generated for incremental builds. This means we will
never have a mismatch between platform and application buildids, although
subsequent builds are no longer guaranteed to have a new buildid.

MozReview-Commit-ID: 19hvu8AQXgN

--HG--
extra : rebase_source : 9f4fd063f89b9b2165d5889fdaa8f83104c4411c
2018-03-09 13:49:55 -08:00
Gurzau Raul
70976c79b0 Backed out changeset 2d981ccf4c2e (bug 1298328) for failing X tests on /experiments/test/xpcshell/test_conditions.js on a CLOSED TREE 2018-03-09 01:53:26 +02:00
Chris Manchester
506c15d221 Bug 1298328 - Fix dependency between buildid.h and libxul, do not re-generate buildid.h for every developer build. r=ted
For automation builds, force buildid.h to be regenerated, and always
#include buildid.h in nsAppRunner.cpp. This will eliminate the mismatch at the
cost of re-linking libxul on every build.

For developer builds, always #include buildid.h in nsAppRunner.cpp, but do not
force buildid.h to be re-generated for incremental builds. This means we will
never have a mismatch between platform and application buildids, although
subsequent builds are no longer guaranteed to have a new buildid.

MozReview-Commit-ID: 19hvu8AQXgN

--HG--
extra : rebase_source : 181b53c2cc47526bd632a0931ba6e65fc74179ef
2018-03-07 13:45:12 -08:00
Tom Prince
1d74db87ce Bug 1424651: Remove unused SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE mozconfig variable; r=ted.mielczarek
MozReview-Commit-ID: CkIg3fiwp1z

--HG--
extra : rebase_source : 5a4a50c8feb477a9b50c30e35b72a316b1f1bc8c
2017-12-10 23:05:05 -07:00
Dorel Luca
2f271a1136 Backed out 4 changesets (bug 1424651) as requested by tomprice r=backout on a CLOSED TREE
Backed out changeset 10ebf78f32bb (bug 1424651)
Backed out changeset 746d96792d18 (bug 1424651)
Backed out changeset 6038fb7b458c (bug 1424651)
Backed out changeset 189fd4f1df41 (bug 1424651)
2017-12-12 06:33:18 +02:00
Tom Prince
4dfc8f7a46 Bug 1424651: Remove unused SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE mozconfig variable; r=ted.mielczarek
MozReview-Commit-ID: CkIg3fiwp1z

--HG--
extra : rebase_source : 475e2d8888ff4b93efa9886581de9d145b51c51c
2017-12-10 23:05:05 -07:00
Gregory Szorc
7b5dac63f8 Bug 1412932 - Inline pgo-profile-run target; r=ted
It is a one-liner and doesn't need to exist in a separate make file.

MozReview-Commit-ID: 5w3eLWD0CuB

--HG--
extra : rebase_source : 15ebb63156f8b4e8d3288dfaf0d9ec2c328b5914
2017-11-07 15:06:45 -08:00
Gregory Szorc
9e26b9fde3 Bug 1412932 - Switch to PGO build in Makefile.in; r=ted
Previously, client.mk made the decision of whether to perform a PGO
build. This required passing around MOZ_PGO and invoking a separate
make target if this variable was set.

In this commit, we move this logic to Makefile.in. We employ a special
mechanism in rules.mk to override the default make target so
`make` evaluates "profiledbuild" if MOZ_PGO is set. This also
required using an explicit target for $(MAKE) invocations inside
the "profiledbuild" rule to avoid infinite recursion.

MozReview-Commit-ID: 8sHiVspMisM

--HG--
extra : rebase_source : 76c8a0c592015802f2b9ad52fe2001012a4611f6
extra : source : 24e28b37fbc0ad55ee449699e09c62042af41b55
2017-11-07 16:38:39 -08:00
Gregory Szorc
67864be6bd Bug 1412932 - Move PGO logic from client.mk into root Makefile.in; r=ted
client.mk is going away. This commit moves the PGO build target out of
it.

We move the code to Makefile.in. This likely isn't its final
home, as we'll want to eliminate Makefile.in at some point. However,
it is easier to move things to the root Makefile.in than other
alternatives. Keep in mind that the logic needs access to primitives
like BUILDSTATUS and variables from mk_add_options, which the
Makefile.in mechanism has easy access to.

MozReview-Commit-ID: 4C43lBhZz4l

--HG--
extra : rebase_source : 0ff1e95dcf8cb0082b019713db0cb7f47f0bccb8
extra : source : 0d2083b8edc43fc13c77f71199e0fcaddb32fcfb
2017-11-07 15:08:11 -08:00
Gregory Szorc
3221d82f84 Bug 1412356 - Update CLOBBER message to not recommend client.mk; r=chmanchester
client.mk hasn't been well-supported for years. Stop recommending it.

MozReview-Commit-ID: KWFhAQPIXWu

--HG--
extra : rebase_source : dafb307d82f1f0a3141d077102fd2fe21add65b4
2017-10-27 10:19:05 -07:00
Mike Hommey
9f2dd4775e Bug 1407468 - Replace multiple !JS_STANDALONE with MOZ_WIDGET_TOOLKIT. r=mshal
--HG--
extra : rebase_source : 4f9f9f583c3422ed0f8d1d65ea8e7575bd9baf2c
2017-10-05 14:50:01 +09:00
Mike Shal
5b41b430f9 Bug 1390916 - Always use --track instead of --no-remove for install manifests; r=gps
The --track flag provides a more accurate accounting of what files were
installed by the manifest, so they can be appropriately removed. For
example, test files are now removed from _tests if an entry in a test
file is deleted.

The --no-remove flag is removed as an alternative, and the --track flag
is now mandatory.

MozReview-Commit-ID: Wiup4Gzwkb

--HG--
extra : rebase_source : 4a44c7fe066ba9b5f1e37ec682464f7f4f6cb2cf
2017-08-08 17:07:07 -04:00
Mike Shal
642e6c18ad Bug 1390916 - Remove references to dist/sdk install manifest; r=gps
The SDK installation code was removed in bug 1333826.

MozReview-Commit-ID: 3h63nFSKOfi

--HG--
extra : rebase_source : 4a99f56e62c749e8991831f1a86cb4d98a58d31e
2017-08-09 15:35:15 -04:00
Mike Shal
83b4b8ddf1 Bug 1390916 - Make .PHONY rule match the actual make targets; r=gps
MozReview-Commit-ID: Gy4YrUtaB5H

--HG--
extra : rebase_source : baa1aa44becab40e1305105b4ab821043ceab9ae
2017-08-10 14:58:04 -04:00
Ted Mielczarek
8715819616 bug 1259832 - package generated sources and upload them along with other build artifacts. r=gps
This change makes us upload an `$(PKG_BASENAME).generated-files.tar.gz` archive
alongside other build artifacts which contains all the generated source files
from the build. A change after this will introduce an `upload-generated-sources`
task to take this artifact and upload the individual files to an S3 bucket.

This will be used to provide links to generated source files when they appear
in stack traces in crash reports.

MozReview-Commit-ID: 6yQAdlZ5q3O

--HG--
extra : rebase_source : d92fb17ae737d1360e9724997f6688e29bedef12
extra : source : 14d18d7cf454c4c3d0f6d49d1d01660e06e4be4b
2017-07-31 12:47:08 -04:00
Wes Kocher
749da2f3e6 Backed out 4 changesets (bug 1259832) for apparently scheduling nightlies on every push/platform a=backout
Backed out changeset 7781a37a4db0 (bug 1259832)
Backed out changeset 73bf88110b38 (bug 1259832)
Backed out changeset 14d18d7cf454 (bug 1259832)
Backed out changeset a02695cbf576 (bug 1259832)

MozReview-Commit-ID: 4Tf6jBcvKLq
2017-08-16 17:26:08 -07:00
Ted Mielczarek
3bf693fcb6 bug 1259832 - package generated sources and upload them along with other build artifacts. r=gps
This change makes us upload an `$(PKG_BASENAME).generated-files.tar.gz` archive
alongside other build artifacts which contains all the generated source files
from the build. A change after this will introduce an `upload-generated-sources`
task to take this artifact and upload the individual files to an S3 bucket.

This will be used to provide links to generated source files when they appear
in stack traces in crash reports.

MozReview-Commit-ID: 6yQAdlZ5q3O

--HG--
extra : rebase_source : 3f6ef734c062e0f5e9c2ca433ffad51fdf14b1ad
2017-07-31 12:47:08 -04:00
Mike Hommey
eaa93380ca Bug 1382502 - Enable sccache on PGO builds. r=ted
Note this will only enable it on try, autoland and mozilla-inbound,
which are the only branches where sccache are enabled at the moment.
Enabling on more builds (or not) is the subject of bug 1373460.

Also note that bug 1181040, that ensured PGO builds weren't using
sccache mentions that back then, link times went up when using sccache
(with -Z7) vs. without, but that was presumably with MSVC 2013. Try
suggests link times are the same using sccache now (still using -Z7,
pending bug 1318370).

--HG--
extra : rebase_source : 9f9d87714f3c841b790eb7d692ea0968b1755b70
2017-07-20 11:31:14 +09:00
Mike Hommey
2a24da6832 Bug 1380964 - Get and upload useful sccache logs. r=gps
When investigating why a particular build got a particularly high number
of cache misses, it is useful to know exactly *what* had cache misses,
and you can't get the missing information after the fact.

We used to have some stats stored in a log file uploaded as artifact
with the old python sccache per bug 1005495 (and before that inline in
the build log), and we actually still have remains of that in the form
of a EXTRA_UPLOAD_FILES in build/mozconfig.cache.

Let's reuse that, and get some useful logs from sccache. Ideally sccache
would give better logs, but those are sufficient for now. We'll figure
things out eventually. https://github.com/mozilla/sccache/issues/151

--HG--
extra : rebase_source : a2e7093697a5b720ab28f155560211d6c0a4347e
2017-07-14 09:46:38 +09:00
Tom Prince
c639c8aedc Bug 1379785 - Stop calling abspath on symbol_archive paths. r=gps
Thunderbird is still using MOZ_PKG_PRETTYNAMES which has spaces in the package
name, which gets mangled by abspath.

MozReview-Commit-ID: 23ngx6gIESI

--HG--
extra : rebase_source : 9038f295e582bec5872898a5e8f052bfacf8f85a
2017-07-10 15:25:04 -06:00