mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-12-15 03:00:30 +00:00
776ef07489
DONTBUILD (NPOTB)
193 lines
8.4 KiB
ReStructuredText
193 lines
8.4 KiB
ReStructuredText
.. _slow:
|
|
|
|
============================
|
|
Why the Build System is Slow
|
|
============================
|
|
|
|
A common complaint about the build system is that it's slow. There are
|
|
many reasons contributing to its slowness. We will attempt to document
|
|
them here.
|
|
|
|
First, it is important to distinguish between a :term:`clobber build`
|
|
and an :term:`incremental build`. The reasons for why each are slow can
|
|
be different.
|
|
|
|
The build does a lot of work
|
|
============================
|
|
|
|
It may not be obvious, but the main reason the build system is slow is
|
|
because it does a lot of work! The source tree consists of a few
|
|
thousand C++ files. On a modern machine, we spend over 120 minutes of CPU
|
|
core time compiling files! So, if you are looking for the root cause of
|
|
slow clobber builds, look at the sheer volume of C++ files in the tree.
|
|
|
|
You don't have enough CPU cores and MHz
|
|
=======================================
|
|
|
|
The build should be CPU bound. If the build system maintainers are
|
|
optimizing the build system perfectly, every CPU core in your machine
|
|
should be 100% saturated during a build. While this isn't currently the
|
|
case (keep reading below), generally speaking, the more CPU cores you
|
|
have in your machine and the more total MHz in your machine, the better.
|
|
|
|
**We highly recommend building with no fewer than 4 physical CPU
|
|
cores.** Please note the *physical* in this sentence. Hyperthreaded
|
|
cores (an Intel Core i7 will report 8 CPU cores but only 4 are physical
|
|
for example) only yield at most a 1.25x speedup per core.
|
|
|
|
We also recommend using the most modern CPU model possible. Haswell
|
|
chips deliver much more performance per CPU cycle than say Sandy Bridge
|
|
CPUs.
|
|
|
|
This cause impacts both clobber and incremental builds.
|
|
|
|
You are building with a slow I/O layer
|
|
======================================
|
|
|
|
The build system can be I/O bound if your I/O layer is slow. Linking
|
|
libxul on some platforms and build architectures can perform gigabytes
|
|
of I/O.
|
|
|
|
To minimize the impact of slow I/O on build performance, **we highly
|
|
recommend building with an SSD.** Power users with enough memory may opt
|
|
to build from a RAM disk. Mechanical disks should be avoided if at all
|
|
possible.
|
|
|
|
Some may dispute the importance of an SSD on build times. It is true
|
|
that the beneficial impact of an SSD can be mitigated if your system has
|
|
lots of memory and the build files stay in the page cache. However,
|
|
operating system memory management is complicated. You don't really have
|
|
control over what or when something is evicted from the page cache.
|
|
Therefore, unless your machine is a dedicated build machine or you have
|
|
more memory than is needed by everything running on your machine,
|
|
chances are you'll run into page cache eviction and you I/O layer will
|
|
impact build performance. That being said, an SSD certainly doesn't
|
|
hurt build times. And, anyone who has used a machine with an SSD will
|
|
tell you how great of an investment it is for performance all around the
|
|
operating system. On top of that, some automated tests are I/O bound
|
|
(like those touching SQLite databases), so an SSD will make tests
|
|
faster.
|
|
|
|
This cause impacts both clobber and incremental builds.
|
|
|
|
You don't have enough memory
|
|
============================
|
|
|
|
The build system allocates a lot of memory, especially when building
|
|
many things in parallel. If you don't have enough free system memory,
|
|
the build will cause swap activity, slowing down your system and the
|
|
build. Even if you never get to the point of swapping, the build system
|
|
performs a lot of I/O and having all accessed files in memory and the
|
|
page cache can significantly reduce the influence of the I/O layer on
|
|
the build system.
|
|
|
|
**We recommend building with no less than 8 GB of system memory.** As
|
|
always, the more memory you have, the better. For a bare bones machine
|
|
doing nothing more than building the source tree, anything more than 16
|
|
GB is likely entering the point of diminishing returns.
|
|
|
|
This cause impacts both clobber and incremental builds.
|
|
|
|
You are building with pymake
|
|
============================
|
|
|
|
Pymake is slower than GNU make. One reason is Python is generally slower
|
|
than C. The build system maintainers are consistently looking at
|
|
optimizing pymake. However, it is death by a thousand cuts.
|
|
|
|
This cause impacts both clobber and incremental builds.
|
|
|
|
You are building on Windows
|
|
===========================
|
|
|
|
Builds on Windows are slow for a few reasons. First, Windows builds use
|
|
pymake, not GNU make (because of compatibility issues with GNU make).
|
|
But, there are other sources of slowness.
|
|
|
|
New processes on Windows are about a magnitude slower to spawn than on
|
|
UNIX-y systems such as Linux. This is because Windows has optimized new
|
|
threads while the \*NIX platforms typically optimize new processes.
|
|
Anyway, the build system spawns thousands of new processes during a
|
|
build. Parts of the build that rely on rapid spawning of new processes
|
|
are slow on Windows as a result. This is most pronounced when running
|
|
*configure*. The configure file is a giant shell script and shell
|
|
scripts rely heavily on new processes. This is why configure on Windows
|
|
can run over a minute slower on Windows.
|
|
|
|
Another reason Windows builds are slower is because Windows lacks proper
|
|
symlink support. On systems that support symlinks, we can generate a
|
|
file into a staging area then symlink it into the final directory very
|
|
quickly. On Windows, we have to perform a full file copy. This incurs
|
|
much more I/O. And if done poorly, can muck with file modification
|
|
times, messing up build dependencies. As of the summer of 2013, the
|
|
impact of symlinks is being mitigated through the use
|
|
of an :term:`install manifest`.
|
|
|
|
These issues impact both clobber and incremental builds.
|
|
|
|
Recursive make traversal is slow
|
|
================================
|
|
|
|
The build system has traditionally been built by employing recursive
|
|
make. Recursive make involves make iterating through directories / make
|
|
files sequentially and executing each in turn. This is inefficient for
|
|
directories containing few targets/tasks because make could be *starved*
|
|
for work when processing these directories. Any time make is starved,
|
|
the build isn't using all available CPU cycles and the build is slower
|
|
as a result.
|
|
|
|
Work has started in bug 907365 to fix this issue by changing the way
|
|
make traverses all the make files.
|
|
|
|
The impact of slow recursive make traversal is mostly felt on
|
|
incremental builds. Traditionally, most of the wall time during a
|
|
no-op build is spent in make traversal.
|
|
|
|
make is inefficient
|
|
===================
|
|
|
|
Compared to modern build backends like Tup or Ninja, make is slow and
|
|
inefficient. We can only make make so fast. At some point, we'll hit a
|
|
performance plateau and will need to use a different tool to make builds
|
|
faster.
|
|
|
|
Please note that clobber and incremental builds are different. A clobber
|
|
build with make will likely be as fast as a clobber build with e.g. Tup.
|
|
However, Tup should vastly outperform make when it comes to incremental
|
|
builds. Therefore, this issue is mostly seen when performing incremental
|
|
builds.
|
|
|
|
C++ header dependency hell
|
|
==========================
|
|
|
|
Modifying a *.h* file can have significant impact on the build system.
|
|
If you modify a *.h* that is used by 1000 C++ files, all of those 1000
|
|
C++ files will be recompiled.
|
|
|
|
Our code base has traditionally been sloppy managing the impact of
|
|
changed headers on build performance. Bug 785103 tracks improving the
|
|
situation.
|
|
|
|
This issue mostly impacts the times of an :term:`incremental build`.
|
|
|
|
A search/indexing service on your machine is running
|
|
====================================================
|
|
|
|
Many operating systems have a background service that automatically
|
|
indexes filesystem content to make searching faster. On Windows, you
|
|
have the Windows Search Service. On OS X, you have Finder.
|
|
|
|
These background services sometimes take a keen interest in the files
|
|
being produced as part of the build. Since the build system produces
|
|
hundreds of megabytes or even a few gigabytes of file data, you can
|
|
imagine how much work this is to index! If this work is being performed
|
|
while the build is running, your build will be slower.
|
|
|
|
OS X's Finder is notorious for indexing when the build is running. And,
|
|
it has a tendency to suck up a whole CPU core. This can make builds
|
|
several minutes slower. If you build with ``mach`` and have the optional
|
|
``psutil`` package built (it requires Python development headers - see
|
|
:ref:`python` for more) and Finder is running during a build, mach will
|
|
print a warning at the end of the build, complete with instructions on
|
|
how to fix it.
|