ccache/manual.txt

CCACHE(1)
=========


NAME
----

ccache - a fast C/C++ compiler cache


SYNOPSIS
--------

[verse]
*ccache* ['options']
*ccache* 'compiler' ['compiler options']
'compiler' ['compiler options']                   (via symbolic link)


DESCRIPTION
-----------

ccache is a compiler cache. It speeds up recompilation of C/C++ code by caching
the result of previous compilations and detecting when the same compilation is
being done again.

ccache has been carefully written to always produce exactly the same compiler
output that you would get without the cache. The only way you should be able to
tell that you are using ccache is the speed. Currently known exceptions to this
goal are listed under <<_bugs,BUGS>>. If you ever discover an undocumented case
where ccache changes the output of your compiler, please let us know.


FEATURES
~~~~~~~~

* Keeps statistics on hits/misses.
* Automatic cache size management.
* Can cache compilations that generate warnings.
* Easy installation.
* Low overhead.
* Optionally uses hard links where possible to avoid copies.
* Optionally compresses files in the cache to reduce disk space.


LIMITATIONS
~~~~~~~~~~~

* Only knows how to cache the compilation of a single C/C++ file. Other types
  of compilations (multi-file compilation, linking, etc) will silently fall
  back to running the real compiler.
* Only works with GCC and compilers that behave similar enough.
* Some compiler flags are not supported. If such a flag is detected, ccache
  will silently fall back to running the real compiler.


RUN MODES
---------

There are two ways to use ccache. You can either prefix your compilation
commands with *ccache* or you can create a symbolic link (named as your
compiler) to ccache. The first method is most convenient if you just want to
try out ccache or wish to use it for some specific projects. The second method
is most useful for when you wish to use ccache for all your compilations.

To install for usage by the first method just make sure *ccache* is in your path.

To install for the second method, do something like this:

    cp ccache /usr/local/bin/
    ln -s ccache /usr/local/bin/gcc
    ln -s ccache /usr/local/bin/g++
    ln -s ccache /usr/local/bin/cc
    ln -s ccache /usr/local/bin/c++

And so forth. This will work as long as +/usr/local/bin+ comes before the path
to the compiler (which is usually in +/usr/bin+). After installing you may wish
to run ``which gcc'' to make sure that the correct link is being used.

NOTE: Do not use a hard link, use a symbolic link. A hard link will cause
``interesting'' problems.


OPTIONS
-------

These options only apply when you invoke ccache as ``ccache''. When invoked as
a compiler (via a symlink as described in the previous section), none of these
options apply. In that case your normal compiler options apply and you should
refer to the compiler's documentation.

*-c, --cleanup*::

    Clean the cache and recalculate the cache file count and size totals.
    Normally the -c option should not be necessary as ccache keeps the cache
    below the specified limits at runtime and keeps statistics up to date on
    each compile. This option is mostly useful if you manually modify the cache
    contents or believe that the cache size statistics may be inaccurate.


*-C, --clear*::

    Clear the entire cache, removing all cached files.

*-F, --max-files*='N'::

    Set the maximum number of object files allowed in the cache. The value is
    stored inside the cache directory and applies to all future compilations.
    Due to the way the value is stored the actual value used is always rounded
    down to the nearest multiple of 16.

*-h, --help*::

    Print an options summary page.

*-M, --max-size*='SIZE'::

    Set the maximum size of the object files stored in the cache. You can
    specify a value in gigabytes, megabytes or kilobytes by appending a G, M or
    K to the value. The default is gigabytes. The actual value stored is
    rounded down to the nearest multiple of 16 kilobytes.

*-s, --show-stats*::

    Print the current statistics summary for the cache.

*-V, --version*::

    Print version and copyright information.

*-z, --zero-stats*::

    Zero the cache statistics (but not the configured limits).


EXTRA OPTIONS
-------------

When run as a compiler, ccache usually just takes the same command line options
as the compiler you are using. The only exception to this is the option
*--ccache-skip*. That option can be used to tell ccache that the next option is
definitely not a input filename, and should be passed along to the compiler
as-is.

The reason this can be important is that ccache does need to parse the command
line and determine what is an input filename and what is a compiler option, as
it needs the input filename to determine the name of the resulting object file
(among other things). The heuristic ccache uses when parsing the command line
is that any argument that exists as a file is treated as an input file name
(usually a C/C++ file). By using *--ccache-skip* you can force an option to not
be treated as an input file name and instead be passed along to the compiler as
a command line option.


ENVIRONMENT VARIABLES
---------------------

ccache uses a number of environment variables to control operation. In most
cases you won't need any of these as the defaults will be fine.

*CCACHE_BASEDIR*::

    If you set the environment variable *CCACHE_BASEDIR* to an absolute path to
    a directory, ccache rewrites absolute paths into relative paths before
    computing the hash that identifies the compilation, but only for paths
    under the specified directory. See the discussion under
    <<_compiling_in_different_directories,COMPILING IN DIFFERENT DIRECTORIES>>.

*CCACHE_CC*::

    You can optionally set *CCACHE_CC* to force the name of the compiler to
    use. If you don't do this then ccache works it out from the command line.

*CCACHE_COMPRESS*::

    If you set the environment variable *CCACHE_COMPRESS* then ccache will
    compress object files and other compiler output it puts in the cache.
    However, this setting has no effect on how files are retrieved from the
    cache; compressed and uncompressed results will still be usable regardless
    of this setting.

*CCACHE_COMPILERCHECK*::

    By default, ccache includes the modification time (mtime) and size of the
    compiler in the hash to ensure that results retrieved from the cache are
    accurate. The *CCACHE_COMPILERCHECK* environment variable can be used to
    select another strategy. Possible values are: *none* (don't hash anything),
    *mtime* (hash the compiler's mtime and size) and *content* (hash the
    content of the compiler binary). Setting the variable to *content* makes
    ccache very slightly slower, but makes it cope better with compiler
    upgrades during a build bootstrapping process. The *none* setting may be
    good for situations where you can safely use the cached results even though
    the compiler's mtime or size has changed (e.g. if the compiler is built as
    part of your build system and the compiler's source has not changed, or if
    the compiler only has changes that don't affect code generation). You
    should only set the variable to *none* if you know what you are doing.

*CCACHE_CPP2*::

    If you set the environment variable *CCACHE_CPP2* then ccache will not use
    the optimisation of avoiding the second call to the preprocessor by
    compiling the preprocessed output that was used for finding the hash in the
    case of a cache miss. This is primarily a debugging option, although it is
    possible that some unusual compilers will have problems with the
    intermediate filename extensions used in this optimisation, in which case
    this option could allow ccache to be used.

*CCACHE_DIR*::

    The *CCACHE_DIR* environment variable specifies where ccache will keep its
    cached compiler output. The default is *$HOME/.ccache*.

*CCACHE_DISABLE*::

    If you set the environment variable *CCACHE_DISABLE* then ccache will just
    call the real compiler, bypassing the cache completely.

*CCACHE_EXTENSION*::

    Normally ccache tries to automatically determine the extension to use for
    intermediate C preprocessor files based on the type of file being compiled.
    Unfortunately this sometimes doesn't work, for example when using the
    ``aCC'' compiler on HP-UX. On systems like this you can use the
    *CCACHE_EXTENSION* option to override the default. On HP-UX set this
    environment variable to *i* if you use the ``aCC'' compiler.

*CCACHE_EXTRAFILES*::

    If you set the environment variable *CCACHE_EXTRAFILES* to a
    colon-separated list of paths then ccache will include the contents of
    those files when calculating the hash sum.

*CCACHE_HARDLINK*::

    If you set the environment variable *CCACHE_HARDLINK* then ccache will
    attempt to use hard links from the cache directory when creating the
    compiler output rather than using a file copy. Using hard links may be
    slightly faster in some situations, but can confuse programs like ``make''
    that rely on modification times. Hard links are never made for compressed
    cache files. This means that you should not set the *CCACHE_COMPRESS*
    variable if you want to use hard links.

*CCACHE_HASHDIR*::

    This tells ccache to hash the current working directory when calculating
    the hash that is used to distinguish two compiles. This prevents a problem
    with the storage of the current working directory in the debug info of a
    object file, which can lead ccache to give a cached object file that has
    the working directory in the debug info set incorrectly. This option is off
    by default as the incorrect setting of this debug info rarely causes
    problems. If you strike problems with GDB not using the correct directory
    then enable this option.

*CCACHE_LOGFILE*::

    If you set the *CCACHE_LOGFILE* environment variable then ccache will write
    some information on what it is doing to the log. This is useful for
    tracking down problems.

*CCACHE_NLEVELS*::

    The environment variable *CCACHE_NLEVELS* allows you to choose the number
    of levels of hash in the cache directory. The default is 2. The minimum is
    1 and the maximum is 8.

*CCACHE_NODIRECT*::

    If you set the environment variable *CCACHE_NODIRECT* then ccache will not
    use the direct mode.

*CCACHE_NOSTATS*::

    If you set the environment variable *CCACHE_NOSTATS* then ccache will not
    update the statistics files on each compilation.

*CCACHE_PATH*::

    You can optionally set *CCACHE_PATH* to a colon-separated path where ccache
    will look for the real compilers. If you don't do this then ccache will
    look for the first executable matching the compiler name in the normal
    *PATH* that isn't a symbolic link to ccache itself.

*CCACHE_PREFIX*::

    This option adds a prefix to the command line that ccache runs when
    invoking the compiler. Also see the section below on using ccache with
    ``distcc''.

*CCACHE_READONLY*::

    The *CCACHE_READONLY* environment variable tells ccache to attempt to use
    existing cached object files, but not to try to add anything new to the
    cache. If you are using this because your *CCACHE_DIR* is read-only, then
    you may find that you also need to set *CCACHE_TEMPDIR* as otherwise ccache
    will fail to create temporary files.

*CCACHE_RECACHE*::

    This forces ccache to not use any cached results, even if it finds them.
    New results are still cached, but existing cache entries are ignored.

*CCACHE_SLOPPINESS*::

    By default, ccache tries to give as few false cache hits as possible.
    However, in certain situations it's possible that you know things that
    ccache can't take for granted. The *CCACHE_SLOPPINESS* environment variable
    makes it possible to tell ccache to relax some checks in order to increase
    the hit rate. The value should be a comma-separated string with options.
    Available options are: *file_macro* (ignore *\_\_FILE\__* being present in
    the source), *include_file_mtime* (don't check the modification time of
    include files in direct mode) and *time_macros* (ignore *\_\_DATE\__* and
    *\_\_TIME__* being present in the source code). See the discussion under
    <<_troubleshooting,TROUBLESHOOTING>> for more information.

*CCACHE_TEMPDIR*::

    The *CCACHE_TEMPDIR* environment variable specifies where ccache will put
    temporary files. The default is *$CCACHE_DIR/tmp*. (Note: In previous
    versions of ccache, *CCACHE_TEMPDIR* had to be on the same filesystem as
    the *CCACHE_DIR* path, but this requirement has been relaxed.)

*CCACHE_UMASK*::

    This sets the umask for ccache and all child processes (such as the
    compiler). This is mostly useful when you wish to share your cache with
    other users. Note that this also affects the file permissions set on the
    object files created from your compilations.

*CCACHE_UNIFY*::

    If you set the environment variable *CCACHE_UNIFY* then ccache will use a
    C/C++ unifier when hashing the preprocessor output if the *-g* option is
    not used. The unifier is slower than a normal hash, so setting this
    environment variable loses a little bit of speed, but it means that ccache
    can take advantage of not recompiling when the changes to the source code
    consist of reformatting only. Note that using *CCACHE_UNIFY* changes the
    hash, so cached compilations with *CCACHE_UNIFY* set cannot be used when
    *CCACHE_UNIFY* is not set and vice versa. The reason the unifier is off by
    default is that it can give incorrect line number information in compiler
    warning messages. Enabling the unifier implies turning off the direct mode.

*CCACHE_VERBOSE*::

    If you set the environment variable *CCACHE_VERBOSE*, ccache prints (to the
    standard output) the executed command lines when running the preprocessor
    and the compiler.


CACHE SIZE MANAGEMENT
---------------------

By default ccache has a one gigabyte limit on the total size of object files in
the cache and no maximum number of object files. You can set different limits
using the *-M*/*--max-size* and *-F*/*--max-files* options. Use *ccache -s* to
see the cache size and the currently configured limits (in addition to other
various statistics).


CACHE COMPRESSION
-----------------

ccache can optionally compress all files it puts into the cache using the
compression library zlib. While this involves a negligible performance
slowdown, it significantly increases the number of files that fit in the cache.
You can turn on compression by setting the *CCACHE_COMPRESS* environment
variable.


HOW IT WORKS
------------

The basic idea is to detect when you are compiling exactly the same code a
second time and reuse the previously produced output. The detection is done by
hashing different kinds of information that should be unique for the
compilation and then using the hash sum to identify the cached output. ccache
uses MD4, a very fast cryptographic hash algorithm, for the hashing. (MD4 is
nowadays too weak to be useful in cryptographic contexts, but it should be safe
enough to be used to identify recompilations.) On a cache hit, ccache is able
to supply all of the correct compiler outputs (including all warnings,
dependency file, etc) from the cache.

ccache has two ways of doing the detection:

* the *direct mode* (hashes the source code and include files directly)
* the *preprocessor mode* (hashes output from the preprocessor)


THE DIRECT MODE
~~~~~~~~~~~~~~~

In the direct mode, the hash is formed of:

* the input source file
* the command line options
* the real compiler's size and modification time (unless *CCACHE_COMPILERCHECK*
  says something else)

Based on the hash, a data structure called ``manifest'' is looked up in the
cache. The manifest contains:

* references to cached compilation results (object file, dependency file, etc)
  that were produced by previous compilations that matched the hash
* paths to the include files that were read at the time the compilation results
  were stored in the cache
* hash sums of the include files at the time the compilation results were
  stored in the cache

The current contents of the include files are then hashed and compared to the
information in the manifest. If there is a match, ccache knows the result of
the compilation. If there is no match, ccache falls back to running the
preprocessor. The output from the preprocessor is parsed to find the include
files that were read. The paths and hash sums of those include files are then
stored in the manifest along with information about the produced compilation
result.

The direct mode will be disabled if any of the following holds:

* the environment variable *CCACHE_NODIRECT* is set
* a modification time of one of the include files is too new (needed to avoid a
  race condition)
* the unifier is enabled (the environment variable *CCACHE_UNIFY* is set)
* a compiler option not supported by the direct mode is used:
** a *-Wp,_X_* compiler option other than *-Wp,-MD,_path_* and
   *-Wp,-MMD,_path_*
** *-Xpreprocessor*
* the string ``\_\_TIME__'' is present outside comments and string literals in
  the source code


THE PREPROCESSOR MODE
~~~~~~~~~~~~~~~~~~~~~

In the preprocessor mode, the hash is formed of:

* the preprocessor output from running the compiler with *-E*
* the command line options except options that affect include files (*-I*,
  *-include*, *-D*, etc; the theory is that these options will change the
  preprocessor output if they have any effect at all)
* the real compiler's size and modification time (unless CCACHE_COMPILERCHECK*
  *says something else)
* any standard error output generated by the preprocessor

Based on the hash, the cached compilation result can be looked up directly in
the cache.


COMPILING IN DIFFERENT DIRECTORIES
----------------------------------

Some information included in the hash that identifies a unique compilation may
contain absolute paths:

* The preprocessed source code may contain absolute paths to include files if
  the compiler option *-g* is used or if absolute paths are given to *-I* and
  similar compiler options.
* Paths specified by compiler options (such as *-I*, *-MF*, etc) may be
  absolute.
* The source code file path may be absolute, and that path may substituted for
  *\_\_FILE__* macros in the source code or included in warnings emitted to
  standard error by the preprocessor.

This means that if you compile the same code in different locations, you can't
share compilation results between the different build directories since you get
cache misses because of the absolute build directory paths that are part of the
hash. To mitigate this problem, you can specify a ``base directory'' by setting
the *CCACHE_BASEDIR* variable to an absolute path to the directory. ccache will
then rewrite absolute paths that are under the base directory (i.e., paths that
have the base directory as a prefix) to relative paths when constructing the
hash. A typical path to use as the base directory is your home directory or
another directory that is a parent of your build directories. (Don't use +/+ as
the base directory since that will make ccache also rewrite paths to system
header files, which doesn't gain anything.)

The drawbacks of using *CCACHE_BASEDIR* are:

* If you specify an absolute path to the source code file, *\_\_FILE__* macros
  will be expanded to a relative path instead.
* If you specify an absolute path to the source code file and compile with
  *-g*, the source code path stored in the object file may point to the wrong
  directory, which may prevent debuggers like GDB from finding the source code.
  Sometimes, a work-around is to change the directory explicitly with the
  ``cd'' command in GDB.


SHARING A CACHE
---------------

A group of developers can increase the cache hit rate by sharing a cache
directory. To share a cache without unpleasant side effects, the following
conditions should to be met:

* Use the same *CCACHE_DIR* environment variable setting.
* Unset the *CCACHE_HARDLINK* environment variable.
* Make sure everyone sets the *CCACHE_UMASK* environment variable to 002. This
  ensure that cached files are accessible to everyone in the group.
* Make sure that all users have write permission in the entire cache directory
  (and that you trust all users of the shared cache).
* Make sure that the setgid bit is set on all directories in the cache. This
  tells the filesystem to inherit group ownership for new directories. The
  command ``find $CCACHE_DIR -type d | xargs chmod g+s'' might be useful for
  this.

The reason to avoid the hard link mode is that the hard links cause unwanted
side effects, as all links to a cached file share the file's modification
timestamp. This results in false dependencies to be triggered by
timestamp-based build systems whenever another user links to an existing file.
Typically, users will see that their libraries and binaries are relinked
without reason.

You may also want to make sure that the developers have *CCACHE_BASEDIR* set
appropriately, as discussed in the previous section.


SHARING A CACHE ON NFS
----------------------

It is possible to put the cache directory on an NFS filesystem (or similar
filesystems), but keep in mind that:

* Having the cache on NFS may slow down compilation. Make sure to do some
  benchmarking to see if it's worth it.
* ccache hasn't been tested very thoroughly on NFS.

A tip is to set CCACHE_TEMPDIR to a directory on the local host to avoid NFS
traffic for temporary files.


USING CCACHE WITH DISTCC
------------------------

``distcc'' is a very useful program for distributing compilation across a range
of compiler servers. It is often useful to combine distcc with ccache so that
compilations that are done are sped up by distcc, but ccache avoids the
compilation completely where possible.

The recommended way of combining distcc and ccache is by using the
*CCACHE_PREFIX* option. You just need to set the environment variable
*CCACHE_PREFIX* to *distcc* and ccache will prefix the command line used with
the compiler with the command ``distcc''.


BUGS
----

ccache doesn't handle the GNU Assembler's *.incbin* directive correctly. This
directive can be embedded in the source code inside an *__asm__* statement in
order to include a file verbatim in the object file. If the included file is
changed, ccache doesn't pick up the change since the inclusion isn't done by
the preprocessor. A workaround of this problem is to set *CCACHE_EXTRAFILES* to
the path of the included file.



TROUBLESHOOTING
---------------

GENERAL
~~~~~~~

A general tip for getting information about what ccache is doing is to enable
debug logging by setting *CCACHE_LOGFILE*. The log contains executed commands,
important decisions that ccache makes, read and written files, etc. Another way
of keeping track of what is happening is to check the output of *ccache -s*.


PERFORMANCE
~~~~~~~~~~~

ccache has been written to perform well out of the box, but sometimes you may
have to do some adjustments of how you use the compiler and ccache in order to
improve performance.

Since ccache works best when I/O is fast, put the cache directory on a fast
storage device if possible. Having lots of free memory so that files in the
cache directory stay in the disk cache is also a good idea.

A good way of monitoring how well ccache works is to run *ccache -s* before and
after your build and then compare the statistics counters. Here are some common
problems and what may be done to increase the hit rate:

* If ``cache hit (preprocessed)'' has been incremented instead of ``cache hit
  (direct)'', ccache has fallen back to preprocessor mode, which is generally
  slower. Some possible reasons are:
** The compiler option *-Xpreprocessor* or *-Wp,_X_* (except *-Wp,-MD,_path_*
   and *Wp,-MMD,_path_*) is used.
** This was the first compilation with a new value of *CCACHE_BASEDIR*.
** A modification time of one of the include files is too new (created the same
   second as the compilation is being done). This check is made to avoid a race
   condition. To fix this, create the include file earlier in the build
   process, if possible, or set *CCACHE_SLOPPINESS* to *include_file_mtime* if
   you are willing to take the risk. (The race condition consists of these
   events: the preprocessor is run; an include file is modified by someone; the
   new include file is hashed by ccache; the real compiler is run on the
   preprocessor's output, which contains data from the old header file; the
   wrong object file is stored in the cache.)
** The *\_\_TIME\__* preprocessor macro is (potentially) being used. ccache
   turns off direct mode if ``\_\_TIME\__'' is present in the source code
   outside comments and string literals. This is done as a safety measure since
   the string indicates that a *\_\_TIME\__* macro _may_ affect the output. (To
   be sure, ccache would have to run the preprocessor, but the sole point of
   the direct mode is to avoid that.) If you know that *\_\_TIME\__* isn't used
   in practise, or don't care if ccache produces objects where *\_\_TIME__* is
   expanded to something in the past, you can set *CCACHE_SLOPPINESS* to
   *time_macros*.
** The *\_\_DATE\__* preprocessor macro is (potentially) being used and the
   date has changed. This is similar to how *\_\_TIME\__* is handled. If
   ``\_\_DATE\__'' is present in the source code outside comments and string
   literals, ccache hashes the current date in order to be able to produce the
   correct object file if the *\_\_DATE\__* macro affects the output. If you
   know that *\_\_DATE\__* isn't used in practise, or don't care if ccache
   produces objects where *\_\_DATE__* is expanded to something in the past,
   you can set *CCACHE_SLOPPINESS* to *time_macros*.
** The *\_\_FILE\__* preprocessor macro is (potentially) being used and the
   file path has changed. If ``\_\_FILE\__'' is present in the source code
   outside comments and string literals, ccache hashes the current input file
   path in order to be able to produce the correct object file if the
   *\_\_FILE\__* macro affects the output. If you know that *\_\_FILE\__* isn't
   used in practise, or don't care if ccache produces objects where
   *\_\_FILE__* is expanded to the wrong path, you can set *CCACHE_SLOPPINESS*
   to *file_macro*.
* If ``cache miss'' has been incremented even though this wasn't the first
  compilation of the same source code, ccache has either detected that
  something has changed anyway or a cleanup has been performed (either
  explicitly or implicitly when a cache limit has been reached). Some perhaps
  unobvious things that may result in a cache miss are usage of *\_\_TIME\__*
  or *\_\_DATE__* macros, or use of automatically generated code that contains
  a timestamp, build counter or other volatile information.
* If ``multiple source files'' has been incremented, it's an indication that
  the compiler has been invoked on several source code files at once. ccache
  doesn't support that. Compile the source code files separately if possible.
* If ``preprocessor error'' has been incremented, one possible reason is that
  precompiled headers are being used. ccache currently doesn't handle that.
* If ``unsupported compiler option'' has been incremented, enable debug logging
  and check which option was rejected.


ERRORS WHEN COMPILING WITH CCACHE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If compilation doesn't work with ccache, but it works without it, one possible
reason is that the compiler can't compile the output from the preprocessor
correctly. A workaround that may work is to set *CCACHE_CPP2*. This will make
cache misses slower, though, so it is better to find and fix the root cause.


CORRUPT OBJECT FILES
~~~~~~~~~~~~~~~~~~~~

It should be noted that ccache is susceptible to general storage problems. If a
bad object file sneaks into the cache for some reason, it will of course stay
bad. Some possible reasons for erroneous object files are bad hardware (disk
drive, disk controller, memory, etc), buggy drivers or file systems, a bad
*CCACHE_PREFIX* command or compiler wrapper. If this happens, you can either
find out which object file is broken by reading the debug log and then delete
the bad object file from the cache, or you can simply clear the whole cache
with *ccache -C* if you don't mind losing other cached results.

There are no reported issues about ccache producing broken object files
reproducibly. That doesn't mean it can't happen, so if you find a repeatable
case, please report it.


MORE INFORMATION
----------------

Credits, mailing list information, bug reporting instructions, source code,
etc, can be found on ccache's web site: <http://ccache.samba.org>.


AUTHOR
------

ccache was originally written by Andrew Tridgell and is currently maintained by
Joel Rosdahl. See <http://ccache.samba.org/credits.html> for a list of
contributors.