ccache/manual.txt

CCACHE(1)
=========


NAME
----

ccache - a fast C/C++ compiler cache


SYNOPSIS
--------

[verse]
*ccache* ['options']
*ccache* 'compiler' ['compiler options']
'compiler' ['compiler options']                   (via symbolic link)


DESCRIPTION
-----------

ccache is a compiler cache. It speeds up recompilation of C/C++ code by caching
the result of previous compilations and detecting when the same compilation is
being done again.

ccache has been carefully written to always produce exactly the same compiler
output that you would get without the cache. The only way you should be able to
tell that you are using ccache is the speed. Currently known exceptions to this
goal are listed under <<_bugs,BUGS>>. If you ever discover an undocumented case
where ccache changes the output of your compiler, please let us know.


FEATURES
~~~~~~~~

* Keeps statistics on hits/misses.
* Automatic cache size management.
* Can cache compilations that generate warnings.
* Easy installation.
* Low overhead.
* Optionally uses hard links where possible to avoid copies.
* Optionally compresses files in the cache to reduce disk space.


LIMITATIONS
~~~~~~~~~~~

* Only knows how to cache the compilation of a single C/C++ file. Other types
  of compilations (multi-file compilation, linking, etc) will silently fall
  back to running the real compiler.
* Only works with GCC and compilers that behave similar enough.
* Some compiler flags are not supported. If such a flag is detected, ccache
  will silently fall back to running the real compiler.


RUN MODES
---------

There are two ways to use ccache. You can either prefix your compilation
commands with *ccache* or you can create a symbolic link (named as your
compiler) to ccache. The first method is most convenient if you just want to
try out ccache or wish to use it for some specific projects. The second method
is most useful for when you wish to use ccache for all your compilations.

To install for usage by the first method just make sure *ccache* is in your path.

To install for the second method, do something like this:

    cp ccache /usr/local/bin/
    ln -s ccache /usr/local/bin/gcc
    ln -s ccache /usr/local/bin/g++
    ln -s ccache /usr/local/bin/cc
    ln -s ccache /usr/local/bin/c++

And so forth. This will work as long as +/usr/local/bin+ comes before the path
to the compiler (which is usually in +/usr/bin+). After installing you may wish
to run ``which gcc'' to make sure that the correct link is being used.

NOTE: Do not use a hard link, use a symbolic link. A hard link will cause
``interesting'' problems.


OPTIONS
-------

These options only apply when you invoke ccache as ``ccache''. When invoked as
a compiler (via a symlink as described in the previous section), none of these
options apply. In that case your normal compiler options apply and you should
refer to the compiler's documentation.

*-c, --cleanup*::

    Clean the cache and recalculate the cache file count and size totals.
    Normally the -c option should not be necessary as ccache keeps the cache
    below the specified limits at runtime and keeps statistics up to date on
    each compile. This option is mostly useful if you manually modify the cache
    contents or believe that the cache size statistics may be inaccurate.


*-C, --clear*::

    Clear the entire cache, removing all cached files.

*-F, --max-files*='N'::

    Set the maximum number of object files allowed in the cache. The value is
    stored inside the cache directory and applies to all future compilations.
    Due to the way the value is stored the actual value used is always rounded
    down to the nearest multiple of 16.

*-h, --help*::

    Print an options summary page.

*-M, --max-size*='SIZE'::

    Set the maximum size of the object files stored in the cache. You can
    specify a value in gigabytes, megabytes or kilobytes by appending a G, M or
    K to the value. The default is gigabytes. The actual value stored is
    rounded down to the nearest multiple of 16 kilobytes.

*-s, --show-stats*::

    Print the current statistics summary for the cache.

*-V, --version*::

    Print version and copyright information.

*-z, --zero-stats*::

    Zero the cache statistics (but not the configured limits).


EXTRA OPTIONS
-------------

When run as a compiler, ccache usually just takes the same command line options
as the compiler you are using. The only exception to this is the option
*--ccache-skip*. That option can be used to tell ccache that the next option is
definitely not a input filename, and should be passed along to the compiler
as-is.

The reason this can be important is that ccache does need to parse the command
line and determine what is an input filename and what is a compiler option, as
it needs the input filename to determine the name of the resulting object file
(among other things). The heuristic ccache uses when parsing the command line
is that any argument that exists as a file is treated as an input file name
(usually a C/C++ file). By using *--ccache-skip* you can force an option to not
be treated as an input file name and instead be passed along to the compiler as
a command line option.


ENVIRONMENT VARIABLES
---------------------

ccache uses a number of environment variables to control operation. In most
cases you won't need any of these as the defaults will be fine.

*CCACHE_BASEDIR*::

    If you set the environment variable *CCACHE_BASEDIR* to an absolute path,
    ccache rewrites absolute paths into relative paths before computing the
    hash that identifies the compilation, but only for paths under the
    specified directory. See the discussion under
    <<_compiling_in_different_directories,COMPILING IN DIFFERENT DIRECTORIES>>.

*CCACHE_CC*::

    You can optionally set *CCACHE_CC* to force the name of the compiler to
    use. If you don't do this then ccache works it out from the command line.

*CCACHE_COMPRESS*::

    If you set the environment variable *CCACHE_COMPRESS* then ccache will
    compress object files and other compiler output it puts in the cache.
    However, this setting has no effect on how files are retrieved from the
    cache; compressed and uncompressed results will still be usable regardless
    of this setting.

*CCACHE_COMPILERCHECK*::

    By default, ccache includes the modification time (mtime) and size of the
    compiler in the hash to ensure that results retrieved from the cache are
    accurate. The *CCACHE_COMPILERCHECK* environment variable can be used to
    select another strategy. Possible values are: *none* (don't hash anything),
    *mtime* (hash the compiler's mtime and size) and *content* (hash the
    content of the compiler binary). Setting the variable to *content* makes
    ccache very slightly slower, but makes it cope better with compiler
    upgrades during a build bootstrapping process. The *none* setting may be
    good for situations where you can safely use the cached results even though
    the compiler's mtime or size has changed (e.g. if the compiler is built as
    part of your build system and the compiler's source has not changed, or if
    the compiler only has changes that don't affect code generation). You
    should only set the variable to *none* if you know what you are doing.

*CCACHE_CPP2*::

    If you set the environment variable *CCACHE_CPP2* then ccache will not use
    the optimisation of avoiding the second call to the preprocessor by
    compiling the preprocessed output that was used for finding the hash in the
    case of a cache miss. This is primarily a debugging option, although it is
    possible that some unusual compilers will have problems with the
    intermediate filename extensions used in this optimisation, in which case
    this option could allow ccache to be used.

*CCACHE_DIR*::

    The *CCACHE_DIR* environment variable specifies where ccache will keep its
    cached compiler output. The default is *$HOME/.ccache*.

*CCACHE_DISABLE*::

    If you set the environment variable *CCACHE_DISABLE* then ccache will just
    call the real compiler, bypassing the cache completely.

*CCACHE_EXTENSION*::

    Normally ccache tries to automatically determine the extension to use for
    intermediate C preprocessor files based on the type of file being compiled.
    Unfortunately this sometimes doesn't work, for example when using the
    ``aCC'' compiler on HP-UX. On systems like this you can use the
    *CCACHE_EXTENSION* option to override the default. On HP-UX set this
    environment variable to *i* if you use the ``aCC'' compiler.

*CCACHE_EXTRAFILES*::

    If you set the environment variable *CCACHE_EXTRAFILES* to a
    space-separated list of paths then ccache will include the contents of
    those files when calculating the hash sum.

*CCACHE_HARDLINK*::

    If you set the environment variable *CCACHE_HARDLINK* then ccache will
    attempt to use hard links from the cache directory when creating the
    compiler output rather than using a file copy. Using hard links may be
    slightly faster in some situations, but can confuse programs like ``make''
    that rely on modification times. Hard links are never made for compressed
    cache files. This means that you should not set the *CCACHE_COMPRESS*
    variable if you want to use hard links.

*CCACHE_HASHDIR*::

    This tells ccache to hash the current working directory when calculating
    the hash that is used to distinguish two compiles. This prevents a problem
    with the storage of the current working directory in the debug info of a
    object file, which can lead ccache to give a cached object file that has
    the working directory in the debug info set incorrectly. This option is off
    by default as the incorrect setting of this debug info rarely causes
    problems. If you strike problems with GDB not using the correct directory
    then enable this option.

*CCACHE_LOGFILE*::

    If you set the *CCACHE_LOGFILE* environment variable then ccache will write
    some log information on cache hits and misses to that file. This is useful
    for tracking down problems.

*CCACHE_NLEVELS*::

    The environment variable *CCACHE_NLEVELS* allows you to choose the number
    of levels of hash in the cache directory. The default is 2. The minimum is
    1 and the maximum is 8.

*CCACHE_NODIRECT*::

    If you set the environment variable *CCACHE_NODIRECT* then ccache will not
    use the direct mode.

*CCACHE_NOSTATS*::

    If you set the environment variable *CCACHE_NOSTATS* then ccache will not
    update the statistics files on each compilation.

*CCACHE_PATH*::

    You can optionally set *CCACHE_PATH* to a colon-separated path where ccache
    will look for the real compilers. If you don't do this then ccache will
    look for the first executable matching the compiler name in the normal
    *PATH* that isn't a symbolic link to ccache itself.

*CCACHE_PREFIX*::

    This option adds a prefix to the command line that ccache runs when
    invoking the compiler. Also see the section below on using ccache with
    ``distcc''.

*CCACHE_READONLY*::

    The *CCACHE_READONLY* environment variable tells ccache to attempt to use
    existing cached object files, but not to try to add anything new to the
    cache. If you are using this because your *CCACHE_DIR* is read-only, then
    you may find that you also need to set *CCACHE_TEMPDIR* as otherwise ccache
    will fail to create temporary files.

*CCACHE_RECACHE*::

    This forces ccache to not use any cached results, even if it finds them.
    New results are still cached, but existing cache entries are ignored.

*CCACHE_TEMPDIR*::

    The *CCACHE_TEMPDIR* environment variable specifies where ccache will put
    temporary files. The default is *$CCACHE_DIR/tmp*. Note that the
    *CCACHE_TEMPDIR* path must be on the same filesystem as the *CCACHE_DIR*
    path, so that renames of files between the two directories can work.

*CCACHE_UMASK*::

    This sets the umask for ccache and all child processes (such as the
    compiler). This is mostly useful when you wish to share your cache with
    other users. Note that this also affects the file permissions set on the
    object files created from your compilations.

*CCACHE_UNIFY*::

    If you set the environment variable *CCACHE_UNIFY* then ccache will use a
    C/C++ unifier when hashing the preprocessor output if the *-g* option is
    not used. The unifier is slower than a normal hash, so setting this
    environment variable loses a little bit of speed, but it means that ccache
    can take advantage of not recompiling when the changes to the source code
    consist of reformatting only. Note that using *CCACHE_UNIFY* changes the
    hash, so cached compilations with *CCACHE_UNIFY* set cannot be used when
    *CCACHE_UNIFY* is not set and vice versa. The reason the unifier is off by
    default is that it can give incorrect line number information in compiler
    warning messages. Enabling the unifier implies turning off the direct mode.

*CCACHE_VERBOSE*::

    If you set the environment variable *CCACHE_VERBOSE*, ccache prints (to the
    standard output) the executed command lines when running the preprocessor
    and the compiler.


CACHE SIZE MANAGEMENT
---------------------

By default ccache has a one gigabyte limit on the total size of object files in
the cache and no maximum number of object files. You can set different limits
using the *-M*/*--max-size* and *-F*/*--max-files* options. Use *ccache -s* to
see the cache size and the currently configured limits (in addition to other
various statistics).


CACHE COMPRESSION
-----------------

ccache can optionally compress all files it puts into the cache using the
compression library zlib. While this involves a negligible performance
slowdown, it significantly increases the number of files that fit in the cache.
You can turn on compression by setting the *CCACHE_COMPRESS* environment
variable.


HOW IT WORKS
------------

The basic idea is to detect when you are compiling exactly the same code a
second time and reuse the previously produced output. The detection is done by
hashing different kinds of information that should be unique for the
compilation and then using the hash sum to identify the cached output. ccache
uses MD4, a very fast cryptographic hash algorithm, for the hashing. (MD4 is
nowadays too weak to be useful in cryptographic contexts, but it should be safe
enough to be used to identify recompilations.) On a cache hit, ccache is able
to supply all of the correct compiler outputs (including all warnings,
dependency file, etc) from the cache.

ccache has two ways of doing the detection:

* the *direct mode* (hashes the source code and include files directly)
* the *preprocessor mode* (hashes output from the preprocessor)


THE DIRECT MODE
~~~~~~~~~~~~~~~

In the direct mode, the hash is formed of:

* the input source file
* the command line options
* the real compiler's size and modification time (unless *CCACHE_COMPILERCHECK*
  says something else)

Based on the hash, a data structure called ``manifest'' is looked up in the
cache. The manifest contains:

* references to cached compilation results (object file, dependency file, etc)
  that were produced by previous compilations that matched the hash
* paths to the include files that were read at the time the compilation results
  were stored in the cache
* hash sums of the include files at the time the compilation results were
  stored in the cache

The current contents of the include files are then hashed and compared to the
information in the manifest. If there is a match, ccache knows the result of
the compilation. If there is no match, ccache falls back to running the
preprocessor. The output from the preprocessor is parsed to find the include
files that were read. The paths and hash sums of those include files are then
stored in the manifest along with information about the produced compilation
result.

The direct mode will be disabled if any of the following holds:

* the environment variable *CCACHE_NODIRECT* is set
* a modification time of one of the include files is too new (needed to avoid a
  race condition)
* the unifier is enabled (the environment variable *CCACHE_UNIFY* is set)
* a *-Wp,_X_* compiler option other than *-Wp,-MD,_path_* and *-Wp,-MMD,_path_*
  is used
* the string ``\_\_TIME__'' is present outside comments and string literals in
  the source code


THE PREPROCESSOR MODE
~~~~~~~~~~~~~~~~~~~~~

In the preprocessor mode, the hash is formed of:

* the preprocessor output from running the compiler with *-E*
* the command line options except options that affect include files (*-I*,
  *-include*, *-D*, etc; the theory is that these options will change the
  preprocessor output if they have any effect at all)
* the real compiler's size and modification time (unless
  *CCACHE_COMPILERCHECK* says something else)
* any standard error output generated by the preprocessor

Based on the hash, the cached compilation result can be looked up directly in
the cache.


COMPILING IN DIFFERENT DIRECTORIES
----------------------------------

Some information included in the hash that identifies a unique compilation may
contain absolute paths:

* The preprocessed source code may contain absolute paths to include files if
  the compiler option *-g* is used or if absolute paths are given to *-I* and
  similar compiler options.
* Paths specified by compiler options (such as *-I*, *-MF*, etc) may be
  absolute.
* The source code file path may be absolute, and that path may substituted for
  *\_\_FILE__* macros in the source code or included in warnings emitted to
  standard error by the preprocessor.

This means that if you compile the same code in different locations, you can't
share compilation results between the different build directories since you get
cache misses because of the absolute build directory paths that are part of the
hash. To mitigate this problem, you can specify a ``base directory'' by setting
the *CCACHE_BASEDIR* variable to an absolute path. ccache will then rewrite
absolute paths that are under the base directory (i.e., paths that have the
base directory as a prefix) to relative paths when constructing the hash. The
typical path to use as base directory is your home directory or another
directory that is a parent of your build directories. (Don't use +/+ as the
base directory since that will make ccache also rewrite paths to system header
files, which doesn't gain anything.)

The drawbacks of using *CCACHE_BASEDIR* are:

* If you specify an absolute path to the source code file, *\_\_FILE__* macros
  will be expanded to a relative path instead.
* If you specify an absolute path to the source code file and compile with
  *-g*, the source code path stored in the object file may point to the wrong
  directory, which may prevent debuggers like GDB from finding the source code.
  Sometimes, a work-around is to change the directory explicitly with the
  ``cd'' command in GDB.


SHARING A CACHE
---------------

A group of developers can increase the cache hit rate by sharing a cache
directory. To share a cache without unpleasant side effects, the following
conditions should to be met:

* Use the same *CCACHE_DIR* environment variable setting.
* Unset the *CCACHE_HARDLINK* environment variable.
* Make sure everyone sets the *CCACHE_UMASK* environment variable to 002. This
  ensure that cached files are accessible to everyone in the group.
* Make sure that all users have write permission in the entire cache directory
  (and that you trust all users of the shared cache).
* Make sure that the setgid bit is set on all directories in the cache. This
  tells the filesystem to inherit group ownership for new directories. The
  command ``find $CCACHE_DIR -type d | xargs chmod g+s'' might be useful for
  this.

The reason to avoid the hard link mode is that the hard links cause unwanted
side effects, as all links to a cached file share the file's modification
timestamp. This results in false dependencies to be triggered by
timestamp-based build systems whenever another user links to an existing file.
Typically, users will see that their libraries and binaries are relinked
without reason.

You may also want to make sure that the developers have *CCACHE_BASEDIR* set
appropriately, as discussed in the previous section.


USING CCACHE WITH DISTCC
------------------------

``distcc'' is a very useful program for distributing compilation across a range
of compiler servers. It is often useful to combine distcc with ccache so that
compilations that are done are sped up by distcc, but ccache avoids the
compilation completely where possible.

The recommended way of combining distcc and ccache is by using the
*CCACHE_PREFIX* option. You just need to set the environment variable
*CCACHE_PREFIX* to *distcc* and ccache will prefix the command line used with
the compiler with the command ``distcc''.


BUGS
----

ccache doesn't handle the GNU Assembler's *.incbin* directive correctly. This
directive can be embedded in the source code inside an *__asm__* statement in
order to include a file verbatim in the object file. If the included file is
changed, ccache doesn't pick up the change since the inclusion isn't done by
the preprocessor. A workaround of this problem is to set *CCACHE_EXTRAFILES* to
the path of the included file.


MORE INFORMATION
----------------

Credits, mailing list information, bug reporting instructions, source code,
etc, can be found on ccache's web site: <http://ccache.samba.org>.


AUTHOR
------

ccache was originally written by Andrew Tridgell and is currently maintained by
Joel Rosdahl. See <http://ccache.samba.org/credits.html> for a list of
contributors.