ccache - a fast compiler cache
Go to file
2002-03-30 11:43:26 +01:00
args.c added args_pop() for neater args processing 2002-03-27 09:40:13 +01:00
ccache_clean.c handle unlink races better 2002-03-28 06:49:31 +01:00
ccache.c added built-in stats and cache management 2002-03-30 11:43:26 +01:00
ccache.h added built-in stats and cache management 2002-03-30 11:43:26 +01:00
cleanup.c added built-in stats and cache management 2002-03-30 11:43:26 +01:00
COPYING added some docs and comments ... 2002-03-27 00:58:31 +01:00
execute.c make the compile as atomic as possible 2002-03-27 01:39:06 +01:00
hash.c fixed handling of non-64 multiple md4 buffers 2002-03-28 02:55:55 +01:00
Makefile added built-in stats and cache management 2002-03-30 11:43:26 +01:00
mdfour.c added some docs and comments ... 2002-03-27 00:58:31 +01:00
mdfour.h first version of C compilercache 2002-03-26 15:46:43 +01:00
README added built-in stats and cache management 2002-03-30 11:43:26 +01:00
stats.c added built-in stats and cache management 2002-03-30 11:43:26 +01:00
util.c added built-in stats and cache management 2002-03-30 11:43:26 +01:00

This is a re-implementation of "compilercache" in C

The original compilercache scripts were by Erik Thiele
(erikyyy@erikyyy.de) and I would like to thank him for an excellent
piece of work. See http://www.erikyyy.de/compilercache/ for the
original shell scripts.

I wrote ccache because I wanted to get a bit more speed out of a
compiler cache and I wanted to remove some of the limitations of the
shell-script version.

Installation
------------

There are two ways to use ccache. You can either prefix your compile
commands with "ccache" or you can create a symbolic link between
ccache and the names of your compilers. The first method is most
convenient if you just want to try out ccache or wish to use it for
some specific projects. The second method is most useful for when you
wish to use ccache for all your compiles.

To install for usage by the first method just copy ccache to somewhere
in your path. 

To install for the second method do something like this:

  cp ccache /usr/local/bin/
  ln -s /usr/local/bin/ccache /usr/local/bin/gcc
  ln -s /usr/local/bin/ccache /usr/local/bin/cc

This will work as long as /usr/local/bin comes before the path to gcc
(which is usually in /usr/bin). After installing you may wish to run
"which gcc" to make sure that the correct link is being used.

Setting cache limits
--------------------

Run "ccache -h" to see a list of options. The main ones you may wish
to look at are "ccache -M" and "ccache -F" for setting the cache size
limits.

You can use "ccache -s" to look at the cache hit/miss statistics.

Configuration
-------------

Configuration of ccache is done via a number of environment variables
and via ccache commands. In most cases you won't need any of these as
the defaults will be fine.

   CCACHE_DIR

   the CCACHE_DIR environment variable specifies where ccache will
   keep its cached compiler output. The default is "$HOME/.ccache". 


   CCACHE_LOGFILE

   If you set the CCACHE_LOGFILE environment variable then ccache will
   write some log information on cache hits and misses in that
   file. This is useful for tracking down problems.


   CCACHE_PATH

   You can optionally set CCACHE_PATH to a colon separated path where ccache will
   look for the real compilers. If you don't do this then ccache will
   look for the first executable in the normal PATH that isn't a link
   to ccache itself.


   CCACHE_DISABLE

   If you set the environment variable CCACHE_DISABLE then ccache will
   just call the real compiler, bypassing the cache completely.


Differences
-----------

The biggest differences between Erik's compilercache script and ccache
are:

- ccache is written in C, which makes it a bit faster (calling out to
  external programs is mostly what slowed down the scripts).

- ccache can automatically find the real compiler

- ccache keeps statistics on hits/misses

- ccache can do automatic cache management

- ccache can cache compiler output that includes warnings. In many
  cases this gives ccache a much higher cache hit rate.

- ccache can handle argument lists with spaces

- compilercache does much smarter token unification using a C
  lexer. This means that reformatting of source code is much more
  likely to result in a cache miss with ccache than with
  compilercache. I may add this in future, although it is perhaps
  debatable whether the much slower parsing is worth it.

Note however that ccache is a very new piece of code (as of March
2002) whereas Erik's scripts have had a *lot* more testing. 

The overall speedup compared to compilercache is quite dramatic. For
compiling rsync I get:

  normal build 20.74 seconds

  compilercache uncached 28.9 seconds
  compilercache cached 10.5 seconds

  ccache uncached 24.6 seconds
  ccache cached 4.6 seconds

Cleaning size management
------------------------

By default ccache has no limit on the cache size. You can set a limit
using the "ccache -M" and "ccache -F" options, which set the size and
number of files limits.

When these limits are reached ccache will reduce the cache to 20%
below the numbers you specified in order to avoid doing the cache
clean operation too often.

How it works
------------

The basic idea is to detect when you are compiling exactly the same
code a 2nd time and use the previously compiled output. You detect
that it is the same code by forming a hash of:

  - the pre-processor output from running the compiler with -E
  - the command line options
  - the real compilers size and modification time
  - any stderr output generated by the compiler

These are hashed using md4 (a strong hash) and a cache file is formed
based on that hash result. When the same compilation is done a second
time ccache is able to supply the correct compiler output (including
all warnings etc) from the cache.

ccache has been carefully written to always produce exactly the same
compiler output that you would get without the cache. If you ever
discover a case where ccache changes the output of your compiler then
please let me know.

===================
Andrew Tridgell
http://samba.org/~tridge/
March 2002

PS: I get far too much email to answer, so don't be surprised if you
have a hard time getting hold of me. You might try the
#samba-technical IRC channel on irc.openprojects.net if you need me
quickly.