16 Commits

Author SHA1 Message Date
Peter Collingbourne
daa4acc2eb Support: Have directory_iterator::status() return FindFirstFileEx/FindNextFile results on Windows.
This allows clients to avoid an unnecessary fs::status() call on each
directory entry. Because the information returned by FindFirstFileEx
is a subset of the information returned by a regular status() call,
I needed to extract a base class from file_status that contains only
that information.

On my machine, this reduces the time required to enumerate a ThinLTO
cache directory containing 520k files from almost 4 minutes to less
than 2 seconds.

Differential Revision: https://reviews.llvm.org/D38716

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315378 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-10 22:19:46 +00:00
Peter Collingbourne
5a653c222d Fix a misleading indentation warning.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306130 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-23 17:17:47 +00:00
Peter Collingbourne
5974d48eea Make the size specification for cache_size_bytes case insensitive.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306129 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-23 17:13:51 +00:00
Peter Collingbourne
5cc49a2645 Add a ThinLTO cache policy for controlling the maximum cache size in bytes.
This is useful when an upper limit on the cache size needs to be
controlled independently of the amount of the amount of free space.

One use case is a machine with a large number of cache directories
(e.g. a buildbot slave hosting a large number of independent build
jobs). By imposing an upper size limit on each cache directory,
users can more easily estimate the server's capacity.

Differential Revision: https://reviews.llvm.org/D34547

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@306126 91177308-0d34-0410-b5e6-96231b3b80d8
2017-06-23 17:05:03 +00:00
Peter Collingbourne
22910652da Support, LTO: When pruning a directory, ignore files matching a prefix.
This is a safeguard against data loss if the user specifies a directory
that is not a cache directory. Teach the existing cache pruning clients
to create files with appropriate names.

Differential Revision: https://reviews.llvm.org/D31109

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@298271 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-20 16:41:57 +00:00
Peter Collingbourne
13c3208403 Fix pessimising moves.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297928 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-16 03:54:38 +00:00
Peter Collingbourne
2995c258a5 Support: Add a cache pruning policy parser.
The idea is that the policy string fully specifies the policy and is portable
between clients.

Differential Revision: https://reviews.llvm.org/D31020

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297927 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-16 03:42:00 +00:00
Peter Collingbourne
2edabdec2a Support: Simplify the CachePruning API. NFCI.
Change the function that implements the pruning into a free function that
takes the policy as a struct argument.

Differential Revision: https://reviews.llvm.org/D31009

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297907 91177308-0d34-0410-b5e6-96231b3b80d8
2017-03-15 22:54:18 +00:00
Pavel Labath
2864c2ae90 Remove TimeValue usage from llvm/Support
Summary:
This is a follow-up to D25416. It removes all usages of TimeValue from
llvm/Support library (except for the actual TimeValue declaration), and replaces
them with appropriate usages of std::chrono. To facilitate this, I have added
small utility functions for converting time points and durations into appropriate
OS-specific types (FILETIME, struct timespec, ...).

Reviewers: zturner, mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25730

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@284966 91177308-0d34-0410-b5e6-96231b3b80d8
2016-10-24 10:59:17 +00:00
Vassil Vassilev
76c5395c55 Missing includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@281450 91177308-0d34-0410-b5e6-96231b3b80d8
2016-09-14 08:55:18 +00:00
Pawel Bylica
01b8445ea6 CachePruning: correct comment about file order. NFC
Summary: Actually the list of cached files is sorted by file size, not by last accessed time. Also remove unused file access time param for a helper function.

Reviewers: joker-eph, chandlerc, davide

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D21639

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@273852 91177308-0d34-0410-b5e6-96231b3b80d8
2016-06-27 08:46:23 +00:00
NAKAMURA Takumi
46303d8ce6 CachePruning.cpp: Don't use errno.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269565 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-14 14:21:39 +00:00
Mehdi Amini
2a866dbec6 CachePruning: early exit if no path supplied
From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266965 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-21 06:43:45 +00:00
Mehdi Amini
0224d5ce63 Add debugging to the cache pruning
From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266686 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-18 21:54:00 +00:00
Mehdi Amini
a2ea8f5e75 CachePruning: fix typo, we accumulate file size here, not time
From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266685 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-18 21:53:55 +00:00
Mehdi Amini
22706dc4c0 Add Cache Pruning support
Incremental LTO will usea cache to store object files.
This patch handles the pruning part of the cache, exposing
a few knobs:

- Pruning interval: the implementation keeps a "timestamp" file in the
  directory and will scan it only after a given interval since the
  last modification of the timestamp file. This is for performance
  purpose, we don't want to scan continuously the folder.
- Entry expiration: this is the time after which a file that hasn't
  been used is remove from the cache.
- Maximum size: expressed in percentage of the available disk space,
  it helps to avoid that we blow up the disk space.

http://reviews.llvm.org/D18422

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@265209 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-02 03:28:26 +00:00