One place in the description referred to the parameter for
.deb/.ddeb files as "-D" as opposed to the correct "-U".
Signed-off-by: Andreas Ziegler <andreas.ziegler@fau.de>
A couple of closely related pieces of work allow more early warning
about low storage/memory conditions:
- New prometheus metrics to track filesystem freespace, and more
details about some errors.
- Frequent checking of $TMPDIR freespace, to trigger fdcache
emergency flushes.
- Switch to floating point prometheus metrics, to communicate
fractions - and short time intervals - accurately.
- Fix startup-time pthread-creation error handling.
Testing is smoke-test-level only as it is hard to create
free-space-limited $TMPDIRs. Locally tested against tiny through
medium tmpfs filesystems, with or without sqlite db also there. Shows
a pleasant stream of diagnostics and metrics during shortage but
generally does not fail outright. However, catching an actual
libstdc++- or kernel-level OOM is beyond our ken.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
On very large servers, it's desirable to be able to interrupt a rescan
or groom cycle. SIGUSR[12] now do that. (Unfortunately, this is not
practically testable in the testsuite, since these cycles are so fast
on that small dataset.) We also expose more internal progress count
about the grooming pass, so the administrator can assess possible need
to interrupt.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
When starting debuginfod with --port=0 it would start using a random port
(possibly different for ipv4 and ipv6). This seems to be an accidental
and not very useful functionality. Just produce an error when started
with --port=0.
https://sourceware.org/bugzilla/show_bug.cgi?id=25728
Signed-off-by: Mark Wielaard <mark@klomp.org>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Programs are sometimes compiled with source path names containing
noise like /./ or // or /foo/../, and these survive into DWARF. This
code allows either raw or canonicalized pathnames in the webapi, by
letting the client pass things verbatim, and letting the server
store/accept both raw and canonicalized path names for source files.
Tests included & docs updated.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Reported-by: Eli Schwartz <eschwartz@archlinux.org>
Tested-by: Eli Schwartz <eschwartz@archlinux.org>
PR25502: debuginfod client should default to XDG cache
Location of client cache now defaults to $XDG_CACHE_HOME/debuginfod_client.
If XDG_CACHE_HOME is not set then fallback to $HOME/.cache/debuginfod_client.
Also maintain backwards compatibility with the previous default path-
if $HOME/.debuginfod_client_cache exists, use that instead of the new
defaults.
Signed-off-by: Aaron Merey <amerey@redhat.com>
It was reported that libarchive (bsdtar) at least as far back as rhel7
(3.1.2) can natively process RPM files, so there's no need to mediate
those accesses through rpm2cpio. There's no noteworthy performance or
testing impact.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Signed-off-by: Mark Wielaard <mark@klomp.org>
Introduce new option --fdcache-prefetch to accelerate repeated
queries from the same debuginfo archive.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Add a '-Z EXT[=CMD]' option to debuginfod, which lets it scan any given
extension and run CMD on it to unwrap distro archives. For example,
for arch-linux pacman files, -Z '.tar.zst=zstdcat' lets debuginfod
grok debug and source content in split-debuginfo files.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Add a facility to service webapi and dwz/altdebug requests that
resolve to archives via a $TMPDIR file cache. This permits
instantaneous dwz resolution during -debuginfo rpm scanning, and also
instantanous duplicate webapi requests. The cache is limited both in
number of entries and in storage space. Heuristics provide
serviceable defaults.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
We switch from a thread per supplied PATH, with a semaphore based
concurrency control, to a fixed number of worker threads collecting
the result of a plain directory traversal being put into a work queue.
This allows maximal continuous concurrency, even if the PATH
directories are dramatically differently sized. There is no more need
to use concurrency-motivated subdirectory wildcards for PATH entries:
just a single top level directory will work fast. doc & tests incl.
Use just one timeout using CURLOPT_LOW_SPEED_TIME (default 90 seconds)
and CURLOPT_LOW_SPEED_LIMIT (100K).
Signed-off-by: Mark Wielaard <mark@klomp.org>
Add support for scanning .deb / .ddeb files, enabled with a new
command line option "-U". Using a synthetic .deb/.ddeb from a Ubuntu
18 machine, extend the debuginfod testsuite with some .deb processing,
if the dpkg-deb binary is installed.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
This webapi extensions allows admins to hook up debuginfod to a
prometheus-compatible monitoring system for general situational
statistics. The metrics are simple enough that local curl requests
can give a user a sense of what's going on. The metrics are
documented as unstable with respect to future versions.
In order to support file/rpm archives that are organized via symlink
trees, add an "-L" option to debuginfod, meaning about the same as for
find(1) or ls(1): to traverse rather than ignore symlinks.
Add the server to the debuginfod/ subdirectory. This is a highly
multithreaded c++11 program (still buildable on rhel7's gcc 4.8,
which is only partly c++11 compliant). Includes an initial suite
of tests, man pages, and a sample systemd service.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Signed-off-by: Aaron Merey <amerey@redhat.com>
lizhenlin