PR27701 implemented curl handle reuse in debuginfod_client objects,
but with an unexpected bug. Server responses returning an error
"latched" because the curl_easy handles for error cases weren't all
systematically removed from the curl multi handle. This prevented
their proper re-addition the next time.
This version of the code simplfies matters by making only the curl
curl_multi handle long-lived. This turns out to be enough, because it
can maintain a pool of long-lived http/https connections and related
data, and lend them out to short-lived curl_easy handles. This mode
handles errors or hung downloads even better, because the easy handles
don't undergo complex state transitions between reuse.
A new test case confirms this correction via the federating debuginfod
instance (cleaning caches between subtests to make sure http* is being
used and reused).
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
New function in system.h that returns true if a string has a given
prefix, false otherwise. Use it in place of strncmp.
Signed-off-by: Martin Liška <mliska@suse.cz>
Add debuginfod_config_cache for reading and writing to cache
configuration files, make use of the function within
debuginfod_clean_cache and debuginfod_query_server.
In debuginfod_query_server, create 000-permission file on failed
queries. Before querying each BUILDID, if corresponding 000 file
detected, compare its stat mtime with parameter from
.cache/cache_miss_s. If mtime is fresher, then return ENOENT and
exit; otherwise unlink the 000 file and proceed to a new query.
tests: add test in run-debuginfod-find.sh
test if the 000 file is created on failed query; if querying the
same failed BUILDID, whether the query should proceed without
going through server; set the cache_miss_s to 0 and query the same
buildid, and this time should go through the server.
Signed-off-by: Alice Zhang <alizhang@redhat.com>
With PR25365, we accidentally lost the ability to rmdir client-cache
directories corresponding to buildids. Bring this back, with some
attention to a possible race between a client doing cleanup and
another client doing lookups at the same time.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Client objects now carry long-lived curl handles for outgoing
connections. This makes it more efficient for multiple sequential
queries, because the TCP connections and/or TLS state info are kept
around awhile, avoiding O(100ms) setup latencies. debuginfod is
adjusted to take advantage of this for federation. Other clients
should gradually do this too, perhaps including elfutils itself (in
the libdwfl->debuginfod_client hooks).
A large gdb session with 117 debuginfo downloads was observed to run
twice as fast (45s vs. 1m30s wall-clock time), just in nuking this
extra setup latency. This was tested via a debuginfod intermediary:
it should be even faster once gdb reuses its own debuginfod_client.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Clang warns about this:
../../debuginfod/debuginfod-client.c:899:28: error: implicit conversion from 'long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Werror,-Wimplicit-int-float-conversion]
pa = (dl > LONG_MAX ? LONG_MAX : (long)dl);
~ ^~~~~~~~
/usr/lib64/clang/10.0.1/include/limits.h:47:19: note: expanded from macro 'LONG_MAX'
^~~~~~~~~~~~
<built-in>:38:22: note: expanded from here
^~~~~~~~~~~~~~~~~~~~
Modified for jakub's observation about LONG_MAX overflow.
Signed-off-by: Timm Bäder <tbaeder@redhat.com>
Use defined constants for permission values. Also add fallback
definitions for them in system.h, to allow for compatibility with
systems that don't provide these macros.
Include system.h in all tests/ files that required it.
Signed-off-by: Érico Rolim <erico.erc@gmail.com>
Signed-off-by: Mark Wielaard <mark@klomp.org>
When a file couldn't be retrieved because of an bad HTTPS certificate
find-debuginfod currently says:
Server query failed: No such file or directory
With this patch it will say:
Server query failed: Connection refused
Signed-off-by: Mark Wielaard <mark@klomp.org>
Check scheme instead of effective url so that user may abbreviate
DEBUGINFOD_URL. Add one test for scheme free http url.
Notice that libcurl does not provide an almighty scheme free url
support, /path/to/something without FILE:// can not be recognized
in most circumstances, therefore for the neatness of our code
structure, DEBUGINFOD_ URL of scheme "FILE" must be input as URI.
Signed-off-by: Alice Zhang <alizhang@redhat.com>
Make it possible to build just the debuginfod client or to create a
dummy libdebuginfod that doesn't link against libcurl. The dummy library
can be used for bootstrapping. For testing purposes you can also build
debuginfod against the dummy libdebuginfod but then the debuginfod
server will not be able to do delegation.
Signed-off-by: Mark Wielaard <mark@klomp.org>
When allocating handle_data we should check for out of memory failures.
Also when the allocation has succeeded make sure we always clean up by
going to out1 on any future errors. So move the curl_multi_init call
earlier, because that goes to out0 on failure.
Signed-off-by: Mark Wielaard <mark@klomp.org>
When is debuginfod_query_server is given an hexadecimal string as
build-id build_id_len will be zero. We were checking the size of
the build_id_bytes destination string instead of the string length
of build_id input string. Make sure the input string is not too
big or strcpy might overwrite then end of the build_id_bytes array.
Signed-off-by: Mark Wielaard <mark@klomp.org>
We need to make sure that we can always place a zero terminator at
the end of suffix when we are copying the filename. So add one more
char to the suffix array. And make sure that we can always add an
extra escape character when we need to escape the current character.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Although we check for and/or create the interval_path right before,
there is still a possibility that the fopen call fails. Handle that
as if the file is unreadable.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Document and sanity check the format of the header string form that can
be passed to debuginfod_add_http_header. It should contain precisely
one colon, which cannot be the first or last character. And the function
should only be used to add optional headers, not replace any existing
standard ones. Anything else isn't supported.
Signed-off-by: Mark Wielaard <mark@klomp.org>
The saga of clean $DEBUGINFOD_PROGRESS=1 output continues. Previous
code would sometimes insert a \n (a blank line) into the output
stream, even if the target file was found in a cache and thus the
progressfn was never called. New code sets a client flag to track
this case more precisely.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Extend the debuginfod client API with a function to stuff outgoing
headers into libcurl http transfers. Use this from debuginfod so
federated trees of debuginfod/httpd can trace back to to the
originating client for administrative purposes. Docs & test included.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
libcurl < 7.42 lacks the CURLOPT_PATH_AS_IS flag, but extraneous
client-side canonicalization is mostly harmless.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Reported-by: Mark Wielaard <mark@klomp.org>
Programs are sometimes compiled with source path names containing
noise like /./ or // or /foo/../, and these survive into DWARF. This
code allows either raw or canonicalized pathnames in the webapi, by
letting the client pass things verbatim, and letting the server
store/accept both raw and canonicalized path names for source files.
Tests included & docs updated.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Reported-by: Eli Schwartz <eschwartz@archlinux.org>
Tested-by: Eli Schwartz <eschwartz@archlinux.org>
A previous commit changed the default_progressfn output format
to \rFOOBAR, to be terminated by an \n when the download finished.
The \n terminator was conditional on the wrong thing (env var
setting, rather than actual progressfn setting), so the \n could
be printed even if an app overrode the default.
This function lets a client know, during or after a progressfn
callback, what the url of the winning outgoing download is/was.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Add a pair of functions to associate a void* parameter with a client
object. Requested by GDB team as a way to pass file names and such
user-interface data through to a progressfn callback.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Update cache_path with the path of the new default directory when this
directory already exists. Previously cache_path was updated only when
creating the new default directory and would otherwise be set to the
old default path.
Signed-off-by: Aaron Merey <amerey@redhat.com>
PR25502: debuginfod client should default to XDG cache
Location of client cache now defaults to $XDG_CACHE_HOME/debuginfod_client.
If XDG_CACHE_HOME is not set then fallback to $HOME/.cache/debuginfod_client.
Also maintain backwards compatibility with the previous default path-
if $HOME/.debuginfod_client_cache exists, use that instead of the new
defaults.
Signed-off-by: Aaron Merey <amerey@redhat.com>
When file:// is used for DEBUGINFOD_URLS, then the response code for a
successful server query is 0 and not 200.
Using file:// can be helpful when you want to test your
debuginfod-client integration against a mocked file tree that mimics the
HTTP URLs from the debuginfod server. This way you don't have to run the
debuginfod server at all.
Fixes https://sourceware.org/bugzilla/show_bug.cgi?id=25600
Signed-off-by: Konrad Kleine <kkleine@redhat.com>
Avoid deleting general files and directories in case a user
mis-sets $DEBUGINFOD_CACHE_PATH or accidentally moves a file
into the cache.
Signed-off-by: Aaron Merey <amerey@redhat.com>
Use just one timeout using CURLOPT_LOW_SPEED_TIME (default 90 seconds)
and CURLOPT_LOW_SPEED_LIMIT (100K).
Signed-off-by: Mark Wielaard <mark@klomp.org>
It may be useful for a debuginfod server operator to know what kinds
of clients make webapi requests. This is mainly as a
telemetry/diagnostic (though the data cannot be really trusted). It
may also be useful to automate downloading of distro packages to a
debuginfod server in the case of an unknown hex buildid. doc/testing
not affected as these are diagnostics.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Signed-off-by: Mark Wielaard <mjw@redhat.com>
This facility allows a default progress-printing function to be
installed if the given environment variable is set. Some larger usage
experience (systemtap fetching kernels) indicates the default timeout
is too short, so forked it into a connection timeout (default short)
and a transfer timeout (default unlimited).
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
If the debuginfod-client isn't configured we should do as little
as possible. Simply return early with ENOSYS if no servers are
configured. This means we won't check
This does change the behavior of the debuginfod_find calls slightly.
Previously we would setup and check the cache if the given build-id
was valid. Which might have provided a result if an earlier client
had run with the same cache and valid server URLs which knew about
that particular build-id. Now we don't return any cached results
unless at least one server is configured.
This prevents selinux errors when the library is used in a confined
setup.
Signed-off-by: Mark Wielaard <mark@klomp.org>
GCC10 warns when converting the value of one enum type into another:
debuginfod-client.c:530:24: error: implicit conversion from ‘CURLcode’
to ‘CURLMcode’ [-Werror=enum-conversion]
530 | curl_res = curl_easy_getinfo(target_handle,
| ^
libcurl has different error code enums. The "easy" interfaces return
a CURLcode error. The "multi" interface functions return a CURLMcode.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Add a mandatory debuginfod_begin()/_end() call pair to manage a client
object that represents persistent but non-global state. From libdwfl,
dlopen the debuginfod.so client library early on. This hopefully
makes sure that the code (and the libcurl.so dependency) is loaded
before the program goes into multi-threaded mode.
Signed-off-by: Mark Wielaard <mark@klomp.org>
For interactive clients such as gdb, interruptibility is important for
usability during longer downloads. This patchset adds a
download-progress callback function to the debuginfod client library,
with which a caller app can interrupt a download as well as be
notified of its quantitative progress.
Introduce the debuginfod/ subdirectory, containing the client for a
new debuginfo-over-http service, in shared-library and command-line
forms. Two functions in libdwfl make calls into the client library to
fetch elf/dwarf files by buildid, as a fallback. Instead of normal
dynamic linking (thus pulling in a variety of curl dependencies),
the libdwfl hooks use dlopen/dlsym. Server & tests coming in patch 2.
Signed-off-by: Aaron Merey <amerey@redhat.com>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
lizhenlin