This patch splits and cleans up error handling logic for loading BTF data.
Previously, if BTF data was parsed successfully, but failed to load into
kernel, we'd report nonsensical error code, instead of error returned from
btf__load(). Now btf__new() and btf__load() are handled separately with proper
cleanup and warning reporting.
Fixes: d29d87f7e612 ("btf: separate btf creation and loading")
Reported-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
We could end up in situation when we have object file w/ all btf
info, but kernel does not support btf yet. In this situation
currently libbpf just set obj->btf to NULL w/o freeing it first.
This patch is fixing it by making sure to run btf__free first.
Fixes: d29d87f7e612 ("btf: separate btf creation and loading")
Signed-off-by: Nikita V. Shirokov <tehnerd@tehnerd.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
With xsk.{o,h} changes we started including Linux UAPI's if_xdp.h
header. We need to sync it along the other UAPI headers.
Also updated README to reflect this.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
This patch makes sure we build AF_XDP-related code as part of libbpf. This
also required copying few uapi/linux headers and adding few used definitions
in include headers.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
This patch syncs the uapi bpf.h to tools/ and also updates
bpf_herlpers.h in tools/
Signed-off-by: Lawrence Brakmo <brakmo@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When checking available canonical candidates for struct/union algorithm
utilizes btf_dedup_is_equiv to determine if candidate is suitable. This
check is not enough when candidate is corresponding FWD for that
struct/union, because according to equivalence logic they are
equivalent. When it so happens that FWD and STRUCT/UNION end in hashing
to the same bucket, it's possible to create remapping loop from FWD to
STRUCT and STRUCT to same FWD, which will cause btf_dedup() to loop
forever.
This patch fixes the issue by additionally checking that type and
canonical candidate are strictly equal (utilizing btf_equal_struct).
Fixes: d5caef5b5655 ("btf: add BTF types deduplication algorithm")
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Default size of dedup table (16k) is good enough for most binaries, even
typical vmlinux images. But there are cases of binaries with huge amount
of BTF types (e.g., allyesconfig variants of kernel), which benefit from
having bigger dedup table size to lower amount of unnecessary hash
collisions. Tools like pahole, thus, can tune this parameter to reach
optimal performance.
This change also serves double purpose of allowing tests to force hash
collisions to test some corner cases, used in follow up patch.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Fix invalid formatting of pointer arg.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
The "ref_type_id" variable needs to be signed for the error handling
to work.
Fixes: d5caef5b5655 ("btf: add BTF types deduplication algorithm")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
For historical reasons the helper to loop over maps in an object
is called bpf_map__for_each while it really should be called
bpf_object__for_each_map. Rename and add a correctly named
define for backward compatibility.
Switch all in-tree users to the correct name (Quentin).
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Quentin Monnet <quentin.monnet@netronome.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
This commit adds AF_XDP support to libbpf. The main reason for this is
to facilitate writing applications that use AF_XDP by offering
higher-level APIs that hide many of the details of the AF_XDP
uapi. This is in the same vein as libbpf facilitates XDP adoption by
offering easy-to-use higher level interfaces of XDP
functionality. Hopefully this will facilitate adoption of AF_XDP, make
applications using it simpler and smaller, and finally also make it
possible for applications to benefit from optimizations in the AF_XDP
user space access code. Previously, people just copied and pasted the
code from the sample application into their application, which is not
desirable.
The interface is composed of two parts:
* Low-level access interface to the four rings and the packet
* High-level control plane interface for creating and setting
up umems and af_xdp sockets as well as a simple XDP program.
Tested-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Previous version of script relied on squashing baseline commit and
rebasing rest of commits on top of it. This doesn't work well with git
histories containing merges. This patch changes approach by cherry-picking
commits that have libbpf-related changes and then rewriting history since
last checkpoint.
This still might fail if there were manually resolved merge conflicts for
libbpf, but it's the best we can get as far as I can tell.
Script now also verifies that state of libbpf in Linux's repository exactly
matches the state of libbpf in github repo.
If everything goes smoothing (including verification step), we clean up after
ourselves and only leave libbpf-sync-XXX branch in github's libbpf repo to
push to remote.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
* tools/libbpf: support bigger BTF data sizes
While it's understandable why kernel limits number of BTF types to 65535
and size of string section to 64KB, in libbpf as user-space library it's
too restrictive. E.g., pahole converting DWARF to BTF type information
for Linux kernel generates more than 3 million BTF types and more than
3MB of strings, before deduplication. So to allow btf__dedup() to do its
work, we need to be able to load bigger BTF sections using btf__new().
Singed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline commit: 789f6bab849e04ea029c09b81dc8401dc0268cf9
Checkpoint commit: 5aab392c55c96f9bb26d9294f965f156a87ee81c
Andrii Nakryiko (1):
tools/libbpf: support bigger BTF data sizes
src/btf.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
--
2.17.1
This script automates the process of applying libbpf-relevant changes
from kernel repository on top of current state of libbpf repository.
It uses CHECKPOINT-COMMIT file to keep track of last commit in kernel
repo up to which libbpf is in sync with. If there are any new libbpf
changes in kernel repository, script extracts them, preserving original
commit metadata. It also creates a "sync commit" using cover letter as
a template, which nicely summarizes changes since last sync with kernel.
Usage: ./scripts/sync-kernel.sh <linux-repo> <libbpf-repo>
If it succeeds, script will create a bunch of local commits in
<libbpf-repo> in separate branch, which can be easily pushed into github
to create a pull request.
Script tries to clean up after itself, except in case of failure. But it
doesn't clean up timestamped branches it creates in both kernel and
libbpf repositories for now. We can add that later.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
bpf_common.h is hardly ever changed so it was not installed together
with other uapi headers. Some environments are still prefer to be
consistent and install all relevant uapi headers so add bpf_common.h to
uapi headers.
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Sync with the following bug fix:
commit a8a1f7d09cfc7e18874786c7634c9e71384fcd4e (HEAD -> bpf-next2, bpf-next/master)
Author: Stanislav Fomichev <sdf@google.com>
Date: Mon Feb 4 16:20:55 2019 -0800
libbpf: fix libbpf_print
With the recent print rework we now have the following problem:
pr_{warning,info,debug} expand to __pr which calls libbpf_print.
libbpf_print does va_start and calls __libbpf_pr with va_list argument.
In __base_pr we again do va_start. Because the next argument is a
va_list, we don't get correct pointer to the argument (and print noting
in my case, I don't know why it doesn't crash tbh).
......
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
Otherwise, we will have the following compilation error:
/home/yhs/work/bcc-new/src/cc/libbpf/src/bpf_prog_linfo.c:
In function ‘dissect_jited_func’:
/home/yhs/work/bcc-new/src/cc/libbpf/src/bpf_prog_linfo.c:88:10:
error: ‘EINVAL’ undeclared (first use in this function)
return -EINVAL;
The original linux/err.h at linux:tools/include directory
does include "asm/errno.h" as well.
Signed-off-by: Yonghong Song <yhs@fb.com>
Builds on debian stretch are failing due to:
../scripts/check-reallocarray.sh: 17: ../scripts/check-reallocarray.sh: Syntax error: Bad fd number
Fix the redirect of stdout and stderr.
Signed-off-by: David Ahern <dsahern@gmail.com>
sync with latest bpf-next tree.
the include/linux/filter.h is created as libbpf.c tries
to use various insn define macros.
Signed-off-by: Yonghong Song <yhs@fb.com>
* Sync from bpf-next
Sync the following commits from bpf-next:
commit ab9e08482122 ("libbpf: Per-symbol visibility for DSO")
commit c034a177d3c8 ("bpf: bpftool, add flag to allow non-compat map definitions")
Signed-off-by: Andrey Ignatov <rdna@fb.com>
* Use -fvisibility=hidden by default for DSO
This is Makefile part of:
commit ab9e08482122 ("libbpf: Per-symbol visibility for DSO")
See original commit for details.
Introduce multiple improvements to Makefile to make the build more
flexible and support install:
* Support overriding CFLAGS by user but keep required flags in place.
ALL_FLAGS is used in Makefile as recommended in [1].
* Add additional BUILD_SHARED flag to build dynamically linked flavor of
the library. If the flag is set, -fPIC is also passed to make it
possible to build .so.
* Support building in a separate directory provided by OBJDIR variable.
* Add multiple install targets. By default the library itself and libbpf
headers are installed (install target). UAPI headers can be optionally
installed by user.
* All installation paths, including PREFIX, library and include
directories can be overridden. UAPI can be made different from include
directory for libbpf headers. That makes it possible to keep latest
<linux/bpf.h> in a place that doesn't conflict with the one installed
e.g. by kernel-headers package and use it in user's build system.
* Support DESTDIR (see [2]).
* Support overriding LDFLAGS.
* Use utilities such as rm directly as recommended in [3].
* Use compiler and related programs (such as ar) via make variables as
recommended in [3].
* In clean rule remove all possible build artifacts not to rely on passed
options (e.g. if build was done w/ BUILD_SHARED, but clean w/o it).
* Document new build options in README.
[1] https://www.gnu.org/software/make/manual/html_node/Command-Variables.html#Command-Variables
[2] https://www.gnu.org/prep/standards/html_node/DESTDIR.html
[3] https://www.gnu.org/software/make/manual/html_node/Utilities-in-Makefiles.html#Utilities-in-Makefiles
Signed-off-by: Andrey Ignatov <rdna@fb.com>
This initial commit added the following files
from bpf-next repository:
src:
<files from linux:tools/lib/bpf>
bpf.c bpf.h btf.c btf.h libbpf.c libbpf.h
libbpf_errno.c netlink.c nlattr.c nlattr.h
str_error.c str_error.h
include:
<files from linux:tools/include/uapi/linux>
uapi/linux/{bpf.h, btf.h}
<files from linux:tools/include/tools>
tools/libc_compat.h
The following files are also added:
include/linux/{err.h, kernel.h, list.h, overflow.h, types.h}
These files are customized headers to satisfy compilation.
Their original counterparts are at linux:tools/include/linux
directory.
Signed-off-by: Yonghong Song <yhs@fb.com>
