Commit Graph

282 Commits

Author SHA1 Message Date
Alexei Starovoitov
1b27702c14 bpf: Check types of arguments passed into helpers
Introduce new helper that reuses existing skb perf_event output
implementation, but can be called from raw_tracepoint programs
that receive 'struct sk_buff *' as tracepoint argument or
can walk other kernel data structures to skb pointer.

In order to do that teach verifier to resolve true C types
of bpf helpers into in-kernel BTF ids.
The type of kernel pointer passed by raw tracepoint into bpf
program will be tracked by the verifier all the way until
it's passed into helper function.
For example:
kfree_skb() kernel function calls trace_kfree_skb(skb, loc);
bpf programs receives that skb pointer and may eventually
pass it into bpf_skb_output() bpf helper which in-kernel is
implemented via bpf_skb_event_output() kernel function.
Its first argument in the kernel is 'struct sk_buff *'.
The verifier makes sure that types match all the way.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20191016032505.2089704-11-ast@kernel.org
2019-10-22 16:15:55 -07:00
Alexei Starovoitov
39cf9fc90f libbpf: Auto-detect btf_id of BTF-based raw_tracepoints
It's a responsiblity of bpf program author to annotate the program
with SEC("tp_btf/name") where "name" is a valid raw tracepoint.
The libbpf will try to find "name" in vmlinux BTF and error out
in case vmlinux BTF is not available or "name" is not found.
If "name" is indeed a valid raw tracepoint then in-kernel BTF
will have "btf_trace_##name" typedef that points to function
prototype of that raw tracepoint. BTF description captures
exact argument the kernel C code is passing into raw tracepoint.
The kernel verifier will check the types while loading bpf program.

libbpf keeps BTF type id in expected_attach_type, but since
kernel ignores this attribute for tracing programs copy it
into attach_btf_id attribute before loading.

Later the kernel will use prog->attach_btf_id to select raw tracepoint
during bpf_raw_tracepoint_open syscall command.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20191016032505.2089704-6-ast@kernel.org
2019-10-22 16:15:55 -07:00
Alexei Starovoitov
bc4a6e9709 bpf: Add attach_btf_id attribute to program load
Add attach_btf_id attribute to prog_load command.
It's similar to existing expected_attach_type attribute which is
used in several cgroup based program types.
Unfortunately expected_attach_type is ignored for
tracing programs and cannot be reused for new purpose.
Hence introduce attach_btf_id to verify bpf programs against
given in-kernel BTF type id at load time.
It is strictly checked to be valid for raw_tp programs only.
In a later patches it will become:
btf_id == 0 semantics of existing raw_tp progs.
btd_id > 0 raw_tp with BTF and additional type safety.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20191016032505.2089704-5-ast@kernel.org
2019-10-22 16:15:55 -07:00
Andrii Nakryiko
4a50ceb043 Makefile: back-port _FILE_OFFSET_BITS=64 and _LARGEFILE64_SOURCE to Makefile
Upstream commit 71dd77fd4bf7 ("libbpf: use LFS (_FILE_OFFSET_BITS) instead
of direct mmap2 syscall") added _FILE_OFFSET_BITS=64 and
_LARGEFILE64_SOURCE CFLAGS. Back-port them to Github's mirror to avoid
compilation problems on ARM.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-22 14:50:23 -07:00
Andrii Nakryiko
4d86cae4f0 ci: disable GCC's -Wstringop-truncation noisy error
This error is usually a false positive for us. Disable it.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
33b374395f sync: adjust sync script for test_libbpf.c rename and bpf_helper_defs.h
Accomodate changes:
- test_libbpf.cpp was renamed to test_libbpf.c;
- bpf_helper_defs.h should be ignored for consistency check at the end,
  as it's not checked in on linux side;

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
ade4409352 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   f05c2001ecc98629cecd47728e4db11e5a17e58d
Checkpoint bpf-next commit: da927466a152a9497c05926a95c6aebba6d3ad5b
Baseline bpf commit:        106c35dda32f8b63f88cad7433f1b8bb0056958a
Checkpoint bpf commit:      9e8acd9c44a0dd52b2922eeb82398c04e356c058

Andrii Nakryiko (7):
  libbpf: Fix struct end padding in btf_dump
  libbpf: Generate more efficient BPF_CORE_READ code
  libbpf: Handle invalid typedef emitted by old GCC
  libbpf: Update BTF reloc support to latest Clang format
  libbpf: Refactor bpf_object__open APIs to use common opts
  libbpf: Add support for field existance CO-RE relocation
  libbpf: Add BPF-side definitions of supported field relocation kinds

Ilya Maximets (1):
  libbpf: Fix passing uninitialized bytes to setsockopt

 src/bpf_core_read.h   |  28 ++++++-
 src/btf.c             |  16 ++--
 src/btf.h             |   4 +-
 src/btf_dump.c        |  19 ++++-
 src/libbpf.c          | 169 ++++++++++++++++++++++++++----------------
 src/libbpf.h          |   4 +-
 src/libbpf_internal.h |  25 +++++--
 src/xsk.c             |   1 +
 8 files changed, 180 insertions(+), 86 deletions(-)

--
2.17.1
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
2f9abb2a26 sync: auto-generate latest BPF helpers
Latest changes to BPF helper definitions.
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
fca60960ea libbpf: Add BPF-side definitions of supported field relocation kinds
Add enum definition for Clang's __builtin_preserve_field_info()
second argument (info_kind). Currently only byte offset and existence
are supported. Corresponding Clang changes introducing this built-in can
be found at [0]

  [0] https://reviews.llvm.org/D67980

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20191015182849.3922287-5-andriin@fb.com
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
0db22b01a1 libbpf: Add support for field existance CO-RE relocation
Add support for BPF_FRK_EXISTS relocation kind to detect existence of
captured field in a destination BTF, allowing conditional logic to
handle incompatible differences between kernels.

Also introduce opt-in relaxed CO-RE relocation handling option, which
makes libbpf emit warning for failed relocations, but proceed with other
relocations. Instruction, for which relocation failed, is patched with
(u32)-1 value.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20191015182849.3922287-4-andriin@fb.com
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
807b9d7be1 libbpf: Refactor bpf_object__open APIs to use common opts
Refactor all the various bpf_object__open variations to ultimately
specify common bpf_object_open_opts struct. This makes it easy to keep
extending this common struct w/ extra parameters without having to
update all the legacy APIs.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20191015182849.3922287-3-andriin@fb.com
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
a3d02f9ab4 libbpf: Update BTF reloc support to latest Clang format
BTF offset reloc was generalized in recent Clang into field relocation,
capturing extra u32 field, specifying what aspect of captured field
needs to be relocated. This changes .BTF.ext's record size for this
relocation from 12 bytes to 16 bytes. Given these format changes
happened in Clang before official released version, it's ok to not
support outdated 12-byte record size w/o breaking ABI.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20191015182849.3922287-2-andriin@fb.com
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
54aac21f7e libbpf: Handle invalid typedef emitted by old GCC
Old GCC versions are producing invalid typedef for __gnuc_va_list
pointing to void. Special-case this and emit valid:

typedef __builtin_va_list __gnuc_va_list;

Reported-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/20191011032901.452042-1-andriin@fb.com
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
d8dd0beb98 libbpf: Generate more efficient BPF_CORE_READ code
Existing BPF_CORE_READ() macro generates slightly suboptimal code. If
there are intermediate pointers to be read, initial source pointer is
going to be assigned into a temporary variable and then temporary
variable is going to be uniformly used as a "source" pointer for all
intermediate pointer reads. Schematically (ignoring all the type casts),
BPF_CORE_READ(s, a, b, c) is expanded into:
({
	const void *__t = src;
	bpf_probe_read(&__t, sizeof(*__t), &__t->a);
	bpf_probe_read(&__t, sizeof(*__t), &__t->b);

	typeof(s->a->b->c) __r;
	bpf_probe_read(&__r, sizeof(*__r), &__t->c);
})

This initial `__t = src` makes calls more uniform, but causes slightly
less optimal register usage sometimes when compiled with Clang. This can
cascase into, e.g., more register spills.

This patch fixes this issue by generating more optimal sequence:
({
	const void *__t;
	bpf_probe_read(&__t, sizeof(*__t), &src->a); /* <-- src here */
	bpf_probe_read(&__t, sizeof(*__t), &__t->b);

	typeof(s->a->b->c) __r;
	bpf_probe_read(&__r, sizeof(*__r), &__t->c);
})

Fixes: 7db3822ab991 ("libbpf: Add BPF_CORE_READ/BPF_CORE_READ_INTO helpers")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20191011023847.275936-1-andriin@fb.com
2019-10-15 19:43:48 -07:00
Ilya Maximets
e94f57a9ab libbpf: Fix passing uninitialized bytes to setsockopt
'struct xdp_umem_reg' has 4 bytes of padding at the end that makes
valgrind complain about passing uninitialized stack memory to the
syscall:

  Syscall param socketcall.setsockopt() points to uninitialised byte(s)
    at 0x4E7AB7E: setsockopt (in /usr/lib64/libc-2.29.so)
    by 0x4BDE035: xsk_umem__create@@LIBBPF_0.0.4 (xsk.c:172)
  Uninitialised value was created by a stack allocation
    at 0x4BDDEBA: xsk_umem__create@@LIBBPF_0.0.4 (xsk.c:140)

Padding bytes appeared after introducing of a new 'flags' field.
memset() is required to clear them.

Fixes: 10d30e301732 ("libbpf: add flags to umem config")
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Link: https://lore.kernel.org/bpf/20191009164929.17242-1-i.maximets@ovn.org
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
bda436be4a libbpf: Fix struct end padding in btf_dump
Fix a case where explicit padding at the end of a struct is necessary
due to non-standart alignment requirements of fields (which BTF doesn't
capture explicitly).

Fixes: 351131b51c7a ("libbpf: add btf_dump API for BTF-to-C conversion")
Reported-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Tested-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/20191008231009.2991130-2-andriin@fb.com
2019-10-15 19:43:48 -07:00
Andrii Nakryiko
a30df5c09f makefile: install new BPF-side headers along libbpf user-land ones
Install BPF-side helper headers:
- bpf_helpers.h
- bpf_helper_defs.h
- bpf_tracing.h
- bpf_endian.h
- bpf_core_read.h

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
e776bf7ec7 sync: teach sync script to generate bpf_helper_defs.h
Linux repo doesn't commit bpf_helper_defs.h, as it's re-generated on
build every time. For Github projection, though, it's much nicer to have
this header be pre-generated during sync and commited. This makes
integration story easier for all the users that use libbpf as
a submodule.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
46688687d5 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   02dc96ef6c25f990452c114c59d75c368a1f4c8f
Checkpoint bpf-next commit: f05c2001ecc98629cecd47728e4db11e5a17e58d
Baseline bpf commit:        1bd63524593b964934a33afd442df16b8f90e2b5
Checkpoint bpf commit:      106c35dda32f8b63f88cad7433f1b8bb0056958a

Andrii Nakryiko (7):
  libbpf: Bump current version to v0.0.6
  libbpf: stop enforcing kern_version, populate it for users
  libbpf: add bpf_object__open_{file, mem} w/ extensible opts
  libbpf: fix bpf_object__name() to actually return object name
  uapi/bpf: fix helper docs
  libbpf: Move bpf_{helpers, helper_defs, endian, tracing}.h into libbpf
  libbpf: Add BPF_CORE_READ/BPF_CORE_READ_INTO helpers

 include/uapi/linux/bpf.h |  32 +++----
 src/bpf_core_read.h      | 167 +++++++++++++++++++++++++++++++++
 src/bpf_endian.h         |  72 +++++++++++++++
 src/bpf_helpers.h        |  41 ++++++++
 src/bpf_tracing.h        | 195 +++++++++++++++++++++++++++++++++++++++
 src/libbpf.c             | 183 ++++++++++++++++++------------------
 src/libbpf.h             |  48 +++++++++-
 src/libbpf.map           |   6 ++
 src/libbpf_internal.h    |  32 +++++++
 9 files changed, 661 insertions(+), 115 deletions(-)
 create mode 100644 src/bpf_core_read.h
 create mode 100644 src/bpf_endian.h
 create mode 100644 src/bpf_helpers.h
 create mode 100644 src/bpf_tracing.h

--
2.17.1
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
19cbbd8f52 sync: auto-generate latest BPF helpers
Latest changes to BPF helper definitions.
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
c87b3a6065 libbpf: Add BPF_CORE_READ/BPF_CORE_READ_INTO helpers
Add few macros simplifying BCC-like multi-level probe reads, while also
emitting CO-RE relocations for each read.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20191008175942.1769476-7-andriin@fb.com
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
4c55ba2b19 libbpf: Move bpf_{helpers, helper_defs, endian, tracing}.h into libbpf
Move bpf_helpers.h, bpf_tracing.h, and bpf_endian.h into libbpf. Move
bpf_helper_defs.h generation into libbpf's Makefile. Ensure all those
headers are installed along the other libbpf headers. Also, adjust
selftests and samples include path to include libbpf now.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20191008175942.1769476-6-andriin@fb.com
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
104006a054 uapi/bpf: fix helper docs
Various small fixes to BPF helper documentation comments, enabling
automatic header generation with a list of BPF helpers.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
bf83a95dee libbpf: fix bpf_object__name() to actually return object name
bpf_object__name() was returning file path, not name. Fix this.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
732f598282 libbpf: add bpf_object__open_{file, mem} w/ extensible opts
Add new set of bpf_object__open APIs using new approach to optional
parameters extensibility allowing simpler ABI compatibility approach.

This patch demonstrates an approach to implementing libbpf APIs that
makes it easy to extend existing APIs with extra optional parameters in
such a way, that ABI compatibility is preserved without having to do
symbol versioning and generating lots of boilerplate code to handle it.
To facilitate succinct code for working with options, add OPTS_VALID,
OPTS_HAS, and OPTS_GET macros that hide all the NULL, size, and zero
checks.

Additionally, newly added libbpf APIs are encouraged to follow similar
pattern of having all mandatory parameters as formal function parameters
and always have optional (NULL-able) xxx_opts struct, which should
always have real struct size as a first field and the rest would be
optional parameters added over time, which tune the behavior of existing
API, if specified by user.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
de3c5a17cb libbpf: stop enforcing kern_version, populate it for users
Kernel version enforcement for kprobes/kretprobes was removed from
5.0 kernel in 6c4fc209fcf9 ("bpf: remove useless version check for prog load").
Since then, BPF programs were specifying SEC("version") just to please
libbpf. We should stop enforcing this in libbpf, if even kernel doesn't
care. Furthermore, libbpf now will pre-populate current kernel version
of the host system, in case we are still running on old kernel.

This patch also removes __bpf_object__open_xattr from libbpf.h, as
nothing in libbpf is relying on having it in that header. That function
was never exported as LIBBPF_API and even name suggests its internal
version. So this should be safe to remove, as it doesn't break ABI.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
1a8a75037b libbpf: Bump current version to v0.0.6
New release cycle started, let's bump to v0.0.6 proactively.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20190930222503.519782-1-andriin@fb.com
2019-10-09 14:42:45 -07:00
Andrii Nakryiko
1a26b51b1c meson: kill meson.build as it's not used anymore
Meson.build was added to facilitate systemd integration, but systemd
integration went different direction and we don't need meson.build
anymore. So remove it and not maintain it anymore.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-01 16:51:32 -07:00
Andrii Nakryiko
2cc0829775 ci: execute install step in CI
Add simple execution of `make install` in Debian and Xenial build to
catch most obvious breakages.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-01 12:56:22 -07:00
Andrii Nakryiko
92cb475558 makefile: fix install target
After latest shared vs static libraries fixes, `make install` target
broke as it relied on now removed $(LIBS) variable. This patch fixes
issue by listing $(SHARED_LIBS) and $(STATIC_LIBS) explicitly.

Tested with and without BUILD_STATIC_ONLY.

Fixes: 8b2782a1f2 ("makefile: support libbpf symbol versioning in shared library mode")
Reported-by: Michal Rostecki <mrostecki@opensuse.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-10-01 12:56:22 -07:00
Andrii Nakryiko
8b2782a1f2 makefile: support libbpf symbol versioning in shared library mode
Similarly to Linux's 1bd63524593b ("libbpf: handle symbol versioning properly
for libbpf.a"), add necessary changes to build static and shared object
files separately with extra shared library flags. This allows to
properly handle symbol versioning in shared library mode, while still
having statically linkable library.

Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-09-30 21:42:42 -07:00
Andrii Nakryiko
886e8149a0 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   b41dae061bbd722b9d7fa828f35d22035b218e18
Checkpoint bpf-next commit: 02dc96ef6c25f990452c114c59d75c368a1f4c8f
Baseline bpf commit:        e3439af4a339acd7fddbd6d59b8ecefaac07a611
Checkpoint bpf commit:      1bd63524593b964934a33afd442df16b8f90e2b5

Yonghong Song (1):
  libbpf: handle symbol versioning properly for libbpf.a

 src/libbpf_internal.h | 16 ++++++++++++++++
 src/xsk.c             |  4 ++--
 2 files changed, 18 insertions(+), 2 deletions(-)

--
2.17.1
2019-09-30 16:10:45 -07:00
Yonghong Song
d275397111 libbpf: handle symbol versioning properly for libbpf.a
bcc uses libbpf repo as a submodule. It brings in libbpf source
code and builds everything together to produce shared libraries.
With latest libbpf, I got the following errors:
  /bin/ld: libbcc_bpf.so.0.10.0: version node not found for symbol xsk_umem__create@LIBBPF_0.0.2
  /bin/ld: failed to set dynamic section sizes: Bad value
  collect2: error: ld returned 1 exit status
  make[2]: *** [src/cc/libbcc_bpf.so.0.10.0] Error 1

In xsk.c, we have
  asm(".symver xsk_umem__create_v0_0_2, xsk_umem__create@LIBBPF_0.0.2");
  asm(".symver xsk_umem__create_v0_0_4, xsk_umem__create@@LIBBPF_0.0.4");
The linker thinks the built is for LIBBPF but cannot find proper version
LIBBPF_0.0.2/4, so emit errors.

I also confirmed that using libbpf.a to produce a shared library also
has issues:
  -bash-4.4$ cat t.c
  extern void *xsk_umem__create;
  void * test() { return xsk_umem__create; }
  -bash-4.4$ gcc -c -fPIC t.c
  -bash-4.4$ gcc -shared t.o libbpf.a -o t.so
  /bin/ld: t.so: version node not found for symbol xsk_umem__create@LIBBPF_0.0.2
  /bin/ld: failed to set dynamic section sizes: Bad value
  collect2: error: ld returned 1 exit status
  -bash-4.4$

Symbol versioning does happens in commonly used libraries, e.g., elfutils
and glibc. For static libraries, for a versioned symbol, the old definitions
will be ignored, and the symbol will be an alias to the latest definition.
For example, glibc sched_setaffinity is versioned.
  -bash-4.4$ readelf -s /usr/lib64/libc.so.6 | grep sched_setaffinity
     756: 000000000013d3d0    13 FUNC    GLOBAL DEFAULT   13 sched_setaffinity@GLIBC_2.3.3
     757: 00000000000e2e70   455 FUNC    GLOBAL DEFAULT   13 sched_setaffinity@@GLIBC_2.3.4
    1800: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS sched_setaffinity.c
    4228: 00000000000e2e70   455 FUNC    LOCAL  DEFAULT   13 __sched_setaffinity_new
    4648: 000000000013d3d0    13 FUNC    LOCAL  DEFAULT   13 __sched_setaffinity_old
    7338: 000000000013d3d0    13 FUNC    GLOBAL DEFAULT   13 sched_setaffinity@GLIBC_2
    7380: 00000000000e2e70   455 FUNC    GLOBAL DEFAULT   13 sched_setaffinity@@GLIBC_
  -bash-4.4$
For static library, the definition of sched_setaffinity aliases to the new definition.
  -bash-4.4$ readelf -s /usr/lib64/libc.a | grep sched_setaffinity
  File: /usr/lib64/libc.a(sched_setaffinity.o)
     8: 0000000000000000   455 FUNC    GLOBAL DEFAULT    1 __sched_setaffinity_new
    12: 0000000000000000   455 FUNC    WEAK   DEFAULT    1 sched_setaffinity

For both elfutils and glibc, additional macros are used to control different handling
of symbol versioning w.r.t static and shared libraries.
For elfutils, the macro is SYMBOL_VERSIONING
(https://sourceware.org/git/?p=elfutils.git;a=blob;f=lib/eu-config.h).
For glibc, the macro is SHARED
(https://sourceware.org/git/?p=glibc.git;a=blob;f=include/shlib-compat.h;hb=refs/heads/master)

This patch used SHARED as the macro name. After this patch, the libbpf.a has
  -bash-4.4$ readelf -s libbpf.a | grep xsk_umem__create
     372: 0000000000017145  1190 FUNC    GLOBAL DEFAULT    1 xsk_umem__create_v0_0_4
     405: 0000000000017145  1190 FUNC    GLOBAL DEFAULT    1 xsk_umem__create
     499: 00000000000175eb   103 FUNC    GLOBAL DEFAULT    1 xsk_umem__create_v0_0_2
  -bash-4.4$
No versioned symbols for xsk_umem__create.
The libbpf.a can be used to build a shared library succesfully.
  -bash-4.4$ cat t.c
  extern void *xsk_umem__create;
  void * test() { return xsk_umem__create; }
  -bash-4.4$ gcc -c -fPIC t.c
  -bash-4.4$ gcc -shared t.o libbpf.a -o t.so
  -bash-4.4$

Fixes: 10d30e301732 ("libbpf: add flags to umem config")
Cc: Kevin Laatz <kevin.laatz@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-09-30 16:10:45 -07:00
Andrii Nakryiko
ede18f80d8 scripts: fix empty cherry-pick handling, fix IGNORE_CONSISTENCY check
Fix two issues I've encountered during latest bpf/bpf-next sync.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-09-30 15:02:52 -07:00
Andrii Nakryiko
07cd489681 libbpf: fix Github-only indentation issue
When appying urgent fix to Github mirror, before it was pushed to linux
repo, there were some indentation issues, which eventually got fixed
upstream, but are still in Github mirror. Fix it to prevent future merge
conflicts.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
2019-09-30 15:02:43 -07:00
Andrii Nakryiko
d2f307c7f6 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   0bb52b0dfc88a155688f492aba8e686147600278
Checkpoint bpf-next commit: b41dae061bbd722b9d7fa828f35d22035b218e18
Baseline bpf commit:        2c238177bd7f4b14bdf7447cc1cd9bb791f147e6
Checkpoint bpf commit:      e3439af4a339acd7fddbd6d59b8ecefaac07a611

Alexei Starovoitov (1):
  tools/bpf: sync bpf.h

Andrii Nakryiko (2):
  libbpf: fix false uninitialized variable warning
  libbpf: Teach btf_dumper to emit stand-alone anonymous enum
    definitions

Daniel Borkmann (1):
  bpf: sync bpf.h to tools infrastructure

Kevin Laatz (1):
  libbpf: add flags to umem config

Toke Høiland-Jørgensen (1):
  libbpf: Remove getsockopt() check for XDP_OPTIONS

 include/uapi/linux/bpf.h    |  7 ++-
 include/uapi/linux/if_xdp.h |  9 ++++
 src/btf_dump.c              | 94 ++++++++++++++++++++++++++++++++++---
 src/libbpf.map              |  1 +
 src/xsk.c                   | 44 +++++++++++------
 src/xsk.h                   | 27 +++++++++++
 6 files changed, 160 insertions(+), 22 deletions(-)

--
2.17.1
2019-09-26 13:29:16 -07:00
Andrii Nakryiko
990cef2a0c libbpf: Teach btf_dumper to emit stand-alone anonymous enum definitions
BTF-to-C converter previously skipped anonymous enums in an assumption
that those are embedded in struct's field definitions. This is not
always the case and a lot of kernel constants are defined as part of
anonymous enums. This change fixes the logic by eagerly marking all
types as either referenced by any other type or not. This is enough to
distinguish two classes of anonymous enums and emit previously omitted
enum definitions.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20190925203745.3173184-1-andriin@fb.com
2019-09-26 13:29:16 -07:00
Andrii Nakryiko
4c2c521513 libbpf: fix false uninitialized variable warning
Some compilers emit warning for potential uninitialized next_id usage.
The code is correct, but control flow is too complicated for some
compilers to figure this out. Re-initialize next_id to satisfy
compiler.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-09-26 13:29:16 -07:00
Toke Høiland-Jørgensen
b1e911e9ba libbpf: Remove getsockopt() check for XDP_OPTIONS
The xsk_socket__create() function fails and returns an error if it cannot
get the XDP_OPTIONS through getsockopt(). However, support for XDP_OPTIONS
was not added until kernel 5.3, so this means that creating XSK sockets
always fails on older kernels.

Since the option is just used to set the zero-copy flag in the xsk struct,
and that flag is not really used for anything yet, just remove the
getsockopt() call until a proper use for it is introduced.

Suggested-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-09-26 13:29:16 -07:00
Kevin Laatz
ae673dc91f libbpf: add flags to umem config
This patch adds a 'flags' field to the umem_config and umem_reg structs.
This will allow for more options to be added for configuring umems.

The first use for the flags field is to add a flag for unaligned chunks
mode. These flags can either be user-provided or filled with a default.

Since we change the size of the xsk_umem_config struct, we need to version
the ABI. This patch includes the ABI versioning for xsk_umem__create. The
Makefile was also updated to handle multiple function versions in
check-abi.

Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Jonathan Lemon <jonathan.lemon@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-09-26 13:29:16 -07:00
Alexei Starovoitov
5a256d12bf tools/bpf: sync bpf.h
sync bpf.h from kernel/ to tools/

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-09-26 13:29:16 -07:00
Ondrej Mosnacek
ae8edc7624 libbpf: fix linker flags for shared library
The -lelf flag needs to be specified *after* the object files, otherwise
the output library produced by some compilers doesn't contain a link to
libelf.so:

(Example from Debian testing run on Travis.)
$ ldd libelf.so
	linux-vdso.so.1 (0x00007ffcbfda9000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f75f8d24000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f75f8f0f000)

Linking against such library then produces 'undefined reference to ...'
errors unless the target links against libelf as well. After this commit
the built library references the libelf library correctly:

$ ldd libbpf.so
	linux-vdso.so.1 (0x00007ffc821f1000)
	libelf.so.1 => /usr/lib/x86_64-linux-gnu/libelf.so.1 (0x00007f70ea3ec000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f70ea22c000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f70ea20f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f70ea433000)

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2019-09-26 00:41:27 -07:00
Ondrej Mosnacek
8f8b4a14fa Travis CI: add sanity check for libelf dependency
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2019-09-26 00:41:27 -07:00
Yonghong Song
476e158b07 sync: latest libbpf changes from kernel
Syncing latest libbpf commits from kernel repository.
Baseline bpf-next commit:   b753c5a7f99f390fc100de18647ce0dcacdceafc
Checkpoint bpf-next commit: 0bb52b0dfc88a155688f492aba8e686147600278
Baseline bpf commit:        91b4db5313a2c793aabc2143efb8ed0cf0fdd097
Checkpoint bpf commit:      2c238177bd7f4b14bdf7447cc1cd9bb791f147e6

Andrii Nakryiko (1):
  libbpf: make libbpf.map source of truth for libbpf version

Ivan Khoronzhuk (1):
  libbpf: use LFS (_FILE_OFFSET_BITS) instead of direct mmap2 syscall

Magnus Karlsson (1):
  libbpf: add support for need_wakeup flag in AF_XDP part

Peter Wu (1):
  bpf: sync bpf.h to tools/

Quentin Monnet (3):
  tools: bpf: synchronise BPF UAPI header with tools
  libbpf: refactor bpf_*_get_next_id() functions
  libbpf: add bpf_btf_get_next_id() to cycle through BTF objects

Stanislav Fomichev (1):
  bpf: sync bpf.h to tools/

 include/uapi/linux/bpf.h    | 12 ++++++---
 include/uapi/linux/if_xdp.h | 13 +++++++++
 src/bpf.c                   | 24 ++++++++---------
 src/bpf.h                   |  1 +
 src/libbpf.map              |  5 ++++
 src/xsk.c                   | 53 +++++++++++++------------------------
 src/xsk.h                   |  6 +++++
 7 files changed, 64 insertions(+), 50 deletions(-)

--
2.17.1
2019-08-27 13:58:15 -07:00
Peter Wu
13e1ee420e bpf: sync bpf.h to tools/
Fix a 'struct pt_reg' typo and clarify when bpf_trace_printk discards
lines. Affects documentation only.

Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-08-27 13:58:15 -07:00
Ivan Khoronzhuk
3e2bab6d2c libbpf: use LFS (_FILE_OFFSET_BITS) instead of direct mmap2 syscall
Drop __NR_mmap2 fork in flavor of LFS, that is _FILE_OFFSET_BITS=64
(glibc & bionic) / LARGEFILE64_SOURCE (for musl) decision. It allows
mmap() to use 64bit offset that is passed to mmap2 syscall. As result
pgoff is not truncated and no need to use direct access to mmap2 for
32 bits systems.

Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-27 13:58:15 -07:00
Quentin Monnet
9084f4cd4d libbpf: add bpf_btf_get_next_id() to cycle through BTF objects
Add an API function taking a BTF object id and providing the id of the
next BTF object in the kernel. This can be used to list all BTF objects
loaded on the system.

v2:
- Rebase on top of Andrii's changes regarding libbpf versioning.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-08-27 13:58:15 -07:00
Quentin Monnet
66d20edaf0 libbpf: refactor bpf_*_get_next_id() functions
In preparation for the introduction of a similar function for retrieving
the id of the next BTF object, consolidate the code from
bpf_prog_get_next_id() and bpf_map_get_next_id() in libbpf.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-08-27 13:58:15 -07:00
Quentin Monnet
d8d6772ab8 tools: bpf: synchronise BPF UAPI header with tools
Synchronise the bpf.h header under tools, to report the addition of the
new BPF_BTF_GET_NEXT_ID syscall command for bpf().

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-08-27 13:58:15 -07:00
Stanislav Fomichev
4397d09cd8 bpf: sync bpf.h to tools/
Sync new sk storage clone flag.

Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-27 13:58:15 -07:00