rdma-core's mlx5dv_devx_umem_reg_ex() passes the page sizes the caller
can use in MLX5_IB_ATTR_DEVX_UMEM_REG_PGSZ_BITMAP. Add the attribute and
choose the UMEM page size, page offset and MTT count the way Linux commit
7610ab57de56 ("RDMA/mlx5: Allow larger pages in DevX umem") does.
Linux looks at the DMA list only. The UMEM is mapped at an IOVA equal to
the DMA address of its first byte, ib_umem_find_best_pgsz() picks the
largest page size of the bitmap that this mapping allows, and
devx_umem_find_best_pgsize() halves it while the first DMA address is not
aligned to it or the length is not a multiple of it, stopping at
PAGE_SIZE, even if that is below every size in the bitmap.
The FreeBSD OFED layer has none of the ib_umem helpers this relies on,
so local copies of the current Linux ones are added.
ib_umem_find_best_pgoff() and ib_umem_dma_offset() come from Linux
commit b045db62f6f6 ("RDMA/mlx5: Use ib_umem_find_best_pgoff() for
SRQ"), ib_umem_num_dma_blocks() from Linux commit a665aca89a41
("RDMA/umem: Split ib_umem_num_pages() into ib_umem_num_dma_blocks()").
The ib_umem_find_best_pgsz() logic includes Linux commit 486055f5e09d
("RDMA/core: Fix best page size finding when it can cross SG entries"),
without which no page size above 4K would ever be found, as FreeBSD's
ib_umem_get() maps every page as an SG entry of its own, and Linux
commit 09ea6837a043 ("RDMA/umem: Be careful about boundary conditions in
ib_umem_find_best_pgsz()").
Without the attribute the bitmap defaults to GENMASK_ULL(63, 12), as in
Linux. The attribute is read with uverbs_copy_from(), FreeBSD's
uverbs_get_const_default() works on an s64 and rejects any value with bit
63 set, the default included. The page size is kept in a u64, Linux
returns it as an unsigned int and so fails registrations that would use
pages of 4G or more.
Reviewed by: kib
Sponsored by: Nvidia networking
MFC after: 1 month
[PATCH 25/31] FreeBSD OFED support for DPDK MLX5 PMD
Struct mlx5_ib_striding_rq_caps was not aligned to 64 bit as
it should have been. Add a 32 bit reserved field.
Differential revision: https://reviews.freebsd.org/D32200
MFC after: 1 month
Import Linux upstream commit 4e2b53a5cb5a ("IB/mlx5: Report inner RSS
capability").
Define MLX5_RX_HASH_INNER as (1UL << 31). Shifting 1 into the sign bit of
a plain int is undefined behaviour, and user space compiles this header
too. The main.c half of that commit is already carried by the preceding
fixup.
Sponsored by: NVidia networking
MFC after: 1 month
[PATCH 22/31] FreeBSD OFED support for DPDK MLX5 PMD
a) The device can support receive Stateless Offloads for the inner
packet's fields only when the packet is processed by TIR which is
enabled to support tunneling. Otherwise, the device treats the
packet as an ordinary non-tunneling packet and receive offloads
can be done only for the outer packet's field.
In order to enable receive Stateless Offloading support for incoming
tunneling traffic the TIR should be created with tunneled_offload_en.
Tunneling offloads is supported only be raw ethernet QP.
This patch includes:
- New QP creation flag for tunneling offloads.
- Reports device capabilities.
b) IB/mlx5: Add support for RSS on the inner packet
Some user space application would like to do RSS on the inner
packet fields instead on the outer.
When MLX5_RX_HASH_INNER is set with one or more of the other
hash fields, then the RSS will be done using the inner packet.
Differential revision: https://reviews.freebsd.org/D32196
MFC after: 1 month
[PATCH 21/31] FreeBSD OFED support for DPDK MLX5 PMD
a) This patch reports the device's striding RQ capabilities to
the user-space:
- min/max_single_stride_log_num_of_bytes: Log of min/max number of
bytes in a single stride.
- min/max_single_wqe_log_num_of_strides: Log of min/max number of
strides in a single WQE.
- supported_qpts: A bit mask to know which QP types support multi-
packet RQ, for now only Raw Packet QPs.
b) Allow creation of a multi-packet receive queue.
In order to create a multi-packet RQ, the following fields in
the mlx5_ib_rwq should be set:
- log_num_strides: Log of number of strides per WQE
- single_stride_log_num_of_bytes: Log of a single stride size
- two_byte_shift_en: When enabled, hardware pads 2 bytes of zeros
before writing the message to memory (e.g. for the IP alignment).
Differential revision: https://reviews.freebsd.org/D32195
MFC after: 1 month
[PATCH 17/31] FreeBSD OFED support for DPDK MLX5 PMD
Software parsing (SWP) is a feature that can be used to instruct the
device to stop using its internal parser and to parse packets on the
transmit path according to offsets set for each packets.
Through this feature, the device allows the handling of checksum and
LSO by the hardware according to the location of IP and TCP/UDP
headers.
Enable SW parsing on Raw Ethernet send queue by default if firmware
supports it and report these capabilities to user space.
Differential revision: https://reviews.freebsd.org/D32191
MFC after: 1 month
[PATCH 15/31] FreeBSD OFED support for DPDK MLX5 PMD
Enable mlx5 based hardware to report packet pacing capabilities
from kernel to user space. Packet pacing allows to limit the rate to any
number between the maximum and minimum, based on user settings.
The capabilities are exposed to user space through query_device by uhw.
The following capabilities are reported:
- The maximum and minimum rate limit in kbps supported by packet pacing.
- Bitmap showing which QP types are supported by packet pacing operation.
Differential revision: https://reviews.freebsd.org/D32189
MFC after: 1 month
[PATCH 14/31] FreeBSD OFED support for DPDK MLX5 PMD
a) Add support to receive specific Vxlan packet in ConnectX-4.
b) Add support to match packet fields which are tunneled,
i.e. support matching the header of the inner packet which is the result of
or bit operation of the original header and the IB_FLOW_SPEC_INNER type.
The combination of IB_FLOW_SPEC_INNER | IB_FLOW_SPEC_VXLAN_TUNNEL is not
needed to be checked, because the IB core has this check already.
Dfiferential revision: https://reviews.freebsd.org/D32188
MFC after: 1 month
PATCH 13/31] FreeBSD OFED support for DPDK MLX5 PMD
The capabilities whether hardware support multi packet WQE or not is
exposed to user space through query_device by uhw.
Differential revision: https://reviews.freebsd.org/D32187
MFC after: 1 month
[PATCH 06/31] FreeBSD OFED support for DPDK MLX5 PMD
CQE compressing reduces PCI overhead by coalescing and compressing
multiple CQEs into a single merged CQE. Successful compressing
improves message rate especially for small packet traffic.
CQE compressing is supported for all 64B CQE formats (with certain
limitations) generated by RQ/Responder or by SQ/Requestor.
IB/mlx5: Report mlx5 CQE compression capabilities when querying device
The capabilities include:
- Max number of compressed and aggregated CQEs in a single session,
while zero means unsupported.
- For Responder, there are two formats of mini CQE: mini CQE with Rx
hash and mini CQE with checksum. They're mutual exclusive.
Differential revision: https://reviews.freebsd.org/D32180
MFC after: 1 month
This is backward ABI-compatible, because the only place in kernel that
uses the structure, namely the mlx5_ib_cq.c:mlx5_ib_create_cq()
function, copies in as much structure members as provided by userspace.
Tested by: Wafa Hamzah <[email protected]>
Sponsored by: Nvidia networking
MFC after: 1 month
Calculate and set UDP source port based on the flow label. If flow label
is not defined in GRH then calculate it based on lqpn/rqpn.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Obtained from: Linux commit 18451db82ef7f943c60a7fce685f16172bda5106
Sponsored by: Chelsio Communications
The main point of these changes is the last commit adding a couple of
helper functions used in ROCE drivers.
This contains changes from the following Linux commits:
a70c07397fd8 RDMA: Introduce and use GID attr helper to read RoCE L2 fields
8f9748602491 IB/cm: Reduce dependency on gid attribute ndev check
adb4a57a7a1d RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev
b4fb4cc5ba83 RDMA/cma: Fix unbalanced cm_id reference count during address resolve
d5665a21250e RDMA/core: Add hash functions to calculate RoCEv2 flowlabel and UDP source port
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
627212c9d49b RDMA/core: Replace open-coded variant of get_device
adee9f3f3bbb RDMA/core: Depend on device_add() to add device attributes
954a8e3aea87 RDMA/cma: Protect cma dev list with lock
722c7b2bfead RDMA/{cma, core}: Avoid callback on rdma_addr_cancel()
f9d08f1e1939 RDMA/core: Rate limit MAD error messages
e1f540c3ed0e RDMA/core: Define client_data_lock as rwlock instead of spinlock
2d65f49ff961 RDMA/core: Use simpler spin lock irq API from blocking context
4512acd0d34c RDMA/core: Remove context entries from list while unregistering device
f7b65d9bf2db RDMA/core: Use simplified list_for_each
93688ddbe1da RDMA/core: No need to protect kfree with spin lock and semaphore
50704e039ab1 RDMA/umem: Restore lockdep check while downgrading lock
eb93c82ed8c7 RDMA/core: Document QP @event_handler function
4269024639f6 RDMA/core: Document CM @event_handler function
77addc524473 RDMA/core: Rename rdma_copy_addr to rdma_copy_src_l2_addr
a362ea1d9e1a RDMA/core: Introduce and use rdma_set_src_addr() between IPv4 and IPv6
89c5691cdd95 RDMA/core: Let protocol specific function typecast sockaddr structure
caf1e3ae9fa6 RDMA/core Introduce and use rdma_find_ndev_for_src_ip_rcu
c31d4b2ddf07 RDMA/core: Protect against changing dst->dev during destination resolve
0e9d2c19bff1 RDMA/core: Consider net ns of gid attribute for RoCE
d6b1764a8c5a RDMA/core: Introduce rdma_read_gid_attr_ndev_rcu() to check GID attribute
6aaecd385685 RDMA/core: Simplify roce_resolve_route_from_path()
0965cc953a23 RDMA/core: Properly return the error code of rdma_set_src_addr_rcu
d4b4dd1b9706 RDMA/umem: Do not use current->tgid to track the mm_struct
c9990ab39b6e RDMA/umem: Move all the ODP related stuff out of ucontext and into per_mm
597ecc5a0954 RDMA/umem: Get rid of struct ib_umem.odp_data
41b4deeaa123 RDMA/umem: Make ib_umem_odp into a sub structure of ib_umem
b5231b019d76 RDMA/umem: Use ib_umem_odp in all function signatures connected to ODP
56ac9dd9177c RDMA/umem: Avoid synchronize_srcu in the ODP MR destruction path
be7a57b41ad8 RDMA/umem: Handle a half-complete start/end sequence
ca748c39ea3f RDMA/umem: Get rid of per_mm->notifier_count
f27a0d50a4bc RDMA/umem: Use umem->owning_mm inside ODP
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
Many of these changes deal with shifting from fetcing SGID attributes
via an index in drivers to including references to SGID attributes in
other structures passed to driver callbacks which can be used directly
avoiding the need for fetching attributes via lookup functions. I had
to fixup the ROCE GID management code directly as the version in
FreeBSD is very different from Linux.
I also tweaked the change from commit
475c8de7bf to use a goto to reduce diffs
with upstream to minimize conflicts when merging upstream changes.
This contains changes from the following Linux commits:
a1a4caeebac9 IB/core: Do not set the gid type when reserving default entries
1c36cf912ad1 IB/core: Store default GID property per-table instead of per-entry
1dfce2945771 IB: Replace ib_query_gid/ib_get_cached_gid with rdma_query_gid
83f6f8d29dd3 IB/core: Make rdma_find_gid_by_filter support all protocols
c3d71b69a75c IB/core: Provide rdma_ versions of the gid cache API
77e786fcbe2e IB/core: Replace ib_query_gid with rdma_get_gid_attr
bf399c2cadfa IB/core: Introduce GID attribute get, put and hold APIs
f4df9a7c34d8 RDMA: Use GID from the ib_gid_attr during the add_gid() callback
b150c3862d21 IB/core: Introduce GID entry reference counts
47ec38666210 RDMA: Convert drivers to use sgid_attr instead of sgid_index
d97099fe53ed IB{cm, core}: Introduce and use ah_attr copy, move, replace APIs
947c99ecfcb4 IB/core: Tidy ib_resolve_eth_dmac
8d9ec9addd6c IB/core: Add a sgid_attr pointer to struct rdma_ah_attr
89af969a6653 RDMA: Convert drivers to use the AH's sgid_attr in post_wr paths
1a1f460ff151 RDMA: Hold the sgid_attr inside the struct ib_ah/qp
7492052a186b IB/mlx4: Use GID attribute from ah attribute
59d40813328f IB/core: Free GID table entry during GID deletion
881456789248 RDMA/cma: Consider net namespace while leaving multicast group
4ed13a5f2d60 IB/cm: Keep track of the sgid_attr that created the cm id
aa74f4878d61 IB: Make init_ah_attr_grh_fields set sgid_attr
f685c19529f0 IB: Make ib_init_ah_from_mcmember set sgid_attr
b7403217656d IB: Make ib_init_ah_attr_from_wc set sgid_attr
ea8c2d8f6014 RDMA/core: Remove unused ib cache functions
a8872d53e9b7 IB/cm: Use sgid_attr from the AV
398391071f25 IB/cm: Replace members of sa_path_rec with 'struct sgid_attr *'
815d456ef21a IB/cm: Pass the sgid_attr through various events
4eefd62c17a9 include/rdma/opa_addr.h: Fix an endianness issue
f8c2d2280cf6 RDMA/core: Remove set-but-not-used variables
28e39894ed4f RDMA/core: Remove ib_find_cached_gid() and ib_find_cached_gid_by_port()
07e7056aff6c IB/core: Simplify check for RoCE route resolve
40ddacf2dda9 RDMA/umem: Don't hold mmap_sem for too long
1215cb7c88ec RDMA/umem: Refactor exit paths in ib_umem_get
c012691508f5 IB/cm: Remove cma_multicast->igmp_joined
4b4671a0f269 IB/IPoIB: Simplify ib_post_(send|recv|srq_recv)() calls
1fec77bf8fcd RDMA/core: Simplify ib_post_(send|recv|srq_recv)() calls
bb039a870c05 IB/core: Allow ULPs to specify NULL as the third ib_post_(send|recv|srq_recv)() argument
643d213a9a03 RDMA/cma: Do not ignore net namespace for unbound cm_id
d274e45ce1ed RDMA/cma: Consider netdevice for RoCE ports
cee104334c98 IB/core: Introduce and use sgid_attr in CM requests
7150c3d5544b RDMA/core: Remove {create,destroy}_ah from mandatory verbs
854633165164 RDMA/core: Prefix _ib to IB/RoCE specific functions
7582df826734 RDMA/core: Avoid holding lock while initializing fields on stack
79d684f02647 RDMA/core: Simplify gid type check in cma_acquire_dev()
ca3a8ace2b12 RDMA/core: Return bool instead of int
05e0b86c413d RDMA/cma: Get rid of 1 bit boolean
e7ff98aefc9e RDMA/cma: Constify path record, ib_cm_event, listen_id pointers
2df7dba855e1 RDMA/core: Constify dst_addr argument
219d2e9dfda9 RDMA/cma: Simplify rdma_resolve_addr() error flow
26e551c5aec5 RDMA: Fix return code check in rdma_set_cq_moderation
58796e67d5d5 IB/ucm: Initialize sgid request GID attribute pointer
dd81b2c8a333 IB/core: Change filter function return type from int to bool
93065ac753e4 mm, oom: distinguish blockable mode for mmu notifiers
0d23ba6034b9 RDMA/ucma: check fd type in ucma_migrate_id()
Part of cee104334c98 was previously merged into FreeBSD. Namely, SGID
index fields had been added to the two param structures instead and
roce_get_net_dev_by_cm_event was implemented using SGID indices.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
e822ff213fe6 IB/cm: Store and restore ah_attr during CM message processing
0e225dcb7681 IB/cm: Store and restore ah_attr during LAP msg processing
a5c57d327272 IB/cm: Avoid AV ah_attr overwriting during LAP message handling
724631a9c6e9 IB/core: Introduce and use rdma_gid_table()
25e62655c793 IB/core: Reduce the places that use zgid
08bb558ac11a IB/core: Make testing MR flags for writability a static inline function
Commit 0e225dcb7681 was partially merged previously as
cm_init_av_for_lap() already existed in FreeBSD matching the version
from the Linux commit.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
a3b641af72ba RDMA/CM: move rdma_id_private to cma_priv.h
6612b4983f7e IB/core: Fix comments of GID query functions
b26c4a1138df IB/{core, ipoib}: Simplify ib_find_gid() for unused ndev
5ac08a341303 IB/cma: Use rdma_protocol_roce() and remove cma_protocol_roce_dev_port()
a9c06aeba997 IB/core: Remove rdma_resolve_ip_route() as exported symbol
a22af59ea9a5 IB/cm: Add and use a helper function to add cm_id's to the port list
cb12a8e2fa5f IB/cm: Introduce and use helper function to get cm_port from path
0a5141593567 IB/core: Refactor ib_init_ah_attr_from_path() for RoCE
e41a7c41947d IB/core: Move rdma_addr_find_l2_eth_by_grh to core_priv.h
98f1f4e0ed26 IB/core: Refer to RoCE port property instead of GID table property
114cc9c4b182 IB/cma: Resolve route only while receiving CM requests
9137108cc3d6 RDMA/rdma_cm: Fix use after free race with process_one_req
190fb9c4d130 IB/core: Refer to RoCE port property to decide building cache
22d24f75a193 IB/core: Search GID only for IB link layer
4ab7cb4bf362 IB/core: Refer to RoCE port property instead of GID table property
3401857ea347 IB/core: Generate GID change event regardless of RoCE GID table property
218b9e3eb8b5 RDMA/cma: Move rdma_cm_state to cma_priv.h
8435168d50e6 RDMA/ucma: Don't allow setting RDMA_OPTION_IB_PATH without an RDMA device
72e1ff0fb7e0 RDMA/core: Update query_gid documentation for HCA drivers
0e1f9b924471 RDMA/providers: Simplify query_gid callback of RoCE providers
598ff6bae689 IB/core: Refactor GID modify code for RoCE
f35faa4ba956 IB/core: Simplify ib_query_gid to always refer to cache
14169e333e71 IB/providers: Avoid zero GID check for RoCE
414448d249d8 RDMA: Use ib_gid_attr during GID modification
ee6548d1d98d RDMA/rdma_cm: Delete rdma_addr_client
44e75052bc2a RDMA/rdma_cm: Make rdma_addr_cancel into a fence
e19c0d237873 RDMA/rdma_cm: Remove process_req and timer sorting
a66ed149b0da IB/core: Don't allow default GID addition at non reseved slots
dc5640f294e4 IB/core: Fix deleting default GIDs when changing mac adddress
22c01ee4b8a8 IB/core: Fix to avoid deleting IPv6 look alike default GIDs
9aa169213d11 RDMA/cma: Do not query GID during QP state transition to RTR
aec05afe641b IB/core: Remove redundant return
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
c5c4e40e90b5 IB/CM: Change sgid to IB GID when handling CM request
54a6d63f14bd IB/mlx4: Potential buffer overflow in _mlx4_set_path()
56d0a7d9a0f0 IB/core: Depend on IPv6 stack to resolve link local address for RoCEv2
1060f8653414 IB/{core/cm}: Fix generating a return AH for RoCEE
981b5a2384b7 RDMA/cma: Introduce and use helper functions to init work
c42388053758 RDMA/cma: Avoid setting path record type twice
4367ec7fe2dd RDMA/cma: Simplify netdev check
151ed9d70097 IB/core: Refactor to avoid unnecessary check on GID lookup miss
b0dd0d335364 IB/core: Avoid unnecessary type cast
86937fcd6ed2 RDMA/core: Avoid redundant memcpy in rdma_addr_find_l2_eth_by_grh
1c43d5d308f0 IB/core: Avoid exporting module internal ib_find_gid_by_filter()
dbb12562f7c2 IB/{core, ipoib}: Simplify ib_find_gid to search only for IB link layer
f6bdb14267ba IB/{core, umad, cm}: Rename ib_init_ah_from_wc to ib_init_ah_attr_from_wc
4ad6a0245ec8 IB/{core, cm, cma, ipoib}: Rename ib_init_ah_from_path to ib_init_ah_attr_from_path
33f93e1ebcf5 IB/cm: Fix sleeping while spin lock is held
16c72e402867 IB/cm: Refactor to avoid setting path record software only fields
af808ece5ce9 IB/SA: Check dlid before SA agent queries for ClassPortInfo
8d20a1f0ecd5 RDMA/cma: Fix rdma_cm raw IB path setting for RoCE
fe75889f2794 RDMA/{cma, ucma}: Simplify and rename rdma_set_ib_paths
9327c7afdce3 RDMA/cma: Provide a function to set RoCE path record L2 parameters
89838118a515 RDMA/cma: Fix rdma_cm path querying for RoCE
cd2a6e7d384b IB/core: Fix ib_wc structure size to remain in 64 bytes boundary
f2290d6d522d IB/core: Attempt DMAC resolution for only RoCE
a6532e713966 RDMA/core: Clarify rdma_ah_find_type
246d8b184c10 IB/cq: Don't force IB_POLL_DIRECT poll context for ib_process_cq_direct
aaebd377c0e9 IB/core: postpone WR initialization during queue drain
15cbc5197693 RDMA/core: Simplify rdma_addr_get_sgid() to not support RoCE
7a2f64ee4a95 RDMA/ucma: Use rdma cm API to query GID
411460ac50b0 RDMA/cma: Introduce API to read GIDs for multiple transports
2493a57bc10f RDMA/cma: Refactor to access multiple fields of rdma_dev_addr
00db63c128dd RDMA/cma: Check existence of netdevice during port validation
925f7ea7a6ba RDMA/cm: Fix access to uninitialized variable
87daac68f77a IB/core: Map iWarp AH type to undefined in rdma_ah_find_type
87915bf82eea RDMA/verbs: Return proper error code for not supported system call
d3b9e8ad425c RDMA/core: Reduce poll batch for direct cq polling
Parts of 1060f8653414 were already merged into FreeBSD. Cosmetic
changes made in rdma_addr_find_l2_eth_by_grh while merging this change
were just to reduce the diff with Linux to ease merging future
changes.
The merge of 56d0a7d9a0f0 differs slightly from the Linux commit as
FreeBSD was missing the original changes to check for link-local
addresses in ib_resolve_eth_dmac.
The merge of 2493a57bc10f is mostly a cosmetic change to reduce diffs
with upstream as local changes in FreeBSD passed the dev_addr as the
last argument instead of the dev_type and if_index already.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
c0b64f58e8d4 RDMA/cma: Avoid triggering undefined behavior
c0348eb06968 IB: Let ib_core resolve destination mac address
5a3dc3237243 IB/cm: Fix memory corruption in handling CM request
fec99ededf6b RDMA/umem: Avoid partial declaration of non-static function
19b57c6c4499 IB/core: Convert OPA AH to IB for Extended LIDs only
e08ce2e82b2f RDMA/core: Make function rdma_copy_addr return void
4190b4e96954 RDMA/core: Rename kernel modify_cq to better describe its usage
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
1cb2fc0db764 IB/mad: Change slid in RMPP recv from 16 to 32 bits
7e93e2cb835c IB/IPoIB: Increase local_lid to 32 bits
d541e45500bd IB/core: Convert ah_attr from OPA to IB when copying to user
6b3c0e6e6d5a IB/CM: Create appropriate path records when handling CM request
e92aa00a5189 IB/CM: Add OPA Path record support to CM
7db20ecd1d97 IB/core: Change wc.slid from 16 to 32 bits
db58540b021a IB/core: Change port_attr.sm_lid from 16 to 32 bits
582faf3150f5 IB/core: Change port_attr.lid size from 16 to 32 bits
ac3a949fb2ff IB/CM: Set appropriate slid and dlid when handling CM request
06f8174a9782 IB/core: Protect sysfs entry on ib_unregister_device
62ede7779904 Add OPA extended LID support
13c19222889d IB/rdmavt, hfi1, qib: Modify check_ah() to account for extended LIDs
d98bb7f7e6fa IB/hfi1: Determine 9B/16B L2 header type based on Address handle
dcc9881e6767 RDMA/(core, ulp): Convert register/unregister event handler to be void
f808c13fd373 lib/interval_tree: fast overlap detection
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
Most of the changes involve renaming InfiBand address handles (AH) to
RDMA address handles including renaming various types and functions.
Address handles now also include type-specific fields and a set of
wrapper accessor functions for getting and setting fields. For some
of the type and function renames, drivers in sys/dev had to be updated
directly (typically via sed).
This contains changes from the following Linux commits:
f039f44fc331 IB/core: Add support for draining IB_POLL_DIRECT completion queues
fedd9e1f7582 IB/cq: Don't process more than the given budget
403cd12e2cf7 IB/umem: Add contiguous ODP support
0008b84ea9af IB/umem: Add support to huge ODP
3e7e1193e28a IB: Replace ib_umem page_size by page_shift
cb8637660ae8 IB/SA: Move functions update_sm_ah() and ib_sa_event()
680562b56939 IB/SA: Remove unwanted braces
dbb6c91fd8e0 IB/SA: Add braces when using sizeof
f96a31871449 IB/SA: Fix lines longer than 80 columns
94d595c56077 IB/core: Add rdma_cap_opa_ah to expose opa address handles
ee1c60b1bff8 IB/SA: Modify SA to implicitly cache Class Port info
aa4656d9a429 IB/core: Move opa_class_port_info definition to header file
2196f2716292 IB/SA: Add support to query opa classport info.
64b4646eaf3d IB/core: Define 'opa' rdma_ah_attr type
44c58487d51a IB/core: Define 'ib' and 'roce' rdma_ah_attr types
d8966fcd4c25 IB/core: Use rdma_ah_attr accessor functions
2224c47ace23 IB/core: Add accessor functions for rdma_ah_attr fields
365231593409 IB/core: Rename ib_destroy_ah to rdma_destroy_ah
bfbfd661c9ea IB/core: Rename ib_query_ah to rdma_query_ah
67b985b6c755 IB/core: Rename ib_modify_ah to rdma_modify_ah
0a18cfe4f6d7 IB/core: Rename ib_create_ah to rdma_create_ah
766b7f6cf0c0 IB/mthca: Rename to_ib_ah_attr to to_rdma_ah_attr
38349389fece IB/mlx5: Rename to_ib_ah_attr to to_rdma_ah_attr
71d53ab47ee6 IB/mlx4: Rename to_ib_ah_attr to to_rdma_ah_attr
90898850ec4e IB/core: Rename struct ib_ah_attr to rdma_ah_attr
4ba66093bdc6 IB/core: Check for global flag when using ah_attr
cf0b9395d084 IB/core: Add braces when using sizeof
cfd519358f50 IB/IPoIB: Remove 'else' when the 'if' has a return.
82ffc226483c IB/CM: Add braces when using sizeof
dfa834e1d97e IB/SA: Introduce path record specific types
c2f8fc4ec440 IB/SA: Rename ib_sa_path_rec to sa_path_rec
9fdca4da4d8c IB/SA: Split struct sa_path_rec based on IB and ROCE specific fields
4c33bd1926cc IB/SA: Add support to query OPA path records
57520751445b IB/SA: Add OPA path record type
87f0faadc685 IB/SA: Add OPA addr header
eed7624552ca RDMA/mlx4: Fix MAD tunneling when SRIOV is enabled
f937d93a9122 RDMA/uverbs: Declare local function static and add brackets to sizeof
d3957b86a406 RDMA/SA: Fix kernel panic in CMA request handler flow
The IPoIB portions of ee1c60b1bff8 were already present in FreeBSD
from an earlier sync with OFED. Likewise, the ib_uverbs_cmd.c
changes from 4ba66093bdc6 were also already present.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
55efcfcd7776 RDMA/core: Fix incorrect structure packing for booleans
aaaca121c7cf RDMA/core: add port state cache
9e2c3f1c7f3e RDMA/core: export ib_get_cached_port_state
93b1f29de71f RDMA/cma: resolve to first active ib port
d43dbacfc063 IB/core: Change the type of an ib_dma_alloc_coherent() argument
6532c380bf40 IB/core: Remove ib_dma_*map_single_attrs()
0bbb3b7496ea IB/rxe, IB/rdmavt: Use dma_virt_ops instead of duplicating it
f57e8ca50e23 IB/mad: Add port_num to error message
828f6fa65ce7 IB/umem: Release pid in error and ODP flow
19b752a19dce IB/cma: Allow port reuse for rdma_id
21d6454a392d RDMA/core: create struct ib_port_cache
25bf14d6f589 IB/core: Add implicit MR flag
d9d0674c0f8a IB/umem: Indicate that process is being terminated
d07d1d70ce1a IB/umem: Update on demand page (ODP) support
a748d60df32e IB/mlx4: Take source GID by index from HW GID table
89052d784bc9 IB/cma: Add default RoCE TOS to CMA configfs
6df6b4a9ce43 IB/cma: Destination and source addr families must match
On FreeBSD, the default RoCE TOS setting is available as per-port
sysctls similar to the existing default RoCE mode sysctls.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This contains changes from the following Linux commits:
850d8fd76507 IB/mlx4: Handle IPv4 header when demultiplexing MAD
a0b3455fcb2d IB/core: Remove debug prints after allocation failure
74226649f42d IB/ipoib: Remove and fix debug prints after allocation failure
870b28524552 IB/mthca: Remove debug prints after allocation failure
aa6aae38f7fb IB/core: Release allocated memory in cache setup failure
f73a1dbc45a5 infiniband: remove WARN that is not kernel bug
5f24410408fd rdma_cm: add rdma_consumer_reject_data helper function
5042a73d3e9d rdma_cm: add rdma_is_consumer_reject() helper function
d3f4aadd614c RDMA/core: Add the function ib_mtu_int_to_enum
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Sponsored by: Chelsio Communications
This is largely pulled from the original Linux commit to add cq.c.
Note that irq_poll is still not supported, but polling should now be
possible whereas it wasn't really before.
Tested by: Wafa Hamzah <[email protected]> (mlx5_ib)
Tested by: John Baldwin <[email protected]> (iw_cxgbe)
Obtained from: Linux commit 14d3a3b2498edadec344cb11e60e66091f5daf63
Sponsored by: Chelsio Communications
According to the IB spec active_speed size should be u16 and not u8 as
before. Changing it to allow further extensions in offered speeds.
Linux commit:
376ceb31ff87 RDMA: Fix link active_speed size
Reviewed by: kib
Tested by: Wafa Hamzah <[email protected]>
MFC after: 1 week
Sponsored by: NVIDIA Networking
Differential revision: https://reviews.freebsd.org/D57084
ib_verbs.h still uses struct dma_attrs *dma_attrs everywhere.
It is beyond my knowledge when that struct got deprecated upstream but
it is still supported by our LinuxKPI. The problem is that the
functions called with that argument (dma_map_single_attrs,
dma_unmap_single_attrs, dma_map_sg_attrs, dma_unmap_sg_attrs) so far
are #defines in LinuxKPI and drop the last argument (attrs) so it was
never a problem.
In preparation to pass the attrs to the actual implementation in LinuxKPI,
which has gained support for them, we now pass dma_sttrs->flags which
is the expected unsigned long bit field.
If anyone has serious interest in updating our ofed implementation they
could look into this some more and remove the usage of struct dma_attrs
entirely.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D55390
This enum also exists as enum ibv_raw_packet_caps in libibverbs/verbs.h.
[khng: cherry-picked from Linux
ebaaee253ad3a3c573ab7d3d77e849056bdfa9ea]
Sponsored by: Juniper Networks, Inc.
MFC after: 7 days
Reviewed by: kib, zlei
Differential Revision: https://reviews.freebsd.org/D42177
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
This is counterpart to e87c494015, which did the same for ethernet.
Suggested by: hselasky
Reviewed by: hselasky, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D39405
Summary:
Because of the intricacies of this code it wasn't purely scripted, but
instead hand-mechanical.
Reviewed by: hselasky
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38560
Backport from Linux 5.17 (drivers/infiniband/hw/mlx5/fs.c)
This fixes creating flow rules from user-space after the
kernel space update based on Linux 5.7-rc1 .
Sponsored by: NVIDIA Networking
Overview:
This is the first stage of a RDMA stack upgrade introducing kernel
changes only based on Linux 5.7-rc1.
This patch is based on about four main areas of work:
- Update of the IB uobjects system:
- The memory holding so-called AH, CQ, PD, SRQ and UCONTEXT objects
is now managed by ibcore. This also require some changes in the
kernel verbs API. The updated verbs changes are typically about
initialize and deinitialize objects, and remove allocation and
free of memory.
- Update of the uverbs IOCTL framework:
- The parsing and handling of user-space commands has been
completely refactored to integrate with the updated IB uobjects
system.
- Various changes and updates to the generic uverbs interfaces in
device drivers including the new uAPI surface.
- The mlx5_ib_devx.c in mlx5ib and related mlx5 core changes.
Dependencies:
- The mlx4ib driver code has been updated with the minimum changes
needed.
- The mlx5ib driver code has been updated with the minimum changes
needed including DV support.
Compatibility:
- All user-space facing APIs are backwards compatible after this
change.
- All kernel-space facing RDMA APIs are backwards compatible after
this change, with exception of ib_create_ah() and ib_destroy_ah()
which takes a new flag.
- The "ib_device_ops" structure exist, but only contains the driver ID
and some structure sizes.
Differences from Linux:
- Infiniband drivers must use the INIT_IB_DEVICE_OPS() macro to set
the sizes needed for allocating various IB objects, when adding
IB device instances.
Security:
- PRIV_NET_RAW is needed to use raw ethernet transmit features.
- PRIV_DRIVER is needed to use other privileged operations.
Based on upstream Linux, Torvalds (5.7-rc1):
8632e9b5645bbc2331d21d892b0d6961c1a08429
MFC after: 1 week
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D31149
Sponsored by: NVIDIA Networking
Since neither ib_post_send() nor ib_post_recv() modify the data structure
their second argument points at, declare that argument const. This change
makes it necessary to declare the 'bad_wr' argument const too and also to
modify all ULPs that call ib_post_send(), ib_post_recv() or
ib_post_srq_recv(). This patch does not change any functionality but makes
it possible for the compiler to verify whether the
ib_post_(send|recv|srq_recv) really do not modify the posted work request.
Linux commit:
f696bf6d64b195b83ca1bdb7cd33c999c9dcf514
7bb1fafc2f163ad03a2007295bb2f57cfdbfb630
d34ac5cd3a73aacd11009c4fc3ba15d7ea62c411
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
Expose ib_ucontext from a given ib_uverbs_file. Drivers that use the ioctl(9)
API may have the ib_uverbs_file and need a way to get the related ib_ucontext
from it, this is enabled by this patch.
Downstream patches from this series will use it.
Linux commit:
7dc08dcfc8c86cb4457e383734ff6844ddaff876
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
All callers to ib_modify_qp_is_ok() provides enum ib_qp_state makes the
checks of out-of-scope redundant. Let's remove them together with updating
function signature to return boolean result.
While at it remove unused "ll" parameter from ib_modify_qp_is_ok().
Linux commit:
19b1f54099b6ee334acbfbcfbdffd1d1f057216d
d31131bba5a1630304c55ea775c48cc84912ab59
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
Add new member rate_limit to ib_qp_attr which holds the packet pacing
rate in kbps, 0 means unlimited.
IB_QP_RATE_LIMIT is added to ib_attr_mask and could be used by RAW
QPs when changing QP state from RTR to RTS, RTS to RTS.
Linux commit:
528e5a1bd3f0e9b760cb3a1062fce7513712a15d
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
Define new option in 'rdma_set_option' to override calculated QP timeout
when requested to provide QP attributes to modify a QP.
At the same time, pack tos_set to be bitfield.
Linux commit:
2c1619edef61a03cb516efaa81750784c3071d10
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
Currently access to hardware stats buffer isn't protected, this can
result in multiple writes and reads at the same time to the same
memory location. This can lead to providing an incorrect value to
the user. Add a mutex to protect against it.
Linux commit:
e945130b52bea65d15f9bdf54949d4cb7a88db7f
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
In order to improve readability, add ib_port_phys_state enum to replace
the use of magic numbers.
Linux commit:
72a7720fca37fec0daf295923f17ac5d88a613e1
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
rdma_reject_msg() returns a pointer to a string message associated with
the transport reject reason codes.
Linux commit:
77a5db13153906a7e00740b10b2730e53385c5a8
MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking
Given all the code does operate on struct ifnet, the last step in this
longer series of changes now is to rename struct net_device to
struct ifnet (that is what it was defined to in the LinuxKPi code).
While mlx4 and OFED are "shared" code the decision was made years ago
to not write it based on the netdevice KPI but the native ifnet KPI
for most of it. This commit simply spells this out and with that
frees "struct netdevice" to be re-done on LinuxKPI to become a more
native/mixed implementation over time as needed by, e.g., wireless
drivers.
Sponsored by: The FreeBSD Foundation
MFC after: 10 days
Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D30515
Ariel Ehrenberg