xprtrdma: Don't provide a reply chunk when expecting a short reply

Currently Linux always offers a reply chunk, even when the reply
can be sent inline (ie. is smaller than 1KB).

On the client, registering a memory region can be expensive. A
server may choose not to use the reply chunk, wasting the cost of
the registration.

This is a change only for RPC replies smaller than 1KB which the
server constructs in the RPC reply send buffer. Because the elements
of the reply must be XDR encoded, a copy-free data transfer has no
benefit in this case.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
Tested-by: Devesh Sharma <devesh.sharma@avagotech.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
This commit is contained in:
Chuck Lever 2015-08-03 13:04:08 -04:00 committed by Anna Schumaker
parent 02eb57d8f4
commit 33943b2974

View File

@ -420,7 +420,7 @@ rpcrdma_marshal_req(struct rpc_rqst *rqst)
* *
* o Read ops return data as write chunk(s), header as inline. * o Read ops return data as write chunk(s), header as inline.
* o If the expected result is under the inline threshold, all ops * o If the expected result is under the inline threshold, all ops
* return as inline (but see later). * return as inline.
* o Large non-read ops return as a single reply chunk. * o Large non-read ops return as a single reply chunk.
*/ */
if (rqst->rq_rcv_buf.flags & XDRBUF_READ) if (rqst->rq_rcv_buf.flags & XDRBUF_READ)
@ -476,17 +476,6 @@ rpcrdma_marshal_req(struct rpc_rqst *rqst)
headerp->rm_body.rm_nochunks.rm_empty[2] = xdr_zero; headerp->rm_body.rm_nochunks.rm_empty[2] = xdr_zero;
/* new length after pullup */ /* new length after pullup */
rpclen = rqst->rq_svec[0].iov_len; rpclen = rqst->rq_svec[0].iov_len;
/* Currently we try to not actually use read inline.
* Reply chunks have the desirable property that
* they land, packed, directly in the target buffers
* without headers, so they require no fixup. The
* additional RDMA Write op sends the same amount
* of data, streams on-the-wire and adds no overhead
* on receive. Therefore, we request a reply chunk
* for non-writes wherever feasible and efficient.
*/
if (wtype == rpcrdma_noch)
wtype = rpcrdma_replych;
} }
if (rtype != rpcrdma_noch) { if (rtype != rpcrdma_noch) {