camcontrol: Support NVMe reservations via the persist subcommand

The NVMe reservation model is closely patterned after SCSI Persistent
Reservations, so support NVMe devices through the existing persist
subcommand rather than adding a transport-specific subcommand.  The
persist subcommand now dispatches on the device protocol, in the same
way the identify subcommand does.

The SCSI syntax is unchanged.  For NVMe devices, the -i modes all map
to the Reservation Report command (read_full_status requests the
extended data structure, which contains 128-bit Host Identifiers), and
the -o service actions map to the corresponding NVMe reservation
operations: register, register_ignore, reserve, preempt,
preempt_abort, release and clear, plus two NVMe-only actions,
unregister and replace.  The reservation type names for -T are shared
between SCSI and NVMe.  A new -c option gives explicit control over
the NVMe Persist Through Power Loss State; -p is equivalent to
-c enable.  SCSI-only options return an error for NVMe devices.

Tested against a PASCARI X200 series SSD (reservation capabilities:
WR_EX, PTPL, IEKEY13): register, reserve, release, replace,
unregister, -c enable/disable (Persist Through Power Loss State change
verified in the report output), reservation report, and the error
paths.  register_ignore is rejected by drives that implement NVMe 1.3
IEKEY semantics, as expected.  SCSI persist against a da(4) device is
unchanged.

sbin/camcontrol/nvres.c:
  New file, the NVMe reservation backend for the persist
  subcommand.

sbin/camcontrol/camcontrol.c:
  Dispatch the persist subcommand to scsipersist() or
  nvmepersist() based on the device protocol.

sbin/camcontrol/camcontrol.h:
  Add the nvmepersist() prototype.

sbin/camcontrol/Makefile:
  Add nvres.c to the build.

sbin/camcontrol/camcontrol.8:
  Document NVMe support in the persist subcommand section, and
  note the differences from SCSI where they occur.

Co-authored-by:	Reid Linnemann <[email protected]>
Co-authored-by:	Claude Fable 5 <[email protected]>
Reviewed by:	imp
MFC after:	1 week
Sponsored by:	Spectra Logic
Differential Revision:	https://reviews.freebsd.org/D59534

sbin/camcontrol/Makefile

@@ -18,7 +18,7 @@ CFLAGS+= -I${SRCTOP}/sbin/nvmecontrol
 SRCS+=	identify_ext.c
 SRCS+=	nc_util.c
 .PATH:	${SRCTOP}/sys/dev/nvme
-SRCS+=	nvme_util.c
+SRCS+=	nvme_util.c nvres.c
 # This is verboten
 .if ${MACHINE_CPUARCH} == "arm"
 WARNS?= 3

sbin/camcontrol/camcontrol.8

@@ -312,6 +312,7 @@
 .Op generic args
 .Aq Fl i Ar action | Fl o Ar action
 .Op Fl a
+.Op Fl c Ar state
 .Op Fl I Ar trans_id
 .Op Fl k Ar key
 .Op Fl K Ar sa_key
@@ -1801,12 +1802,23 @@ register values that would be used for the firmware download command.
 Do not ask for confirmation.
 .El
 .It Ic persist
-Persistent reservation support.
+Persistent reservation support for
+.Tn SCSI
+and
+.Tn NVMe
+devices.
 Persistent reservations are a way to reserve a particular
 .Tn SCSI
-LUN for use by one or more
+LUN or
+.Tn NVMe
+namespace for use by one or more initiators (hosts).
+The
+.Tn NVMe
+reservation model is closely patterned after
 .Tn SCSI
-initiators.
+Persistent Reservations, so the two share this interface;
+.Nm
+issues the commands appropriate for the device given.
 If the
 .Fl i
 option is specified,
@@ -1814,7 +1826,9 @@ option is specified,
 will issue the
 .Tn SCSI
 PERSISTENT RESERVE IN
-command using the requested service action.
+command using the requested service action, or the
+.Tn NVMe
+Reservation Report command.
 If the
 .Fl o
 option is specified,
@@ -1822,18 +1836,25 @@ option is specified,
 will issue the
 .Tn SCSI
 PERSISTENT RESERVE OUT
-command using the requested service action.
+command using the requested service action, or the corresponding
+.Tn NVMe
+reservation command.
 One of those two options is required.
 .Pp
 Persistent reservations are complex, and fully explaining them is outside
 the scope of this manual.
 Please visit
 https://www.t10.org
-and download the latest SPC spec for a full explanation of persistent
-reservations.
+and download the latest SPC spec, or visit
+https://www.nvmexpress.org
+and download the latest NVMe Base Specification, for a full explanation of
+persistent reservations.
 .Bl -tag -width 8n
 .It Fl i Ar mode
 Specify the service action for the PERSISTENT RESERVE IN command.
+For
+.Tn NVMe
+devices, all of the supported modes issue the Reservation Report command.
 Supported service actions:
 .Bl -tag -width 19n
 .It read_keys
@@ -1843,11 +1864,24 @@ registered keys.
 Report the persistent reservation, if any.
 .It report_capabilities
 Report the persistent reservation capabilities of the LUN.
+Not supported for
+.Tn NVMe
+devices; the reservation capabilities of a namespace are in the Identify
+Namespace data.
 .It read_full_status
 Report the full status of persistent reservations on the LUN.
+For
+.Tn NVMe
+devices, this requests the extended data structure, which contains 128-bit
+Host Identifiers.
 .El
 .It Fl o Ar mode
 Specify the service action for the PERSISTENT RESERVE OUT command.
+For
+.Tn NVMe
+devices, each service action maps to the corresponding
+.Tn NVMe
+reservation operation.
 For service actions like register that are components of other service
 action names, the entire name must be specified.
 Otherwise, enough of the service action name must be specified to
@@ -1867,6 +1901,19 @@ This is similar to the register subcommand, except that the Reservation Key
 is ignored.
 The Service Action Reservation Key will overwrite any previous key
 registered for the initiator.
+.It unregister
+Unregister the key specified as the Reservation Key.
+This action is only supported for
+.Tn NVMe
+devices; on
+.Tn SCSI
+devices a key is unregistered with the register action.
+.It replace
+Replace the key specified as the Reservation Key with the key specified as
+the Service Action Reservation Key.
+This action is only supported for
+.Tn NVMe
+devices.
 .It reserve
 Create a reservation.
 A key must be registered with the LUN before the LUN can be reserved, and
@@ -1894,16 +1941,39 @@ operation being performed.
 Register another initiator with the LUN, and establish a reservation on the
 LUN for that initiator.
 The Reservation Key and Service Action Reservation Key must be specified.
+Not supported for
+.Tn NVMe
+devices.
 .It replace_lost
 Replace Lost Reservation information.
+Not supported for
+.Tn NVMe
+devices.
 .El
 .It Fl a
 Set the All Target Ports (ALL_TG_PT) bit.
 This requests that the key registration be applied to all target ports and
 not just the particular target port that receives the command.
 This only applies to the register and register_ignore actions.
+Not supported for
+.Tn NVMe
+devices.
+.It Fl c Ar state
+Change the Persist Through Power Loss State for the namespace.
+The state may be set to
+.Dq enable
+or
+.Dq disable .
+By default, no change is made to the current setting.
+This option is only supported for
+.Tn NVMe
+devices, and only applies to the register, register_ignore, unregister and
+replace actions.
 .It Fl I Ar tid
 Specify a Transport ID.
+Not supported for
+.Tn NVMe
+devices.
 This only applies to the Register and Register and Move service actions for
 Persistent Reserve Out.
 Multiple Transport IDs may be specified with multiple
@@ -1993,12 +2063,25 @@ Specify the Service Action Reservation Key.
 This may be in decimal, octal or hexadecimal format.
 The value is zero by default if not otherwise specified.
 The value must be between 0 and 2^64 - 1, inclusive.
+For
+.Tn NVMe
+devices, this is the new reservation key for the register, register_ignore
+and replace actions, and the key to preempt for the preempt and
+preempt_abort actions.
 .It Fl p
 Enable the Activate Persist Through Power Loss bit.
 This is only used for the register and register_ignore actions.
 This requests that the reservation persist across power loss events.
+For
+.Tn NVMe
+devices, this enables Persist Through Power Loss State, and is equivalent
+to specifying
+.Fl c Cm enable .
 .It Fl s Ar scope
 Specify the scope of the reservation.
+Not supported for
+.Tn NVMe
+devices.
 The scope may be specified by name or by number.
 The scope is ignored for register, register_ignore and clear.
 If the desired scope isn't available by name, you may specify the number.
@@ -2018,6 +2101,9 @@ Reserve Out command.
 .It Fl S
 Enable the SPEC_I_PT bit.
 This only applies to the Register service action of Persistent Reserve Out.
+Not supported for
+.Tn NVMe
+devices.
 You must also specify at least one Transport ID with
 .Fl I
 if this option is set.
@@ -2029,10 +2115,18 @@ Specify the reservation type.
 The reservation type may be specified by name or by number.
 If the desired reservation type isn't available by name, you may specify
 the number.
+The same names are used for
+.Tn SCSI
+and
+.Tn NVMe
+devices.
 Supported reservation type names:
 .Bl -tag -width 11n
 .It read_shared
 Read Shared mode.
+Not supported for
+.Tn NVMe
+devices.
 .It wr_ex
 Write Exclusive mode.
 May also be specified as
@@ -2041,6 +2135,9 @@ May also be specified as
 Read Exclusive mode.
 May also be specified as
 .Dq read_exclusive .
+Not supported for
+.Tn NVMe
+devices.
 .It ex_ac
 Exclusive access mode.
 May also be specified as
@@ -2069,6 +2166,9 @@ By default, the target will not unregister the initiator that sends the
 Register and Move request.
 This option only applies to the Register and Move service action of the
 Persistent Reserve Out command.
+Not supported for
+.Tn NVMe
+devices.
 .El
 .It Ic attrib
 Issue the

sbin/camcontrol/camcontrol.c

@@ -228,7 +228,7 @@ static struct camcontrol_opts option_table[] = {
 	{"security", CAM_CMD_SECURITY, CAM_ARG_NONE, "d:e:fh:k:l:qs:T:U:y"},
 	{"hpa", CAM_CMD_HPA, CAM_ARG_NONE, "Pflp:qs:U:y"},
 	{"ama", CAM_CMD_AMA, CAM_ARG_NONE, "fqs:"},
-	{"persist", CAM_CMD_PERSIST, CAM_ARG_NONE, "ai:I:k:K:o:ps:ST:U"},
+	{"persist", CAM_CMD_PERSIST, CAM_ARG_NONE, "ac:i:I:k:K:o:ps:ST:U"},
 	{"attrib", CAM_CMD_ATTRIB, CAM_ARG_NONE, "a:ce:F:p:r:s:T:w:V:"},
 	{"opcodes", CAM_CMD_OPCODES, CAM_ARG_NONE, "No:s:T"},
 	{"zone", CAM_CMD_ZONE, CAM_ARG_NONE, "ac:l:No:P:"},
@@ -10144,8 +10144,8 @@ usage(int printlong)
 "                              [-q] [-s max_sectors] [-U pwd] [-y]\n"
 "        camcontrol ama        [dev_id][generic args] [-f] [-q] [-s max_sectors]\n"
 "        camcontrol persist    [dev_id][generic args] <-i action|-o action>\n"
-"                              [-a][-I tid][-k key][-K sa_key][-p][-R rtp]\n"
-"                              [-s scope][-S][-T type][-U]\n"
+"                              [-a][-c state][-I tid][-k key][-K sa_key][-p]\n"
+"                              [-R rtp][-s scope][-S][-T type][-U]\n"
 "        camcontrol attrib     [dev_id][generic args] <-r action|-w attr>\n"
 "                              [-a attr_num][-c][-e elem][-F form1,form1]\n"
 "                              [-p part][-s start][-T type][-V vol]\n"
@@ -10207,7 +10207,7 @@ usage(int printlong)
 "powermode   send the ATA CHECK POWER MODE command to the named device\n"
 "fwdownload  program firmware of the named device with the given image\n"
 "security    report or send ATA security commands to the named device\n"
-"persist     send the SCSI PERSISTENT RESERVE IN or OUT commands\n"
+"persist     send SCSI PERSISTENT RESERVE or NVMe reservation commands\n"
 "attrib      send the SCSI READ or WRITE ATTRIBUTE commands\n"
 "opcodes     send the SCSI REPORT SUPPORTED OPCODES command\n"
 "zone        manage Zoned Block (Shingled) devices\n"
@@ -10829,12 +10829,26 @@ main(int argc, char **argv)
 		error = sanitize(cam_dev, argc, argv, combinedopt, task_attr,
 				 retry_count, timeout);
 		break;
-	case CAM_CMD_PERSIST:
-		error = scsipersist(cam_dev, argc, argv, combinedopt,
-		    task_attr, retry_count, timeout,
-		    arglist & CAM_ARG_VERBOSE,
-		    arglist & CAM_ARG_ERR_RECOVER);
+	case CAM_CMD_PERSIST: {
+		struct ccb_pathinq cpi;
+
+		if (get_cpi(cam_dev, &cpi) != 0) {
+			warnx("couldn't get CPI");
+			error = 1;
+			break;
+		}
+		if (cpi.protocol == PROTO_NVME)
+			error = nvmepersist(cam_dev, argc, argv, combinedopt,
+			    retry_count, timeout,
+			    arglist & CAM_ARG_VERBOSE,
+			    arglist & CAM_ARG_ERR_RECOVER);
+		else
+			error = scsipersist(cam_dev, argc, argv, combinedopt,
+			    task_attr, retry_count, timeout,
+			    arglist & CAM_ARG_VERBOSE,
+			    arglist & CAM_ARG_ERR_RECOVER);
 		break;
+	}
 	case CAM_CMD_ATTRIB:
 		error = scsiattrib(cam_dev, argc, argv, combinedopt,
 		    task_attr, retry_count, timeout,

sbin/camcontrol/camcontrol.h

@@ -77,6 +77,9 @@ int get_ata_status(struct cam_device *dev, union ccb *ccb, uint8_t *error,
 		   uint8_t *status);
 int nvme_get_data(struct cam_device *dev, int nvme_data_type, void *data,
 		  size_t data_len);
+int nvmepersist(struct cam_device *device, int argc, char **argv,
+		char *combinedopt, int retry_count, int timeout,
+		int verbosemode, int err_recover);
 int camxferrate(struct cam_device *device);
 int fwdownload(struct cam_device *device, int argc, char **argv,
 	       char *combinedopt, int printerrors, int task_attr,

sbin/camcontrol/nvres.c

@@ -0,0 +1,674 @@
+/*-
+ * Copyright (c) 2020-2021 Spectra Logic Corporation
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions, and the following disclaimer,
+ *    without modification.
+ * 2. Redistributions in binary form must reproduce at minimum a disclaimer
+ *    substantially similar to the "NO WARRANTY" disclaimer below
+ *    ("Disclaimer") and any redistribution must be conditioned upon
+ *    including a substantially similar Disclaimer requirement for further
+ *    binary redistribution.
+ *
+ * NO WARRANTY
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGES.
+ *
+ * Authors: Ken Merry           (Spectra Logic Corporation)
+ *          Reid Linnemann      (Spectra Logic Corporation)
+ */
+/*
+ * NVMe Reservation support for camcontrol(8).
+ */
+
+#include <sys/ioctl.h>
+#include <sys/stdint.h>
+#include <sys/types.h>
+#include <sys/endian.h>
+#include <sys/sbuf.h>
+#include <sys/queue.h>
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <string.h>
+#include <strings.h>
+#include <fcntl.h>
+#include <ctype.h>
+#include <limits.h>
+#include <err.h>
+
+#include <cam/cam.h>
+#include <cam/cam_debug.h>
+#include <cam/cam_ccb.h>
+#include <cam/scsi/scsi_all.h>
+#include <cam/scsi/scsi_pass.h>
+#include <dev/nvme/nvme.h>
+#include <camlib.h>
+#include "camcontrol.h"
+
+/*
+ * PERSISTENT RESERVE OUT style action names, translated to the NVMe
+ * reservation opcode, the action field within that command, and whether
+ * to set the Ignore Existing Key bit.  The names match the camcontrol
+ * persist -o actions where SCSI and NVMe have equivalent operations;
+ * unregister and replace are NVMe-only.
+ */
+struct nvres_out_action {
+	uint8_t		opc;	/* NVMe reservation opcode */
+	int		action;	/* action field within the command */
+	int		iekey;	/* set Ignore Existing Key */
+};
+
+static struct nvres_out_action nvres_out_actions[] = {
+	{ NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_REGISTER, 0 },
+	{ NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_REGISTER, 1 },
+	{ NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_UNREGISTER, 0 },
+	{ NVME_OPC_RESERVATION_REGISTER, NVME_RESV_REG_REPLACE, 0 },
+	{ NVME_OPC_RESERVATION_ACQUIRE, NVME_RESV_ACQUIRE_ACQUIRE, 0 },
+	{ NVME_OPC_RESERVATION_ACQUIRE, NVME_RESV_ACQUIRE_PREEMPT, 0 },
+	{ NVME_OPC_RESERVATION_ACQUIRE, NVME_RESV_ACQUIRE_PREEMPT_ABT, 0 },
+	{ NVME_OPC_RESERVATION_RELEASE, NVME_RESV_REL_RELEASE, 0 },
+	{ NVME_OPC_RESERVATION_RELEASE, NVME_RESV_REL_CLEAR, 0 }
+};
+
+/* Values are indices into nvres_out_actions[]. */
+static struct scsi_nv nvres_out_action_names[] = {
+	{ "register", 0 },
+	{ "register_ignore", 1 },
+	{ "unregister", 2 },
+	{ "replace", 3 },
+	{ "reserve", 4 },
+	{ "preempt", 5 },
+	{ "preempt_abort", 6 },
+	{ "release", 7 },
+	{ "clear", 8 }
+};
+
+/*
+ * PERSISTENT RESERVE IN style mode names.  All of them map to the NVMe
+ * Reservation Report command; read_full_status additionally requests the
+ * extended data structure (128-bit host IDs).  Values are the eds flag.
+ */
+static struct scsi_nv nvres_in_modes[] = {
+	{ "read_keys", 0 },
+	{ "read_reservation", 0 },
+	{ "read_full_status", 1 }
+};
+
+/* Change Persist Through Power Loss State values for -c. */
+static struct scsi_nv nvres_cptpl_vals[] = {
+	{ "disable", NVME_RESV_CPTPL_0 },
+	{ "enable", NVME_RESV_CPTPL_1 }
+};
+
+static struct scsi_nv nvres_res_types[] = {
+	{ "write_exclusive", NVME_RESV_TYPE_WR_EX },
+	{ "wr_ex", NVME_RESV_TYPE_WR_EX },
+	{ "exclusive_access", NVME_RESV_TYPE_EX_AC },
+	{ "ex_ac", NVME_RESV_TYPE_EX_AC },
+	{ "write_exclusive_reg_only", NVME_RESV_TYPE_WR_EX_RO },
+	{ "wr_ex_ro", NVME_RESV_TYPE_WR_EX_RO },
+	{ "exclusive_access_reg_only", NVME_RESV_TYPE_EX_AC_RO },
+	{ "ex_ac_ro", NVME_RESV_TYPE_EX_AC_RO },
+	{ "write_exclusive_all_regs", NVME_RESV_TYPE_WR_EX_AR },
+	{ "wr_ex_ar", NVME_RESV_TYPE_WR_EX_AR },
+	{ "exclusive_access_all_regs", NVME_RESV_TYPE_EX_AC_AR },
+	{ "ex_ac_ar", NVME_RESV_TYPE_EX_AC_AR }
+};
+
+static int
+nvmeres_acquire(struct cam_device *device, int verbosemode, int err_recover,
+		int rtype, int iekey, int racqa, uint64_t crkey, uint64_t prkey,
+		int retry_count, int timeout)
+{
+	union ccb *ccb = NULL;
+	uint64_t data[2];
+	int error = 0;
+
+	ccb = cam_getccb(device);
+	if (ccb == NULL) {
+		warnx("%s: error allocating CCB", __func__);
+		error = 1;
+		goto bailout;
+	}
+	CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio);
+
+	data[0] = htole64(crkey);
+	data[1] = htole64(prkey);
+
+	nvme_resv_acquire(/*nvmeio*/ &ccb->nvmeio,
+			  /*retries*/ retry_count,
+			  /*cbfcnp*/ NULL,
+			  /*nsid*/ device->target_lun,
+			  /*rtype*/ rtype,
+			  /*iekey*/ iekey,
+			  /*rrega*/ racqa,
+			  /*data_ptr*/ (uint8_t *)data,
+			  /*dxfer_len*/ sizeof(data),
+			  /*timeout*/ timeout ? timeout : 30000);
+
+	if (err_recover != 0)
+		ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
+
+	if (cam_send_ccb(device, ccb) < 0) {
+		warn("error sending NVMe Reservation Acquire");
+		error = 1;
+		goto bailout;
+	}
+
+	if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) {
+		warnx("NVMe Reservation Acquire failed");
+		if (verbosemode != 0) {
+			cam_error_print(device, ccb, CAM_ESF_ALL, CAM_EPF_ALL,
+			    stderr);
+		}
+		error = 1;
+		goto bailout;
+	}
+
+bailout:
+	if (ccb != NULL) {
+		cam_freeccb(ccb);
+	}
+	return (error);
+}
+
+static int
+nvmeres_register(struct cam_device *device, int verbosemode, int err_recover,
+		 int cptpl, int iekey, int rrega, uint64_t curkey,
+		 uint64_t newkey, int retry_count, int timeout)
+{
+	union ccb *ccb = NULL;
+	uint64_t data[2];
+	int error = 0;
+
+	ccb = cam_getccb(device);
+	if (ccb == NULL) {
+		warnx("%s: error allocating CCB", __func__);
+		error = 1;
+		goto bailout;
+	}
+	CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio);
+
+	data[0] = htole64(curkey);
+	data[1] = htole64(newkey);
+
+	nvme_resv_register(/*nvmeio*/ &ccb->nvmeio,
+			   /*retries*/ retry_count,
+			   /*cbfcnp*/ NULL,
+			   /*nsid*/ device->target_lun,
+			   /*cptpl*/ cptpl,
+			   /*iekey*/ iekey,
+			   /*rrega*/ rrega,
+			   /*data_ptr*/ (uint8_t*)data,
+			   /*dxfer_len*/ sizeof(data),
+			   /*timeout*/ timeout ? timeout: 30000);
+
+	if (err_recover != 0)
+		ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
+
+	if (cam_send_ccb(device, ccb) < 0) {
+		warn("error sending NVMe Reservation Registration");
+		error = 1;
+		goto bailout;
+	}
+
+	if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) {
+		warnx("NVMe Reservation Register failed");
+		if (verbosemode != 0) {
+			cam_error_print(device, ccb, CAM_ESF_ALL,
+					CAM_EPF_ALL, stderr);
+		}
+		error = 1;
+		goto bailout;
+	}
+
+
+bailout:
+	if (ccb != NULL) {
+		cam_freeccb(ccb);
+	}
+	return (error);
+}
+
+static int
+nvmeres_release(struct cam_device *device, int verbosemode, int err_recover,
+		int rtype, int iekey, int rrela, uint64_t curkey,
+		int retry_count, int timeout)
+{
+	union ccb *ccb = NULL;
+	uint64_t data[1];
+	int error = 0;
+
+	ccb = cam_getccb(device);
+	if (ccb == NULL) {
+		warnx("%s: error allocating CCB", __func__);
+		error = 1;
+		goto bailout;
+	}
+	CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio);
+
+	data[0] = htole64(curkey);
+
+	nvme_resv_release(/*nvmeio*/ &ccb->nvmeio,
+			  /*retries*/ retry_count,
+			  /*cbfcnp*/ NULL,
+			  /*nsid*/ device->target_lun,
+			  /*rtype*/ rtype,
+			  /*iekey*/ iekey,
+			  /*rrela*/ rrela,
+			  /*data_ptr*/ (uint8_t*)data,
+			  /*dxfer_len*/ sizeof(data),
+			  /*timeout*/ timeout ? timeout: 30000);
+
+	if (err_recover != 0)
+		ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
+
+	if (cam_send_ccb(device, ccb) < 0) {
+		warn("error sending NVMe Reservation Release");
+		error = 1;
+		goto bailout;
+	}
+
+	if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) {
+		warnx("NVMe Reservation Release failed");
+		if (verbosemode != 0) {
+			cam_error_print(device, ccb, CAM_ESF_ALL,
+					CAM_EPF_ALL, stderr);
+		}
+		error = 1;
+		goto bailout;
+	}
+
+
+bailout:
+	if (ccb != NULL) {
+		cam_freeccb(ccb);
+	}
+	return (error);
+}
+
+static int
+nvmeres_report(struct cam_device *device, int verbosemode, int err_recover,
+	       int eds, int retry_count, int timeout)
+{
+	uint8_t report_data[8192];
+	union ccb *ccb = NULL;
+	struct nvme_resv_status *rstatus;
+	struct nvme_resv_status_ext *extstatus;
+	struct sbuf *hostid_sbuf = NULL;
+	const char *rtype_str = NULL;
+	int num_ents = 0;
+	int error = 0;
+	int i;
+
+	ccb = cam_getccb(device);
+	if (ccb == NULL) {
+		warnx("%s: error allocating CCB", __func__);
+		error = 1;
+		goto bailout;
+	}
+	CCB_CLEAR_ALL_EXCEPT_HDR(&ccb->nvmeio);
+
+	bzero(report_data, sizeof(report_data));
+
+	nvme_resv_report(/*nvmeio*/ &ccb->nvmeio,
+			 /*retries*/ retry_count,
+			 /*cbfcnp*/ NULL,
+			 /*nsid*/ device->target_lun,
+			 /*report_type*/ eds,
+			 /*data_ptr*/ report_data,
+			 /*dxfer_len */ sizeof(report_data),
+			 /*timeout*/ timeout ? timeout : 30000);
+
+	if (err_recover != 0)
+		ccb->ccb_h.flags |= CAM_PASS_ERR_RECOVER;
+
+	if (cam_send_ccb(device, ccb) < 0) {
+		warn("error sending NVMe Reservation Report");
+		error = 1;
+		goto bailout;
+	}
+
+	if ((ccb->ccb_h.status & CAM_STATUS_MASK) != CAM_REQ_CMP) {
+		warnx("NVMe Reservation Report failed");
+		if (verbosemode != 0) {
+			cam_error_print(device, ccb, CAM_ESF_ALL,
+					CAM_EPF_ALL, stderr);
+		}
+		error = 1;
+		goto bailout;
+	}
+
+	hostid_sbuf = sbuf_new(NULL, NULL, 64, SBUF_FIXEDLEN);
+	if (hostid_sbuf == NULL) {
+		warn("error allocating sbuf");
+		error = 1;
+		goto bailout;
+	}
+	rstatus = (struct nvme_resv_status *)report_data;
+	/*
+	 * XXX KDM evidently there is no concept of a residual in NVMe.
+	 */
+	if (eds == 0)
+		nvme_resv_status_swapbytes(rstatus,sizeof(report_data));
+	else
+		nvme_resv_status_ext_swapbytes((struct nvme_resv_status_ext *)
+		    report_data,sizeof(report_data));
+
+	num_ents = (rstatus->regctl[1] << 8) | rstatus->regctl[0];
+	printf("Generation: %u\n", rstatus->gen);
+	printf("Reservation type: %u", rstatus->rtype);
+	if (rstatus->rtype > 0) {
+		rtype_str = scsi_nv_to_str(nvres_res_types,
+		    sizeof(nvres_res_types) / sizeof(nvres_res_types[0]),
+		    rstatus->rtype);
+		printf(" (%s)", rtype_str?rtype_str:"invalid");
+	}
+	printf("\n");
+	printf("Number of Registered Controllers: %u\n", num_ents);
+	printf("Persist Through Power Loss State: %u\n", rstatus->ptpls);
+
+	/*
+	 * The number of entries comes from the device.  Constrain it to
+	 * the number that actually fit in our data buffer, in case the
+	 * device reports more.
+	 */
+	if (eds == 1) {
+		extstatus = (struct nvme_resv_status_ext *)report_data;
+		if (num_ents > (int)((sizeof(report_data) -
+		    sizeof(*extstatus)) / sizeof(extstatus->ctrlr[0])))
+			num_ents = (sizeof(report_data) -
+			    sizeof(*extstatus)) / sizeof(extstatus->ctrlr[0]);
+		for (i = 0; i < num_ents; i++) {
+			sbuf_clear(hostid_sbuf);
+			nvme_hostid_sbuf(hostid_sbuf,
+			    (uint8_t*)(extstatus->ctrlr[i].hostid), 1);
+			sbuf_finish(hostid_sbuf);
+			printf("Controller ID:  0x%04x\n",
+			    extstatus->ctrlr[i].ctrlr_id);
+			printf("Reservation Status:  %u\n",
+			    extstatus->ctrlr[i].rcsts);
+			printf("Host Identifier:  %s\n", sbuf_data(hostid_sbuf));
+			printf("Reservation Key:  0x%016jx\n",
+			    (uintmax_t)extstatus->ctrlr[i].rkey);
+		}
+	} else {
+		if (num_ents > (int)((sizeof(report_data) -
+		    sizeof(*rstatus)) / sizeof(rstatus->ctrlr[0])))
+			num_ents = (sizeof(report_data) -
+			    sizeof(*rstatus)) / sizeof(rstatus->ctrlr[0]);
+		for (i = 0; i < num_ents; i++) {
+			sbuf_clear(hostid_sbuf);
+			nvme_hostid_sbuf(hostid_sbuf,
+			    (uint8_t*)&(rstatus->ctrlr[i].hostid), 0);
+			sbuf_finish(hostid_sbuf);
+			printf("Controller ID:  0x%04x\n",
+			    rstatus->ctrlr[i].ctrlr_id);
+			printf("Reservation Status:  %u\n",
+			    rstatus->ctrlr[i].rcsts);
+			printf("Host Identifier:  %s\n", sbuf_data(hostid_sbuf));
+			printf("Reservation Key:  0x%016jx\n",
+			    (uintmax_t)rstatus->ctrlr[i].rkey);
+		}
+	}
+bailout:
+	if (ccb != NULL) {
+		cam_freeccb(ccb);
+	}
+	if (hostid_sbuf != NULL) {
+		sbuf_delete(hostid_sbuf);
+	}
+	return (error);
+}
+
+int
+nvmepersist(struct cam_device *device, int argc, char **argv, char *combinedopt,
+	    int retry_count, int timeout, int verbosemode, int err_recover)
+{
+	uint64_t curkey = 0, other_key = 0;
+	int eds = 0, cptpl = 0, iekey = 0;
+	int error = 0;
+	int in = 0, out = 0;
+	int opc = -1, op_action = -1, res_type = 0;
+	int c;
+
+	while ((c = getopt(argc, argv, combinedopt)) != -1) {
+		switch (c) {
+		case 'i': {
+			scsi_nv_status status;
+			int table_entry = 0;
+
+			status = scsi_get_nv(nvres_in_modes,
+			    sizeof(nvres_in_modes) / sizeof(nvres_in_modes[0]),
+			    optarg, &table_entry, SCSI_NV_FLAG_IG_CASE);
+			if (status == SCSI_NV_FOUND) {
+				in = 1;
+				opc = NVME_OPC_RESERVATION_REPORT;
+				eds = nvres_in_modes[table_entry].value;
+			} else {
+				warnx("%s: %s in mode %s for NVMe devices",
+				    __func__, (status == SCSI_NV_AMBIGUOUS) ?
+				    "ambiguous" : "invalid", optarg);
+				error = 1;
+				goto bailout;
+			}
+			break;
+		}
+		case 'o': {
+			scsi_nv_status status;
+			struct nvres_out_action *oa;
+			int table_entry = 0;
+
+			status = scsi_get_nv(nvres_out_action_names,
+			    sizeof(nvres_out_action_names) /
+			    sizeof(nvres_out_action_names[0]),
+			    optarg, &table_entry, SCSI_NV_FLAG_IG_CASE);
+			if (status == SCSI_NV_FOUND) {
+				out = 1;
+				oa = &nvres_out_actions[
+				    nvres_out_action_names[table_entry].value];
+				opc = oa->opc;
+				op_action = oa->action;
+				iekey = oa->iekey;
+			} else {
+				warnx("%s: %s out action %s for NVMe devices",
+				    __func__, (status == SCSI_NV_AMBIGUOUS) ?
+				    "ambiguous" : "invalid", optarg);
+				error = 1;
+				goto bailout;
+			}
+			break;
+		}
+		case 'k':
+		case 'K': {
+			char *endptr;
+			uint64_t tmpval;
+
+			tmpval = strtoumax(optarg, &endptr, 0);
+			if (*endptr != '\0') {
+				warnx("%s: invalid key argument %s", __func__,
+				    optarg);
+				error = 1;
+				goto bailout;
+			}
+			if (c == 'k')
+				curkey = tmpval;
+			else
+				other_key = tmpval;
+			break;
+		}
+		case 'c': {
+			scsi_nv_status status;
+			int table_entry = 0;
+
+			status = scsi_get_nv(nvres_cptpl_vals,
+			    sizeof(nvres_cptpl_vals) /
+			    sizeof(nvres_cptpl_vals[0]),
+			    optarg, &table_entry, SCSI_NV_FLAG_IG_CASE);
+			if (status == SCSI_NV_FOUND)
+				cptpl = nvres_cptpl_vals[table_entry].value;
+			else {
+				warnx("%s: %s persist through power loss "
+				    "value %s", __func__,
+				    (status == SCSI_NV_AMBIGUOUS) ?
+				    "ambiguous" : "invalid", optarg);
+				error = 1;
+				goto bailout;
+			}
+			break;
+		}
+		case 'p':
+			/*
+			 * SCSI APTPL is approximately NVMe Change Persist
+			 * Through Power Loss State, enabled.  The default
+			 * is to make no change to the current setting;
+			 * -c gives explicit control.
+			 */
+			cptpl = NVME_RESV_CPTPL_1;
+			break;
+		case 'T': {
+			scsi_nv_status status;
+			int table_entry = 0;
+
+			status = scsi_get_nv(nvres_res_types,
+			    sizeof(nvres_res_types) /sizeof(nvres_res_types[0]),
+			    optarg, &table_entry, SCSI_NV_FLAG_IG_CASE);
+			if (status == SCSI_NV_FOUND)
+				res_type = nvres_res_types[table_entry].value;
+			else {
+				warnx("%s: %s reservation type %s", __func__,
+				    (status == SCSI_NV_AMBIGUOUS) ?
+				    "ambiguous" : "invalid", optarg);
+				error = 1;
+				goto bailout;
+			}
+
+			break;
+		}
+		case 'a':
+		case 'I':
+		case 's':
+		case 'S':
+		case 'U':
+			warnx("%s: option -%c is not supported for NVMe "
+			    "devices", __func__, c);
+			error = 1;
+			goto bailout;
+		}
+	}
+	if ((in + out) != 1) {
+		warnx("%s: you must specify one of -i or -o", __func__);
+		error = 1;
+		goto bailout;
+	}
+
+	switch (opc) {
+	case NVME_OPC_RESERVATION_REPORT:
+		error = nvmeres_report(device, verbosemode, err_recover, eds,
+		    retry_count, timeout);
+		break;
+	case NVME_OPC_RESERVATION_REGISTER:
+		switch (op_action) {
+		case NVME_RESV_REG_REGISTER:
+			if (other_key == 0) {
+				warnx("-K newkey is required");
+				error = 1;
+				goto bailout;
+			}
+			break;
+		case NVME_RESV_REG_REPLACE:
+			if (other_key == 0) {
+				warnx("-K newkey is required");
+				error = 1;
+				goto bailout;
+			}
+			/* fallthrough */
+		case NVME_RESV_REG_UNREGISTER:
+			if (!iekey && curkey == 0) {
+				warnx("-k curkey is required");
+				error = 1;
+				goto bailout;
+			}
+			break;
+		}
+		error = nvmeres_register(device, verbosemode, err_recover,
+		    cptpl, iekey, op_action, curkey,
+		    other_key, retry_count, timeout);
+		break;
+	case NVME_OPC_RESERVATION_ACQUIRE:
+		switch (op_action) {
+		case NVME_RESV_ACQUIRE_PREEMPT:
+			/* fallthrough */
+		case NVME_RESV_ACQUIRE_PREEMPT_ABT:
+			if (other_key == 0) {
+				warnx("-K prekey is required");
+				error = 1;
+				goto bailout;
+			}
+			/* fallthrough */
+		case NVME_RESV_ACQUIRE_ACQUIRE:
+			if (curkey == 0) {
+				warnx("-k curkey is required");
+				error = 1;
+				goto bailout;
+			}
+			if (res_type == -1) {
+				warnx("-T resv_type is required");
+				error = 1;
+				goto bailout;
+			}
+			break;
+		}
+		error = nvmeres_acquire(device, verbosemode, err_recover,
+					res_type, iekey, op_action, curkey,
+					other_key, retry_count, timeout);
+		break;
+	case NVME_OPC_RESERVATION_RELEASE:
+		switch (op_action) {
+		case NVME_RESV_REL_RELEASE:
+			if (res_type == -1) {
+				warnx("-T resv_type is required");
+				error = 1;
+				goto bailout;
+			}
+			/* fallthrough */
+		case NVME_RESV_REL_CLEAR:
+			if (curkey == 0) {
+				warnx("-k curkey is required");
+				error = 1;
+				goto bailout;
+			}
+			break;
+		}
+
+		error = nvmeres_release(device, verbosemode, err_recover,
+					res_type, iekey, op_action, curkey,
+					retry_count, timeout);
+		break;
+	default:
+		warnx("%s: unknown reservation opcode %#x", __func__, opc);
+		error = 1;
+		goto bailout;
+		break;
+	}
+bailout:
+
+	return (error);
+}