ray.serve.schema.DeploymentSchema#

pydantic model ray.serve.schema.DeploymentSchema[source]#

Specifies options for one deployment within a Serve application. For each deployment this can optionally be included in ServeApplicationSchema to override deployment options specified in code.

PublicAPI (beta): This API is in beta and may change before becoming stable.

Show JSON schema

{
   "title": "DeploymentSchema",
   "description": "Specifies options for one deployment within a Serve application. For each deployment\nthis can optionally be included in `ServeApplicationSchema` to override deployment\noptions specified in code.\n\n**PublicAPI (beta):** This API is in beta and may change before becoming stable.",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "description": "Globally-unique name identifying this deployment.",
         "type": "string"
      },
      "num_replicas": {
         "title": "Num Replicas",
         "description": "The number of processes that handle requests to this deployment. Uses a default if null.",
         "default": 1,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "route_prefix": {
         "title": "Route Prefix",
         "description": "Requests to paths under this HTTP path prefix will be routed to this deployment. When null, no HTTP endpoint will be created. When omitted, defaults to the deployment's name. Routing is done based on longest-prefix match, so if you have deployment A with a prefix of \"/a\" and deployment B with a prefix of \"/a/b\", requests to \"/a\", \"/a/\", and \"/a/c\" go to A and requests to \"/a/b\", \"/a/b/\", and \"/a/b/c\" go to B. Routes must not end with a \"/\" unless they're the root (just \"/\"), which acts as a catch-all.",
         "default": 1,
         "type": "string"
      },
      "max_concurrent_queries": {
         "title": "Max Concurrent Queries",
         "description": "The max number of pending queries in a single replica. Uses a default if null.",
         "default": 1,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "user_config": {
         "title": "User Config",
         "description": "Config to pass into this deployment's reconfigure method. This can be updated dynamically without restarting replicas",
         "default": 1,
         "type": "object"
      },
      "autoscaling_config": {
         "title": "Autoscaling Config",
         "description": "Config specifying autoscaling parameters for the deployment's number of replicas. If null, the deployment won't autoscale its number of replicas; the number of replicas will be fixed at num_replicas.",
         "default": 1,
         "type": "object"
      },
      "graceful_shutdown_wait_loop_s": {
         "title": "Graceful Shutdown Wait Loop S",
         "description": "Duration that deployment replicas will wait until there is no more work to be done before shutting down. Uses a default if null.",
         "default": 1,
         "minimum": 0,
         "type": "number"
      },
      "graceful_shutdown_timeout_s": {
         "title": "Graceful Shutdown Timeout S",
         "description": "Serve controller waits for this duration before forcefully killing the replica for shutdown. Uses a default if null.",
         "default": 1,
         "minimum": 0,
         "type": "number"
      },
      "health_check_period_s": {
         "title": "Health Check Period S",
         "description": "Frequency at which the controller will health check replicas. Uses a default if null.",
         "default": 1,
         "exclusiveMinimum": 0,
         "type": "number"
      },
      "health_check_timeout_s": {
         "title": "Health Check Timeout S",
         "description": "Timeout that the controller will wait for a response from the replica's health check before marking it unhealthy. Uses a default if null.",
         "default": 1,
         "exclusiveMinimum": 0,
         "type": "number"
      },
      "ray_actor_options": {
         "title": "Ray Actor Options",
         "description": "Options set for each replica actor.",
         "default": 1,
         "allOf": [
            {
               "$ref": "#/definitions/RayActorOptionsSchema"
            }
         ]
      },
      "is_driver_deployment": {
         "title": "Is Driver Deployment",
         "description": "Indicate Whether the deployment is driver deployment Driver deployments are spawned one per node.",
         "default": 1,
         "type": "boolean"
      }
   },
   "required": [
      "name"
   ],
   "additionalProperties": false,
   "definitions": {
      "RayActorOptionsSchema": {
         "title": "RayActorOptionsSchema",
         "description": "Options with which to start a replica actor.\n\n**PublicAPI (beta):** This API is in beta and may change before becoming stable.",
         "type": "object",
         "properties": {
            "runtime_env": {
               "title": "Runtime Env",
               "description": "This deployment's runtime_env. working_dir and py_modules may contain only remote URIs.",
               "default": {},
               "type": "object"
            },
            "num_cpus": {
               "title": "Num Cpus",
               "description": "The number of CPUs required by the deployment's application per replica. This is the same as a ray actor's num_cpus. Uses a default if null.",
               "minimum": 0,
               "type": "number"
            },
            "num_gpus": {
               "title": "Num Gpus",
               "description": "The number of GPUs required by the deployment's application per replica. This is the same as a ray actor's num_gpus. Uses a default if null.",
               "minimum": 0,
               "type": "number"
            },
            "memory": {
               "title": "Memory",
               "description": "Restrict the heap memory usage of each replica. Uses a default if null.",
               "minimum": 0,
               "type": "number"
            },
            "object_store_memory": {
               "title": "Object Store Memory",
               "description": "Restrict the object store memory used per replica when creating objects. Uses a default if null.",
               "minimum": 0,
               "type": "number"
            },
            "resources": {
               "title": "Resources",
               "description": "The custom resources required by each replica.",
               "default": {},
               "type": "object"
            },
            "accelerator_type": {
               "title": "Accelerator Type",
               "description": "Forces replicas to run on nodes with the specified accelerator type.",
               "type": "string"
            }
         },
         "additionalProperties": false
      }
   }
}

Fields

Validators

num_replicas_and_autoscaling_config_mutually_exclusive » all fields

field autoscaling_config: Optional[Dict] = DEFAULT.VALUE#

Config specifying autoscaling parameters for the deployment’s number of replicas. If null, the deployment won’t autoscale its number of replicas; the number of replicas will be fixed at num_replicas.

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field graceful_shutdown_timeout_s: float = DEFAULT.VALUE#

Serve controller waits for this duration before forcefully killing the replica for shutdown. Uses a default if null.

Constraints

minimum = 0

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field graceful_shutdown_wait_loop_s: float = DEFAULT.VALUE#

Duration that deployment replicas will wait until there is no more work to be done before shutting down. Uses a default if null.

Constraints

minimum = 0

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field health_check_period_s: float = DEFAULT.VALUE#

Frequency at which the controller will health check replicas. Uses a default if null.

Constraints

exclusiveMinimum = 0

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field health_check_timeout_s: float = DEFAULT.VALUE#

Timeout that the controller will wait for a response from the replica’s health check before marking it unhealthy. Uses a default if null.

Constraints

exclusiveMinimum = 0

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field is_driver_deployment: bool = DEFAULT.VALUE#

Indicate Whether the deployment is driver deployment Driver deployments are spawned one per node.

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field max_concurrent_queries: int = DEFAULT.VALUE#

The max number of pending queries in a single replica. Uses a default if null.

Constraints

exclusiveMinimum = 0

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field name: str [Required]#

Globally-unique name identifying this deployment.

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field num_replicas: Optional[int] = DEFAULT.VALUE#

The number of processes that handle requests to this deployment. Uses a default if null.

Constraints

exclusiveMinimum = 0

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field ray_actor_options: ray.serve.schema.RayActorOptionsSchema = DEFAULT.VALUE#

Options set for each replica actor.

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

field route_prefix: Optional[str] = DEFAULT.VALUE#

Requests to paths under this HTTP path prefix will be routed to this deployment. When null, no HTTP endpoint will be created. When omitted, defaults to the deployment’s name. Routing is done based on longest-prefix match, so if you have deployment A with a prefix of “/a” and deployment B with a prefix of “/a/b”, requests to “/a”, “/a/”, and “/a/c” go to A and requests to “/a/b”, “/a/b/”, and “/a/b/c” go to B. Routes must not end with a “/” unless they’re the root (just “/”), which acts as a catch-all.

Validated by

_route_prefix_format
num_replicas_and_autoscaling_config_mutually_exclusive

field user_config: Optional[Dict] = DEFAULT.VALUE#

Config to pass into this deployment’s reconfigure method. This can be updated dynamically without restarting replicas

Validated by

num_replicas_and_autoscaling_config_mutually_exclusive

classmethod deployment_schema_route_prefix_format(v)#: The route_prefix 1. must start with a / character 2. must not end with a / character (unless the entire prefix is just /) 3. cannot contain wildcards (must not have “{” or “}”)

get_user_configured_option_names() → Set[str][source]#

Get set of names for all user-configured options.

Any field not set to DEFAULT.VALUE is considered a user-configured option.

validator num_replicas_and_autoscaling_config_mutually_exclusive » all fields[source]#

Ray 3.0.0.dev0

ray.serve.schema.DeploymentSchema

ray.serve.schema.DeploymentSchema#