ray.serve.Deployment

class ray.serve.Deployment

Class (or function) decorated with the @serve.deployment decorator.

This is run on a number of replica actors. Requests to those replicas call this class.

One or more deployments can be composed together into an Application which is then run via serve.run or a config file.

Example:

@serve.deployment
class MyDeployment:
    def __init__(self, name: str):
        self._name = name

    def __call__(self, request):
        return "Hello world!"

    app = MyDeployment.bind()
    # Run via `serve.run` or the `serve run` CLI command.
    serve.run(app)

PublicAPI: This API is stable across Ray releases.

property name: str

Unique name of this deployment.

property func_or_class: Union[Callable, str]

Underlying class or function that this deployment wraps.

property num_replicas: int

Current target number of replicas.

property user_config: Any

Current dynamic user-provided config options.

property max_concurrent_queries: int

Current max outstanding queries from each handle.

property route_prefix: Optional[str]

HTTP route prefix that this deployment is exposed under.

property ray_actor_options: Optional[Dict]

Actor options such as resources required for each replica.

bind(*args, **kwargs) → ray.serve.deployment.Application

Bind the arguments to the deployment and return an Application.

The returned Application can be deployed using serve.run (or via config file) or bound to another deployment for composition.

PublicAPI (beta): This API is in beta and may change before becoming stable.

deploy(*init_args, _blocking=True, **init_kwargs)

Deploy or update this deployment.

Parameters

• init_args – args to pass to the class __init__ method. Not valid if this deployment wraps a function.

• init_kwargs – kwargs to pass to the class __init__ method. Not valid if this deployment wraps a function.

Warning

DEPRECATED: This API is deprecated and may be removed in future Ray releases. See https://docs.ray.io/en/latest/serve/index.html for more information.

delete()

Delete this deployment.

Warning

DEPRECATED: This API is deprecated and may be removed in future Ray releases. See https://docs.ray.io/en/latest/serve/index.html for more information.

get_handle(sync: Optional[bool] = True) → Union[ray.serve.handle.RayServeHandle, ray.serve.handle.RayServeSyncHandle]

Get a ServeHandle to this deployment to invoke it from Python.

Parameters

sync – If true, then Serve will return a ServeHandle that works everywhere. Otherwise, Serve will return an asyncio-optimized ServeHandle that’s only usable in an asyncio loop.

Returns

ServeHandle

Warning

DEPRECATED: This API is deprecated and may be removed in future Ray releases. See https://docs.ray.io/en/latest/serve/index.html for more information.

options(func_or_class: Optional[Callable] = None, name: Union[ray.serve._private.utils.DEFAULT, str] = DEFAULT.VALUE, version: Union[ray.serve._private.utils.DEFAULT, str] = DEFAULT.VALUE, num_replicas: Optional[Union[ray.serve._private.utils.DEFAULT, int]] = DEFAULT.VALUE, init_args: Union[ray.serve._private.utils.DEFAULT, Tuple[Any]] = DEFAULT.VALUE, init_kwargs: Union[ray.serve._private.utils.DEFAULT, Dict[Any, Any]] = DEFAULT.VALUE, route_prefix: Optional[Union[ray.serve._private.utils.DEFAULT, str]] = DEFAULT.VALUE, ray_actor_options: Optional[Union[ray.serve._private.utils.DEFAULT, Dict]] = DEFAULT.VALUE, user_config: Optional[Union[ray.serve._private.utils.DEFAULT, Any]] = DEFAULT.VALUE, max_concurrent_queries: Union[ray.serve._private.utils.DEFAULT, int] = DEFAULT.VALUE, autoscaling_config: Optional[Union[ray.serve._private.utils.DEFAULT, Dict, ray.serve.config.AutoscalingConfig]] = DEFAULT.VALUE, graceful_shutdown_wait_loop_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, graceful_shutdown_timeout_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, health_check_period_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, health_check_timeout_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, is_driver_deployment: bool = DEFAULT.VALUE, _internal: bool = False) → ray.serve.deployment.Deployment

Return a copy of this deployment with updated options.

Only those options passed in will be updated, all others will remain unchanged from the existing deployment.

Refer to the @serve.deployment decorator docs for available arguments.

PublicAPI: This API is stable across Ray releases.

set_options(func_or_class: Optional[Callable] = None, name: Union[ray.serve._private.utils.DEFAULT, str] = DEFAULT.VALUE, version: Union[ray.serve._private.utils.DEFAULT, str] = DEFAULT.VALUE, num_replicas: Optional[Union[ray.serve._private.utils.DEFAULT, int]] = DEFAULT.VALUE, init_args: Union[ray.serve._private.utils.DEFAULT, Tuple[Any]] = DEFAULT.VALUE, init_kwargs: Union[ray.serve._private.utils.DEFAULT, Dict[Any, Any]] = DEFAULT.VALUE, route_prefix: Optional[Union[ray.serve._private.utils.DEFAULT, str]] = DEFAULT.VALUE, ray_actor_options: Optional[Union[ray.serve._private.utils.DEFAULT, Dict]] = DEFAULT.VALUE, user_config: Optional[Union[ray.serve._private.utils.DEFAULT, Any]] = DEFAULT.VALUE, max_concurrent_queries: Union[ray.serve._private.utils.DEFAULT, int] = DEFAULT.VALUE, autoscaling_config: Optional[Union[ray.serve._private.utils.DEFAULT, Dict, ray.serve.config.AutoscalingConfig]] = DEFAULT.VALUE, graceful_shutdown_wait_loop_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, graceful_shutdown_timeout_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, health_check_period_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, health_check_timeout_s: Union[ray.serve._private.utils.DEFAULT, float] = DEFAULT.VALUE, is_driver_deployment: bool = DEFAULT.VALUE, _internal: bool = False) → None

Overwrite this deployment’s options in-place.

Only those options passed in will be updated, all others will remain unchanged.

Refer to the @serve.deployment decorator docstring for all non-private arguments.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.