ray.serve.handle.RayServeSyncHandle#

class ray.serve.handle.RayServeSyncHandle[source]#

A handle used to make requests to the ingress deployment of an application.

This is returned by serve.run and can be used to invoke the application from Python rather than over HTTP. For example:

import ray
from ray import serve
from ray.serve.handle import RayServeSyncHandle

@serve.deployment
class Ingress:
    def __call__(self, name: str) -> str:
        return f"Hello {name}"

app = Ingress.bind()
handle: RayServeSyncHandle = serve.run(app)

# Prints "Hello Mr. Magoo"
print(ray.get(handle.remote("Mr. Magoo")))

PublicAPI (beta): This API is in beta and may change before becoming stable.

options(*, method_name: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, multiplexed_model_id: Union[str, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE, stream: Union[bool, ray.serve._private.utils.DEFAULT] = DEFAULT.VALUE) → ray.serve.handle.RayServeSyncHandle[source]#

Set options for this handle and return an updated copy of it.

Example:

# The following two lines are equivalent:
obj_ref = handle.other_method.remote(*args)
obj_ref = handle.options(method_name="other_method").remote(*args)
obj_ref = handle.options(multiplexed_model_id="model1").remote(*args)
remote(*args, **kwargs) → ray.ObjectRef[source]#

Issue an asynchronous request to the __call__ method of the deployment.

Returns a Ray ObjectRef whose results can be waited for or retrieved using ray.wait or ray.get, respectively.

obj_ref = handle.remote(*args)
result = ray.get(obj_ref)