{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc37c39a-7406-4c13-a754-b8e95fd970a0",
   "metadata": {},
   "source": [
    "# Streaming\n",
    "\n",
    "All `LLM`s implement the `Runnable` interface, which comes with default implementations of all methods, ie. ainvoke, batch, abatch, stream, astream. This gives all `LLM`s basic support for streaming.\n",
    "\n",
    "Streaming support defaults to returning an Iterator (or AsyncIterator in the case of async streaming) of a single value, the final result returned by the underlying `LLM` provider. This obviously doesn't give you token-by-token streaming, which requires native support from the `LLM` provider, but ensures your code that expects an iterator of tokens can work for any of our `LLM` integrations.\n",
    "\n",
    "See which [integrations support token-by-token streaming here](/docs/integrations/llms/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9baa0527-b97d-41d3-babd-472ec5e59e3e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Verse 1:\n",
      "Bubbles dancing in my glass\n",
      "Clear and crisp, it's such a blast\n",
      "Refreshing taste, it's like a dream\n",
      "Sparkling water, you make me beam\n",
      "\n",
      "Chorus:\n",
      "Oh sparkling water, you're my delight\n",
      "With every sip, you make me feel so right\n",
      "You're like a party in my mouth\n",
      "I can't get enough, I'm hooked no doubt\n",
      "\n",
      "Verse 2:\n",
      "No sugar, no calories, just pure bliss\n",
      "You're the perfect drink, I must confess\n",
      "From lemon to lime, so many flavors to choose\n",
      "Sparkling water, you never fail to amuse\n",
      "\n",
      "Chorus:\n",
      "Oh sparkling water, you're my delight\n",
      "With every sip, you make me feel so right\n",
      "You're like a party in my mouth\n",
      "I can't get enough, I'm hooked no doubt\n",
      "\n",
      "Bridge:\n",
      "Some may say you're just plain water\n",
      "But to me, you're so much more\n",
      "You bring a sparkle to my day\n",
      "In every single way\n",
      "\n",
      "Chorus:\n",
      "Oh sparkling water, you're my delight\n",
      "With every sip, you make me feel so right\n",
      "You're like a party in my mouth\n",
      "I can't get enough, I'm hooked no doubt\n",
      "\n",
      "Outro:\n",
      "So here's to you, my dear sparkling water\n",
      "You'll always be my go-to drink forever\n",
      "With your effervescence and refreshing taste\n",
      "You'll always have a special place."
     ]
    }
   ],
   "source": [
    "from langchain.llms import OpenAI\n",
    "\n",
    "llm = OpenAI(model=\"gpt-3.5-turbo-instruct\", temperature=0, max_tokens=512)\n",
    "for chunk in llm.stream(\"Write me a song about sparkling water.\"):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d81140f2-384b-4470-bf93-957013c6620b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
