{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b0ed136e-6983-4893-ae1b-b75753af05f8",
   "metadata": {},
   "source": [
    "# Google Drive\n",
    "\n",
    "This notebook covers how to retrieve documents from `Google Drive`.\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "1. Create a Google Cloud project or use an existing project\n",
    "1. Enable the [Google Drive API](https://console.cloud.google.com/flows/enableapi?apiid=drive.googleapis.com)\n",
    "1. [Authorize credentials for desktop app](https://developers.google.com/drive/api/quickstart/python#authorize_credentials_for_a_desktop_application)\n",
    "1. `pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib`\n",
    "\n",
    "## Retrieve the Google Docs\n",
    "\n",
    "By default, the `GoogleDriveRetriever` expects the `credentials.json` file to be `~/.credentials/credentials.json`, but this is configurable using the `GOOGLE_ACCOUNT_FILE` environment variable. \n",
    "The location of `token.json` uses the same directory (or use the parameter `token_path`). Note that `token.json` will be created automatically the first time you use the retriever.\n",
    "\n",
    "`GoogleDriveRetriever` can retrieve a selection of files with some requests. \n",
    "\n",
    "By default, If you use a `folder_id`, all the files inside this folder can be retrieved to `Document`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35b94a93-97de-4af8-9cca-de9ffb7930c3",
   "metadata": {},
   "source": [
    "You can obtain your folder and document id from the URL:\n",
    "* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
    "* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
    "\n",
    "The special value `root` is for your personal home."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2216c83f-68e4-4d2f-8ea2-5878fb18bbe7",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-09T10:45:59.795842403Z",
     "start_time": "2023-05-09T10:45:59.445262457Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain_googledrive.retrievers import GoogleDriveRetriever\n",
    "\n",
    "folder_id = \"root\"\n",
    "# folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'\n",
    "\n",
    "retriever = GoogleDriveRetriever(\n",
    "    num_results=2,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa339ca0-f478-440c-ba80-0e5f41a19ce1",
   "metadata": {},
   "source": [
    "By default, all files with these MIME types can be converted to `Document`.\n",
    "\n",
    "- `text/text`\n",
    "- `text/plain`\n",
    "- `text/html`\n",
    "- `text/csv`\n",
    "- `text/markdown`\n",
    "- `image/png`\n",
    "- `image/jpeg`\n",
    "- `application/epub+zip`\n",
    "- `application/pdf`\n",
    "- `application/rtf`\n",
    "- `application/vnd.google-apps.document` (GDoc)\n",
    "- `application/vnd.google-apps.presentation` (GSlide)\n",
    "- `application/vnd.google-apps.spreadsheet` (GSheet)\n",
    "- `application/vnd.google.colaboratory` (Notebook colab)\n",
    "- `application/vnd.openxmlformats-officedocument.presentationml.presentation` (PPTX)\n",
    "- `application/vnd.openxmlformats-officedocument.wordprocessingml.document` (DOCX)\n",
    "\n",
    "It's possible to update or customize this. See the documentation of `GoogleDriveRetriever`.\n",
    "\n",
    "But, the corresponding packages must be installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9dadec48",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install unstructured"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f3b6aa0-b45d-4e37-8c50-5bebe70fdb9d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-09T10:46:00.990310466Z",
     "start_time": "2023-05-09T10:45:59.798774595Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "retriever.get_relevant_documents(\"machine learning\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ff33817-8619-4897-8742-2216b9934d2a",
   "metadata": {},
   "source": [
    "You can customize the criteria to select the files. A set of predefined filter are proposed:\n",
    "\n",
    "| Template                                 | Description                                                           |\n",
    "| --------------------------------------   | --------------------------------------------------------------------- |\n",
    "| `gdrive-all-in-folder`                   | Return all compatible files from a `folder_id`                        |\n",
    "| `gdrive-query`                           | Search `query` in all drives                                          |\n",
    "| `gdrive-by-name`                         | Search file with name `query`                                         |\n",
    "| `gdrive-query-in-folder`                 | Search `query` in `folder_id` (and sub-folders in `_recursive=true`)  |\n",
    "| `gdrive-mime-type`                       | Search a specific `mime_type`                                         |\n",
    "| `gdrive-mime-type-in-folder`             | Search a specific `mime_type` in `folder_id`                          |\n",
    "| `gdrive-query-with-mime-type`            | Search `query` with a specific `mime_type`                            |\n",
    "| `gdrive-query-with-mime-type-and-folder` | Search `query` with a specific `mime_type` and in `folder_id`         |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9977c712-9659-4959-b508-f59cc7d49d44",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retriever = GoogleDriveRetriever(\n",
    "    template=\"gdrive-query\",  # Search everywhere\n",
    "    num_results=2,  # But take only 2 documents\n",
    ")\n",
    "for doc in retriever.get_relevant_documents(\"machine learning\"):\n",
    "    print(\"---\")\n",
    "    print(doc.page_content.strip()[:60] + \"...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5a0f3ef-26fb-4a5c-85f0-5aba90b682b1",
   "metadata": {},
   "source": [
    "Else, you can customize the prompt with a specialized `PromptTemplate`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0bbebde-0487-4d20-9d77-8070e4f0e0d6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "\n",
    "retriever = GoogleDriveRetriever(\n",
    "    template=PromptTemplate(\n",
    "        input_variables=[\"query\"],\n",
    "        # See https://developers.google.com/drive/api/guides/search-files\n",
    "        template=\"(fullText contains '{query}') \"\n",
    "        \"and mimeType='application/vnd.google-apps.document' \"\n",
    "        \"and modifiedTime > '2000-01-01T00:00:00' \"\n",
    "        \"and trashed=false\",\n",
    "    ),\n",
    "    num_results=2,\n",
    "    # See https://developers.google.com/drive/api/v3/reference/files/list\n",
    "    includeItemsFromAllDrives=False,\n",
    "    supportsAllDrives=False,\n",
    ")\n",
    "for doc in retriever.get_relevant_documents(\"machine learning\"):\n",
    "    print(f\"{doc.metadata['name']}:\")\n",
    "    print(\"---\")\n",
    "    print(doc.page_content.strip()[:60] + \"...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b6fed29-1666-452e-b677-401613270388",
   "metadata": {},
   "source": [
    "## Use Google Drive 'description' metadata\n",
    "\n",
    "Each Google Drive has a `description` field in metadata (see the *details of a file*).\n",
    "Use the `snippets` mode to return the description of selected files.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "342dbe12-ed83-40f4-8957-0cc8c4609542",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retriever = GoogleDriveRetriever(\n",
    "    template=\"gdrive-mime-type-in-folder\",\n",
    "    folder_id=folder_id,\n",
    "    mime_type=\"application/vnd.google-apps.document\",  # Only Google Docs\n",
    "    num_results=2,\n",
    "    mode=\"snippets\",\n",
    "    includeItemsFromAllDrives=False,\n",
    "    supportsAllDrives=False,\n",
    ")\n",
    "retriever.get_relevant_documents(\"machine learning\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
