{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7960ce8a-859a-41f4-a886-0d1502ed1105",
   "metadata": {},
   "source": [
    "# SearchApi\n",
    "\n",
    "This notebook shows examples of how to use SearchApi to search the web. Go to [https://www.searchapi.io/](https://www.searchapi.io/) to sign up for a free account and get API key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "70871a99-ffee-47d7-8e02-82eb99971f28",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"SEARCHAPI_API_KEY\"] = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2e26a518-c41c-4d75-9a79-67602ca2ec43",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.utilities import SearchApiAPIWrapper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8c0977f3-c136-400a-8024-f4f00645b981",
   "metadata": {},
   "outputs": [],
   "source": [
    "search = SearchApiAPIWrapper()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f573767d-4144-4407-8149-5fdddab99c63",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Barack Hussein Obama II'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.run(\"Obama's first name?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f4f75ae-2e1e-42db-a991-3ac111029f56",
   "metadata": {},
   "source": [
    "## Using as part of a Self Ask With Search Chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "17a9b1ad-6e84-4949-8ebd-8c52f6b296e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"OPENAI_API_KEY\"] = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "cf8970a5-00e1-46bd-ba53-6a974eebbc10",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m Yes.\n",
      "Follow up: How old was Plato when he died?\u001b[0m\n",
      "Intermediate answer: \u001b[36;1m\u001b[1;3meighty\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mFollow up: How old was Socrates when he died?\u001b[0m\n",
      "Intermediate answer: \u001b[36;1m\u001b[1;3m| Socrates | \n",
      "| -------- | \n",
      "| Born | c. 470 BC Deme Alopece, Athens | \n",
      "| Died | 399 BC (aged approximately 71) Athens | \n",
      "| Cause of death | Execution by forced suicide by poisoning | \n",
      "| Spouse(s) | Xanthippe, Myrto | \n",
      "\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mFollow up: How old was Aristotle when he died?\u001b[0m\n",
      "Intermediate answer: \u001b[36;1m\u001b[1;3m62 years\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mSo the final answer is: Plato\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Plato'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.agents import AgentType, Tool, initialize_agent\n",
    "from langchain.llms.openai import OpenAI\n",
    "from langchain.utilities import SearchApiAPIWrapper\n",
    "\n",
    "llm = OpenAI(temperature=0)\n",
    "search = SearchApiAPIWrapper()\n",
    "tools = [\n",
    "    Tool(\n",
    "        name=\"Intermediate Answer\",\n",
    "        func=search.run,\n",
    "        description=\"useful for when you need to ask with search\",\n",
    "    )\n",
    "]\n",
    "\n",
    "self_ask_with_search = initialize_agent(\n",
    "    tools, llm, agent=AgentType.SELF_ASK_WITH_SEARCH, verbose=True\n",
    ")\n",
    "self_ask_with_search.run(\"Who lived longer: Plato, Socrates, or Aristotle?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc433d06-579b-45e5-a256-2bb30bbefb93",
   "metadata": {},
   "source": [
    "## Custom parameters\n",
    "\n",
    "SearchApi wrapper can be customized to use different engines like [Google News](https://www.searchapi.io/docs/google-news), [Google Jobs](https://www.searchapi.io/docs/google-jobs), [Google Scholar](https://www.searchapi.io/docs/google-scholar), or others which can be found in [SearchApi](https://www.searchapi.io/docs/google) documentation. All parameters supported by SearchApi can be passed when executing the query. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6d0b4411-780a-4dcf-91b6-f3544e31e532",
   "metadata": {},
   "outputs": [],
   "source": [
    "search = SearchApiAPIWrapper(engine=\"google_jobs\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "34e79449-6b33-4b45-9306-7e3dab1b8599",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Azure AI Engineer Be an XpanderCandidatar-meCandidatar-meCandidatar-me\\n\\nShare:\\n\\nAzure AI Engineer\\n\\nA área Digital Xperience da Xpand IT é uma equipa tecnológica de rápido crescimento que se concentra em tecnologias Microsoft e Mobile. A sua principal missão é fornecer soluções de software de alta qualidade que atendam às necessidades do utilizador final, num mundo tecnológico continuamente exigente e em ritmo acelerado, proporcionando a melhor experiência em termos de personalização, performance'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.run(\"AI Engineer\", location=\"Portugal\", gl=\"pt\")[0:500]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d414513d-f374-4af0-a129-e878d4311a1e",
   "metadata": {},
   "source": [
    "## Getting results with metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b16b7cd9-f0fe-4030-a36b-bbb52b19da18",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e8adb325-2ad0-4a39-9bc2-d220ec3a29be",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'search_metadata': {'id': 'search_qVdXG2jzvrlqTzayeYoaOb8A',\n",
      "                     'status': 'Success',\n",
      "                     'created_at': '2023-09-25T15:22:30Z',\n",
      "                     'request_time_taken': 3.21,\n",
      "                     'parsing_time_taken': 0.03,\n",
      "                     'total_time_taken': 3.24,\n",
      "                     'request_url': 'https://scholar.google.com/scholar?q=Large+Language+Models&hl=en',\n",
      "                     'html_url': 'https://www.searchapi.io/api/v1/searches/search_qVdXG2jzvrlqTzayeYoaOb8A.html',\n",
      "                     'json_url': 'https://www.searchapi.io/api/v1/searches/search_qVdXG2jzvrlqTzayeYoaOb8A'},\n",
      " 'search_parameters': {'engine': 'google_scholar',\n",
      "                       'q': 'Large Language Models',\n",
      "                       'hl': 'en'},\n",
      " 'search_information': {'query_displayed': 'Large Language Models',\n",
      "                        'total_results': 6420000,\n",
      "                        'page': 1,\n",
      "                        'time_taken_displayed': 0.06},\n",
      " 'organic_results': [{'position': 1,\n",
      "                      'title': 'ChatGPT for good? On opportunities and '\n",
      "                               'challenges of large language models for '\n",
      "                               'education',\n",
      "                      'data_cid': 'uthwmf2nU3EJ',\n",
      "                      'link': 'https://www.sciencedirect.com/science/article/pii/S1041608023000195',\n",
      "                      'publication': 'E Kasneci, K Seßler, S Küchemann, M '\n",
      "                                     'Bannert… - Learning and individual …, '\n",
      "                                     '2023 - Elsevier',\n",
      "                      'snippet': '… state of large language models and their '\n",
      "                                 'applications. We then highlight how these '\n",
      "                                 'models can be … With regard to challenges, '\n",
      "                                 'we argue that large language models in '\n",
      "                                 'education require …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '8166055256995715258',\n",
      "                                                    'total': 410,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=8166055256995715258&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '8166055256995715258',\n",
      "                                                    'total': 10,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=8166055256995715258&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:uthwmf2nU3EJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'edarxiv.org',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://edarxiv.org/5er8f/download?format=pdf'},\n",
      "                      'authors': [{'name': 'E Kasneci',\n",
      "                                   'id': 'bZVkVvoAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=bZVkVvoAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'K Seßler',\n",
      "                                   'id': 'MbMBoN4AAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=MbMBoN4AAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'S Küchemann',\n",
      "                                   'id': 'g1jX5QUAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=g1jX5QUAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'M Bannert',\n",
      "                                   'id': 'TjfQ8QkAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=TjfQ8QkAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 2,\n",
      "                      'title': 'Large language models in medicine',\n",
      "                      'data_cid': 'Ph9AwHTmhzAJ',\n",
      "                      'link': 'https://www.nature.com/articles/s41591-023-02448-8',\n",
      "                      'publication': 'AJ Thirunavukarasu, DSJ Ting, K '\n",
      "                                     'Elangovan… - Nature medicine, 2023 - '\n",
      "                                     'nature.com',\n",
      "                      'snippet': '… HuggingChat offers a free-to-access '\n",
      "                                 'chatbot with a similar interface to ChatGPT '\n",
      "                                 'but uses Large Language Model Meta AI '\n",
      "                                 '(LLaMA) as its backend model 30 . Finally, '\n",
      "                                 'cheap imitations of …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '3497017024792502078',\n",
      "                                                    'total': 25,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=3497017024792502078&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '3497017024792502078',\n",
      "                                                    'total': 3,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=3497017024792502078&hl=en&as_sdt=0,33'}},\n",
      "                      'authors': [{'name': 'AJ Thirunavukarasu',\n",
      "                                   'id': '3qb1AYwAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=3qb1AYwAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'DSJ Ting',\n",
      "                                   'id': 'KbrpC8cAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=KbrpC8cAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'K Elangovan',\n",
      "                                   'id': 'BE_lVTQAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=BE_lVTQAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 3,\n",
      "                      'title': 'Extracting training data from large language '\n",
      "                               'models',\n",
      "                      'data_cid': 'mEYsWK6bWKoJ',\n",
      "                      'link': 'https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting',\n",
      "                      'publication': 'N Carlini, F Tramer, E Wallace, M '\n",
      "                                     'Jagielski… - 30th USENIX Security …, '\n",
      "                                     '2021 - usenix.org',\n",
      "                      'snippet': '… language model trained on scrapes of the '\n",
      "                                 'public Internet, and are able to extract '\n",
      "                                 'hundreds of verbatim text sequences from the '\n",
      "                                 'model’… models are more vulnerable than '\n",
      "                                 'smaller models. …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '12274731957504198296',\n",
      "                                                    'total': 742,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=12274731957504198296&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '12274731957504198296',\n",
      "                                                    'total': 8,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=12274731957504198296&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:mEYsWK6bWKoJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:mEYsWK6bWKoJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'usenix.org',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://www.usenix.org/system/files/sec21-carlini-extracting.pdf'},\n",
      "                      'authors': [{'name': 'N Carlini',\n",
      "                                   'id': 'q4qDvAoAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=q4qDvAoAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'F Tramer',\n",
      "                                   'id': 'ijH0-a8AAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=ijH0-a8AAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'E Wallace',\n",
      "                                   'id': 'SgST3LkAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=SgST3LkAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'M Jagielski',\n",
      "                                   'id': '_8rw_GMAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=_8rw_GMAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 4,\n",
      "                      'title': 'Emergent abilities of large language models',\n",
      "                      'data_cid': 'hG0iVOrOguoJ',\n",
      "                      'link': 'https://arxiv.org/abs/2206.07682',\n",
      "                      'publication': 'J Wei, Y Tay, R Bommasani, C Raffel, B '\n",
      "                                     'Zoph… - arXiv preprint arXiv …, 2022 - '\n",
      "                                     'arxiv.org',\n",
      "                      'snippet': 'Scaling up language models has been shown to '\n",
      "                                 'predictably improve performance and sample '\n",
      "                                 'efficiency on a wide range of downstream '\n",
      "                                 'tasks. This paper instead discusses an …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '16898296257676733828',\n",
      "                                                    'total': 621,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=16898296257676733828&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '16898296257676733828',\n",
      "                                                    'total': 12,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=16898296257676733828&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:hG0iVOrOguoJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:hG0iVOrOguoJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'arxiv.org',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://arxiv.org/pdf/2206.07682.pdf?trk=cndc-detail'},\n",
      "                      'authors': [{'name': 'J Wei',\n",
      "                                   'id': 'wA5TK_0AAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=wA5TK_0AAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'Y Tay',\n",
      "                                   'id': 'VBclY_cAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=VBclY_cAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'R Bommasani',\n",
      "                                   'id': 'WMBXw1EAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=WMBXw1EAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'C Raffel',\n",
      "                                   'id': 'I66ZBYwAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=I66ZBYwAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'B Zoph',\n",
      "                                   'id': 'NL_7iTwAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=NL_7iTwAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 5,\n",
      "                      'title': 'A survey on evaluation of large language '\n",
      "                               'models',\n",
      "                      'data_cid': 'ZYohnzOz-XgJ',\n",
      "                      'link': 'https://arxiv.org/abs/2307.03109',\n",
      "                      'publication': 'Y Chang, X Wang, J Wang, Y Wu, K Zhu… - '\n",
      "                                     'arXiv preprint arXiv …, 2023 - arxiv.org',\n",
      "                      'snippet': '… 3.1 Natural Language Processing Tasks … '\n",
      "                                 'the development of language models, '\n",
      "                                 'particularly large language models, was to '\n",
      "                                 'enhance performance on natural language '\n",
      "                                 'processing tasks, …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '8717195588046785125',\n",
      "                                                    'total': 31,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=8717195588046785125&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '8717195588046785125',\n",
      "                                                    'total': 3,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=8717195588046785125&hl=en&as_sdt=0,33'},\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:ZYohnzOz-XgJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'arxiv.org',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://arxiv.org/pdf/2307.03109'},\n",
      "                      'authors': [{'name': 'X Wang',\n",
      "                                   'id': 'Q7Ieos8AAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=Q7Ieos8AAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'J Wang',\n",
      "                                   'id': 'YomxTXQAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=YomxTXQAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'Y Wu',\n",
      "                                   'id': 'KVeRu2QAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=KVeRu2QAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'K Zhu',\n",
      "                                   'id': 'g75dFLYAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=g75dFLYAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 6,\n",
      "                      'title': 'Evaluating large language models trained on '\n",
      "                               'code',\n",
      "                      'data_cid': '3tNvW3l5nU4J',\n",
      "                      'link': 'https://arxiv.org/abs/2107.03374',\n",
      "                      'publication': 'M Chen, J Tworek, H Jun, Q Yuan, HPO '\n",
      "                                     'Pinto… - arXiv preprint arXiv …, 2021 - '\n",
      "                                     'arxiv.org',\n",
      "                      'snippet': '… We introduce Codex, a GPT language model '\n",
      "                                 'finetuned on publicly available code from '\n",
      "                                 'GitHub, and study its Python code-writing '\n",
      "                                 'capabilities. A distinct production version '\n",
      "                                 'of Codex …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '5664817468434011102',\n",
      "                                                    'total': 941,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=5664817468434011102&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '5664817468434011102',\n",
      "                                                    'total': 2,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=5664817468434011102&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:3tNvW3l5nU4J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:3tNvW3l5nU4J:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'arxiv.org',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://arxiv.org/pdf/2107.03374.pdf?trk=public_post_comment-text'},\n",
      "                      'authors': [{'name': 'M Chen',\n",
      "                                   'id': '5fU-QMwAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=5fU-QMwAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'J Tworek',\n",
      "                                   'id': 'ZPuESCQAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=ZPuESCQAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'Q Yuan',\n",
      "                                   'id': 'B059m2EAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=B059m2EAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 7,\n",
      "                      'title': 'Large language models in machine translation',\n",
      "                      'data_cid': 'sY5m_Y3-0Y4J',\n",
      "                      'link': 'http://research.google/pubs/pub33278.pdf',\n",
      "                      'publication': 'T Brants, AC Popat, P Xu, FJ Och, J Dean '\n",
      "                                     '- 2007 - research.google',\n",
      "                      'snippet': '… the benefits of largescale statistical '\n",
      "                                 'language modeling in ma… trillion tokens, '\n",
      "                                 'resulting in language models having up to '\n",
      "                                 '300 … is inexpensive to train on large data '\n",
      "                                 'sets and approaches the …',\n",
      "                      'type': 'PDF',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '10291286509313494705',\n",
      "                                                    'total': 737,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=10291286509313494705&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '10291286509313494705',\n",
      "                                                    'total': 31,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=10291286509313494705&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:sY5m_Y3-0Y4J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:sY5m_Y3-0Y4J:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'research.google',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'http://research.google/pubs/pub33278.pdf'},\n",
      "                      'authors': [{'name': 'FJ Och',\n",
      "                                   'id': 'ITGdg6oAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=ITGdg6oAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'J Dean',\n",
      "                                   'id': 'NMS69lQAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=NMS69lQAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 8,\n",
      "                      'title': 'A watermark for large language models',\n",
      "                      'data_cid': 'BlSyLHT4iiEJ',\n",
      "                      'link': 'https://arxiv.org/abs/2301.10226',\n",
      "                      'publication': 'J Kirchenbauer, J Geiping, Y Wen, J '\n",
      "                                     'Katz… - arXiv preprint arXiv …, 2023 - '\n",
      "                                     'arxiv.org',\n",
      "                      'snippet': '… To derive this watermark, we examine what '\n",
      "                                 'happens in the language model just before it '\n",
      "                                 'produces a probability vector. The last '\n",
      "                                 'layer of the language model outputs a vector '\n",
      "                                 'of logits l(t). …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '2417017327887471622',\n",
      "                                                    'total': 104,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=2417017327887471622&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '2417017327887471622',\n",
      "                                                    'total': 4,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=2417017327887471622&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:BlSyLHT4iiEJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:BlSyLHT4iiEJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'arxiv.org',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://arxiv.org/pdf/2301.10226.pdf?curius=1419'},\n",
      "                      'authors': [{'name': 'J Kirchenbauer',\n",
      "                                   'id': '48GJrbsAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=48GJrbsAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'J Geiping',\n",
      "                                   'id': '206vNCEAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=206vNCEAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'Y Wen',\n",
      "                                   'id': 'oUYfjg0AAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=oUYfjg0AAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'J Katz',\n",
      "                                   'id': 'yPw4WjoAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=yPw4WjoAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 9,\n",
      "                      'title': 'ChatGPT and other large language models are '\n",
      "                               'double-edged swords',\n",
      "                      'data_cid': 'So0q8TRvxhYJ',\n",
      "                      'link': 'https://pubs.rsna.org/doi/full/10.1148/radiol.230163',\n",
      "                      'publication': 'Y Shen, L Heacock, J Elias, KD Hentel, B '\n",
      "                                     'Reig, G Shih… - Radiology, 2023 - '\n",
      "                                     'pubs.rsna.org',\n",
      "                      'snippet': '… Large Language Models (LLMs) are deep '\n",
      "                                 'learning models trained to understand and '\n",
      "                                 'generate natural language. Recent studies '\n",
      "                                 'demonstrated that LLMs achieve great success '\n",
      "                                 'in a …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '1641121387398204746',\n",
      "                                                    'total': 231,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=1641121387398204746&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '1641121387398204746',\n",
      "                                                    'total': 3,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=1641121387398204746&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:So0q8TRvxhYJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'authors': [{'name': 'Y Shen',\n",
      "                                   'id': 'XaeN2zgAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=XaeN2zgAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'L Heacock',\n",
      "                                   'id': 'tYYM5IkAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=tYYM5IkAAAAJ&hl=en&oi=sra'}]},\n",
      "                     {'position': 10,\n",
      "                      'title': 'Pythia: A suite for analyzing large language '\n",
      "                               'models across training and scaling',\n",
      "                      'data_cid': 'aaIDvsMAD8QJ',\n",
      "                      'link': 'https://proceedings.mlr.press/v202/biderman23a.html',\n",
      "                      'publication': 'S Biderman, H Schoelkopf… - '\n",
      "                                     'International …, 2023 - '\n",
      "                                     'proceedings.mlr.press',\n",
      "                      'snippet': '… large language models, we prioritize '\n",
      "                                 'consistency in model … out the most '\n",
      "                                 'performance from each model. For example, we '\n",
      "                                 '… models, as it is becoming widely used for '\n",
      "                                 'the largest models, …',\n",
      "                      'inline_links': {'cited_by': {'cites_id': '14127511396791067241',\n",
      "                                                    'total': 89,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cites=14127511396791067241&as_sdt=5,33&sciodt=0,33&hl=en'},\n",
      "                                       'versions': {'cluster_id': '14127511396791067241',\n",
      "                                                    'total': 3,\n",
      "                                                    'link': 'https://scholar.google.com/scholar?cluster=14127511396791067241&hl=en&as_sdt=0,33'},\n",
      "                                       'related_articles_link': 'https://scholar.google.com/scholar?q=related:aaIDvsMAD8QJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                       'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:aaIDvsMAD8QJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",
      "                      'resource': {'name': 'mlr.press',\n",
      "                                   'format': 'PDF',\n",
      "                                   'link': 'https://proceedings.mlr.press/v202/biderman23a/biderman23a.pdf'},\n",
      "                      'authors': [{'name': 'S Biderman',\n",
      "                                   'id': 'bO7H0DAAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=bO7H0DAAAAAJ&hl=en&oi=sra'},\n",
      "                                  {'name': 'H Schoelkopf',\n",
      "                                   'id': 'XLahYIYAAAAJ',\n",
      "                                   'link': 'https://scholar.google.com/citations?user=XLahYIYAAAAJ&hl=en&oi=sra'}]}],\n",
      " 'related_searches': [{'query': 'large language models machine',\n",
      "                       'highlighted': ['machine'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=1&q=large+language+models+machine&qst=ib'},\n",
      "                      {'query': 'large language models pruning',\n",
      "                       'highlighted': ['pruning'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=2&q=large+language+models+pruning&qst=ib'},\n",
      "                      {'query': 'large language models multitask learners',\n",
      "                       'highlighted': ['multitask learners'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=3&q=large+language+models+multitask+learners&qst=ib'},\n",
      "                      {'query': 'large language models speech recognition',\n",
      "                       'highlighted': ['speech recognition'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=4&q=large+language+models+speech+recognition&qst=ib'},\n",
      "                      {'query': 'large language models machine translation',\n",
      "                       'highlighted': ['machine translation'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=5&q=large+language+models+machine+translation&qst=ib'},\n",
      "                      {'query': 'emergent abilities of large language models',\n",
      "                       'highlighted': ['emergent abilities of'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=6&q=emergent+abilities+of+large+language+models&qst=ir'},\n",
      "                      {'query': 'language models privacy risks',\n",
      "                       'highlighted': ['privacy risks'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=7&q=language+models+privacy+risks&qst=ir'},\n",
      "                      {'query': 'language model fine tuning',\n",
      "                       'highlighted': ['fine tuning'],\n",
      "                       'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=8&q=language+model+fine+tuning&qst=ir'}],\n",
      " 'pagination': {'current': 1,\n",
      "                'next': 'https://scholar.google.com/scholar?start=10&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                'other_pages': {'2': 'https://scholar.google.com/scholar?start=10&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '3': 'https://scholar.google.com/scholar?start=20&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '4': 'https://scholar.google.com/scholar?start=30&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '5': 'https://scholar.google.com/scholar?start=40&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '6': 'https://scholar.google.com/scholar?start=50&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '7': 'https://scholar.google.com/scholar?start=60&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '8': 'https://scholar.google.com/scholar?start=70&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '9': 'https://scholar.google.com/scholar?start=80&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",
      "                                '10': 'https://scholar.google.com/scholar?start=90&q=Large+Language+Models&hl=en&as_sdt=0,33'}}}\n"
     ]
    }
   ],
   "source": [
    "search = SearchApiAPIWrapper(engine=\"google_scholar\")\n",
    "results = search.results(\"Large Language Models\")\n",
    "pprint.pp(results)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
