o ȳgx?@sdZddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z m Z mZmZmZdefddZGd d d eZGd d d e ZdS) aaUtility for using SearxNG meta search API. SearxNG is a privacy-friendly free metasearch engine that aggregates results from `multiple search engines `_ and databases and supports the `OpenSearch `_ specification. More details on the installation instructions `here. <../../integrations/searx.html>`_ For the search API refer to https://docs.searxng.org/dev/search_api.html Quick Start ----------- In order to use this utility you need to provide the searx host. This can be done by passing the named parameter :attr:`searx_host ` or exporting the environment variable SEARX_HOST. Note: this is the only required parameter. Then create a searx search instance like this: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # when the host starts with `http` SSL is disabled and the connection # is assumed to be on a private network searx_host='http://self.hosted' search = SearxSearchWrapper(searx_host=searx_host) You can now use the ``search`` instance to query the searx API. Searching --------- Use the :meth:`run() ` and :meth:`results() ` methods to query the searx API. Other methods are available for convenience. :class:`SearxResults` is a convenience wrapper around the raw json result. Example usage of the ``run`` method to make a search: .. code-block:: python s.run(query="what is the best search engine?") Engine Parameters ----------------- You can pass any `accepted searx search API `_ parameters to the :py:class:`SearxSearchWrapper` instance. In the following example we are using the :attr:`engines ` and the ``language`` parameters: .. code-block:: python # assuming the searx host is set as above or exported as an env variable s = SearxSearchWrapper(engines=['google', 'bing'], language='es') Search Tips ----------- Searx offers a special `search syntax `_ that can also be used instead of passing engine parameters. For example the following query: .. code-block:: python s = SearxSearchWrapper("langchain library", engines=['github']) # can also be written as: s = SearxSearchWrapper("langchain library !github") # or even: s = SearxSearchWrapper("langchain library !gh") In some situations you might want to pass an extra string to the search query. For example when the `run()` method is called by an agent. The search suffix can also be used as a way to pass extra parameters to searx or the underlying search engines. .. code-block:: python # select the github engine and pass the search suffix s = SearchWrapper("langchain library", query_suffix="!gh") s = SearchWrapper("langchain library") # select github the conventional google search syntax s.run("large language models", query_suffix="site:github.com") *NOTE*: A search suffix can be defined on both the instance and the method level. The resulting query will be the concatenation of the two with the former taking precedence. See `SearxNG Configured Engines `_ and `SearxNG Search Syntax `_ for more details. Notes ----- This wrapper is based on the SearxNG fork https://github.com/searxng/searxng which is better maintained than the original Searx project and offers more features. Public searxNG instances often use a rate limiter for API usage, so you might want to use a self hosted instance and disable the rate limiter. If you are self-hosting an instance you can customize the rate limiter for your own network as described `here `_. For a list of public SearxNG instances see https://searx.space/ N)AnyDictListOptional)get_from_dict_or_env) BaseModel ConfigDictField PrivateAttrmodel_validatorreturncCs dddS)Nenjson)languageformatrrrf/var/www/html/chatdoc2/venv/lib/python3.10/site-packages/langchain_community/utilities/searx_search.py_get_default_paramss rcsfeZdZUdZdZeed<deffdd Zdefdd Ze de fd d Z e de fd d Z Z S) SearxResultsz,Dict like wrapper around search api results._datadatacs t|}t|||_dS)zATake a raw result from Searx and make it into a dict like object.N)rloadssuper__init____dict__)selfr json_data __class__rrrs   zSearxResults.__init__r cCs|jS)z$Text representation of searx result.)rrrrr__str__szSearxResults.__str__cC |dS)zGSilence mypy for accessing this field. :meta private: resultsgetr rrrr#s zSearxResults.resultscCr")z#Helper accessor on the json result.answersr$r rrrr&s zSearxResults.answers)__name__ __module__ __qualname____doc__rstr__annotations__rr!propertyrr#r& __classcell__rrrrrs  rc@seZdZUdZeZeed<dZe ed<dZ e ed<e e dZeed<d Zeeed <gZeee ed <gZeee ed <dZee ed <dZeed<d Zeeed<eddededefddZeddZdedefddZ dedefddZ! d(de d eee d eee d ee dede f dd Z" d)de d eee d ee dede f d!d"Z# d(de d#ed eee d eee d ee dedeefd$d%Z$ d)de d#ed eee d ee dedeef d&d'Z%d S)*SearxSearchWrapperaWrapper for Searx API. To use you need to provide the searx host by passing the named parameter ``searx_host`` or exporting the environment variable ``SEARX_HOST``. In some situations you might want to disable SSL verification, for example if you are running searx locally. You can do this by passing the named parameter ``unsecure``. You can also pass the host url scheme as ``http`` to disable SSL. Example: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://localhost:8888") Example with SSL disabled: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # note the unsecure parameter is not needed if you pass the url scheme as # http searx = SearxSearchWrapper(searx_host="http://localhost:8888", unsecure=True) _resultr searx_hostFunsecure)default_factoryparamsNheadersengines categories query_suffix k aiosessionbefore)modevaluesr cCs|di}t}i|||d<|d}|r!d||dd<|d}|r1d||dd<t|dd}|dsItd|d d |}n |d rRd |d <||d<|S)z?Validate that custom searx params are merged with default ones.r4r6,r7r1 SEARX_HOSThttpzRWarning: missing the url scheme on host ! assuming secure https:// zhttps://zhttp://Tr2)r%rjoinr startswithprint)clsr> user_paramsdefaultr6r7r1rrrvalidate_paramss*       z"SearxSearchWrapper.validate_paramsforbid)extracCs@tj|j|j||j d}|jstd|jt|j}||_ |S)zActual request to searx API.r5r4verifySearx API returned an error: ) requestsr%r1r5r2ok ValueErrortextrr0)rr4 raw_resultresrrr_searx_api_querys  z#SearxSearchWrapper._searx_api_queryc sV|jskt4IdHR}|j|d}|jrd|d<|j|jfi|4IdH }|js2td|j t | IdH}||_ WdIdHn 1IdHsNwYWdIdH|S1IdHsdwY|S|jj|j|j||j d4IdH!}|jstd|j t | IdH}||_ WdIdH|S1IdHswY|S)N)r5r4FsslrNrL) r;aiohttp ClientSessionr5r2r%r1rPrQrRrr0)rr4sessionkwargsresponseresultrrr_asearx_api_query sB *   z$SearxSearchWrapper._asearx_api_queryqueryrZc Ksd|i}i|j||}|jr"t|jdkr"|dd|j7<t|tr7t|dkr7|dd|7<t|trIt|dkrId||d<t|tr[t|dkr[d||d<||}t|jdkrn|jd} | St|j dkrddd |j d |j D} | Sd } | S) aoRun query through Searx API and parse results. You can pass any other params to the searx query API. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: str: The result of the query. Raises: ValueError: If an error occurred with the query. Example: This will make a query to the qwant engine: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://my.searx.host") searx.run("what is the weather in France ?", engine="qwant") # the same result can be achieved using the `!` syntax of searx # to select the engine using `query_suffix` searx.run("what is the weather in France ?", query_suffix="!qwant") qrrBr?r6r7 cSg|]}|ddqScontentrr$.0rrrr dz*SearxSearchWrapper.run..NNo good search result found) r4r8len isinstancer+listrCrUr&r#r:) rr^r6r7r8rZ_paramsr4rTtoretrrrrun%s((   zSearxSearchWrapper.runc sd|i}i|j||}|jr#t|jdkr#|dd|j7<t|tr8t|dkr8|dd|7<t|trJt|dkrJd||d<||IdH}t|jdkr`|jd}|St|j dkryddd |j d|j D}|Sd }|S) z Asynchronously version of `run`.r_rrBr?r6Nr`cSrarbr$rdrrrrgrhz+SearxSearchWrapper.arun..ri) r4r8rjrkr+rlrCr]r&r#r:) rr^r6r8rZrmr4rTrnrrrarunjs&   zSearxSearchWrapper.arun num_resultsc Ksd|i}i|j||}|jr"t|jdkr"|dd|j7<t|tr7t|dkr7|dd|7<t|trIt|dkrId||d<t|tr[t|dkr[d||d<||jd|} t| dkrpdd igSd d | DS) a$Run query through Searx API and returns the results with metadata. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. num_results: Limit the number of results to return. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: Dict with the following keys: { snippet: The description of the result. title: The title of the result. link: The link to the result. engines: The engines used for the result. category: Searx category of the result. } r_rrBr?r6r7NResultNo good Search Result was foundcS4g|]}|dd|d|d|d|ddqSrcrtitleurlr6category)snippetrvlinkr6rxr$rer\rrrrg z.SearxSearchWrapper.results..) r4r8rjrkr+rlrCrUr#) rr^rqr6r7r8rZrmr4r#rrrr#s"  zSearxSearchWrapper.resultsc sd|i}i|j||}|jr#t|jdkr#|dd|j7<t|tr8t|dkr8|dd|7<t|trJt|dkrJd||d<||IdHjd|}t|dkrbddigSd d |DS) zdAsynchronously query with json results. Uses aiohttp. See `results` for more info. r_rrBr?r6NrrrscSrtrur$r{rrrrgr|z/SearxSearchWrapper.aresults..) r4r8rjrkr+rlrCr]r#) rr^rqr6r8rZrmr4r#rrraresultss    zSearxSearchWrapper.aresults)NNr)Nr)&r'r(r)r*r r0rr,r1r+r2boolr rr4dictr5rr6rr7r8r:intr;rr classmethodrrIr model_configrUr]rorpr#r}rrrrr/s       H  '   = r/)r*rtypingrrrrrWrOlangchain_core.utilsrpydanticrrr r r rrrr/rrrrs