Skip to main content

A client for communicating with the Google Search Appliance.

Project description

canonicalwebteam.gsa: Python GSA client

.. image::
:alt: build status

A client library for the `Google Search Appliance <>`_, to make retrieving search results in Python easier.


This module is in PyPi as :code:`canonicalwebteam.gsa`. You should be able to install it simply with:

.. code:: bash

pip install canonicalwebteam.gsa


This is a basic client for querying a Google Search Appliance.

Making queries

You can query the GSA using the :code:`search` method.

.. code:: python

search_client = GSAClient(base_url="")

first_ten_results ="hello world")

first_thirty_results ="hello world", num=30)

results_twenty_to_forty =
"hello world", start=20, num=20

This will set the `q <>`_,
`start <>`_ (default: 0) and
`num <>`_ (default: 10) and
`lr <>`_ (default: '') parameters.
No other `search parameters <>`_,
will be provided, so they will all fall back to their defaults.

The returned results object will attempt to map each of the GSA's
`standard result XML tags <>`_
into a more readable format:

.. code:: python

'estimated_total_results': int, # "M": GSA's estimate, see below
'document_filtering': bool, # "FI": Is filtering enabled?
'next_url': str, # "NU": GSA URL for querying the next set of results, if available
'previous_url': str, # "PU": Ditto for previous set of results
'items': [
'index': int, # "R[N]": The number of this result in the index of all results
'url': str, # "U": The URL of the resulting page
'encoded_url': str, # "UE": The above URL, encoded
'title': str, # "T": The page title
'relevancy': int, # "RK": How relevant is this result to the query? From 0 to 10
'appliance_id': str, # "ENT_SOURCE": The serial number of the GSA
'summary': str, # "S": Summary text for this result
'language': str, # "LANG": The language of the page
'details': {} # "FS": Name:value pairs of any extra info
'link_supported': bool, # "L": “link:” special query term is supported,
'cache': { # "C": Dictionary, or "None" if cache is not available
'size': str, # "C[SZ]": Human readable size of cached page
'cache_id': str, # "C[CID]": ID of document in GSA's cache
'encoding': str # "C[ENC]": The text encoding of the cached page

Filtering by domain or language

You can filter your search results by specifying specific domains or a
`specific language <>`_.

.. code:: python

english_results ="hello world", language="lang_en")
non_english_results ="hello world", language="-lang_en")
domain_specific_results =
"hello world",
domains=["", ""]

*NB:* If no search results are found with the specified ``language``, the GSA will fall back to returning any results it finds in all languages.

Getting accurate totals

At the time of writing, the Google Search Appliance will return an "estimate" of
the total number of results with each query, but this estimate is usually wildly
inaccurate, sometimes out by more than a factor of 10! This is true even with
`rc <>`_

With the :code:`total_results` method, the client will attempt to request results
990 - 1000. This will usually result in the GSA returning the last page of
results, which allows us to find the actual total number of results.

.. code:: python

total = search_client.total_results("hello world", domains=[], language='')

Django view

To simplify usage of the GSA client with Django, a Django view is included
with this module.


At the minimum, need to provide the :code:`SEARCH_SERVER_URL` setting to tell the view
where to find the GSA:

.. code:: python

SEARCH_SERVER_URL = '' # Required: GSA location
SEARCH_DOMAINS = [''] # Optional: By default, limit results to this set of domains
SEARCH_LANGUAGE = 'lang_zh-CN' # Optional: By default, limit results to this language

from canonicalwebteam.gsa.views import SearchView
urlpatterns += [url(r'^search/?$', SearchView.as_view(template_name="search.html"))]

This view will then be available to be queried:

- :code:``
- :code:`` (overrides :code:`SEARCH_DOMAINS`)
- :code:`` (exclude results in Chinese, overrides :code:`SEARCH_LANGUAGE`)

After retrieving search results, the view will pass the context object to the specified :code:`template_name` (in this case :code:`search.html`).

The context object will be structured as follows:

.. code:: python

'query': str, # The value of the `q` parameters passed to the view
'limit': int, # The value of the `limit` parameter, or the default of 10
'offset': int, # The value of the `offset` parameter, or the default of 0
'error': None|str, # None, or a description of the error if one occurred
'results': {
'items': [], # The list of items as returned from the GSAClient (see above)
'total': int, # The exact total number of results available
'start': int, # The index of the first result in the set
'end': int, # The index of the last result in the set
'next_offset': int|None, # The offset for the next page of results, if available
'previous_offset': int|None, # The offset for the previous page of results, if available
'last_page_offset': int, # The offset for the last page of results
'last_page': int, # The final page number (calculated from "limit" and "total")
'current_page': int, # The current page number (calculated from "limit" and "end")
'penultimate_page': int # The second-to-last page

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

canonicalwebteam.gsa-2.0.1.tar.gz (6.3 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page