gallicaGetter 0.0.5

A tool for pulling word occurrence ('n-gram') data from the Gallica periodical archive.

gallicaGetter

This tool wraps a few endpoints from the Gallica API to allow multi-threaded data retrieval with support for generators. Go ahead, explore some archived French periodicals, and a few international editions too!

I developed this tool into a graphing app similar to Google's n-gram viewer for books.

I owe inspiration for part of the API integration to queries written by the team at Gallicagram.

Current endpoints are:

• 'sru' -- for a term, get the number of occurrences over a time range or fetch all the periodical issues the term appears in.
• 'content' -- occurrence context and page numbers
• 'papers' -- paper titles and publishing range data, from their Gallica codes
• 'issues' -- years published for a given paper (used internally in papers)

Installation

pip install gallicaGetter

SRU quickstart

Build the wrapper object using the connect() factory:

import gallicaGetter

sruWrapper = gallicaGetter.connect('sru')

Then, retrieve records or counts using get().

get(terms, generate=False, **params)

PARAMETERS:

• terms: a string, or list of strings, to search for.
• startDate: lower year boundary for the search.
• endDate: upper year boundary for the search.
• codes: string paper codes to restrict the search. Can be found in the URL of a Gallica periodical's page.
• grouping: 'year', 'month', or 'all'.
  • year: returns a count of occurrences for each year in the range.
  • month: returns a count of occurrences per month in the range.
  • all: fetches metadata (a Record object) for each occurrence.
• generate: if True, returns a generator object. Otherwise, returns a list of results.
  • If you're using the 'all' grouping, a generator can help reduce memory usage for large requests. Still working out the behavior.
• linkTerm: a string that restricts the search to occurrences within its proximity.
• linkDistance: proximity distance, in words.
• numRecords: limit number of records to return.

SRU Examples

Retrieve the number of occurrences of "Victor Hugo" across the Gallica archive from 1800 to 1900, by year, running 30 requests in parallel.

import gallicaGetter

sruWrapper = gallicaGetter.connect('sru', numWorkers=30)

records = sruWrapper.get(
    terms="Victor Hugo",
    startDate="1800",
    endDate="1900",
    grouping="year"
)

for record in records:
    print(record.getRow())

Retrieve all issues that mention "Brazza" from 1890 to 1900.

import gallicaGetter

sruWrapper = gallicaGetter.connect('sru')

records = sruWrapper.get(
    terms="Brazza",
    startDate="1890",
    endDate="1900",
    grouping="all"
)

for record in records:
    print(record.getRow())

Retrieve all occurrences of "Brazza" within 10 words of "Congo" in the paper "Le Temps" from 1890 to 1900.

import gallicaGetter

sruWrapper = gallicaGetter.connect('sru')

records = sruWrapper.get(
    terms="Brazza",
    startDate="1890",
    endDate="1900",
    linkTerm="Congo",
    linkDistance=10,
    grouping="all",
    codes="cb34431794k"
)

for record in records:
    print(record.getRow())

Retrieve all issues mentioning "Paris" in the papers "Le Temps" and "Le Figaro" from 1890 to 1900, using a generator.

import gallicaGetter

sruWrapper = gallicaGetter.connect('sru')

recordGenerator = sruWrapper.get(
    terms="Paris",
    startDate="1890",
    endDate="1900",
    grouping="all",
    codes=["cb34431794k", "cb34355551z"],
    generate=True
)

for i in range(10):
    print(next(recordGenerator).getRow())

Content example

This wrapper pairs best with an SRU fetch since the ark code for an issue is in the SRU response.

Retrieve text context for all occurrences of "guerre" in an issue of the Figaro.

import gallicaGetter

contentWrapper = gallicaGetter.connect('content')

data = contentWrapper.get(
    ark='bpt6k270178t',
    term='guerre',
)

for numOccurrences, pages in data:
    print(numOccurrences, pages)

Papers example

Retrieve metadata from a Gallica periodical's code. Example for "Le Temps":

import gallicaGetter

papersWrapper = gallicaGetter.connect('papers')

metadata = papersWrapper.get('cb34431794k')

for record in metadata:
    print(record.getRow())
    print(record.isContinuous())

Issues example

The papers wrapper calls this internally, but it might be useful. For a paper code, retrieve all years with at least one issue archived on Gallica.

import gallicaGetter

issuesWrapper = gallicaGetter.connect('issues')

years = issuesWrapper.get('cb34431794k')