Spanish spelling encodes both the syllable boundaries and the stress, so the library needs no dictionary: diphthongs, hiatuses and triphthongs, the silent u of qu and gu, the vocalic y, the h inside a diphthong and the consonant clusters give the syllables; the written accent, or the ending of the word when there is none, gives the stressed syllable. Adverbs in -mente and hyphenated compounds carry two stresses.

>>> from ests.syllables import stress_type, syllabify, word_stress, word_stresses

>>> syllabify("murciélago")
['mur', 'cié', 'la', 'go']

>>> syllabify("averiguáis")
['a', 've', 'ri', 'guáis']

>>> word_stress("construir"), stress_type("construir")
(1, 'aguda')

>>> word_stresses("fácilmente")
[0, 2]

More in the documentation.

The library allows extracting the following statistics from a text:

• the number of sentences
• the number of words
• the number of unique words
• the number of long words
• the number of complex words
• the number of simple words
• the number of monosyllable words
• the number of polysyllable words
• the number of characters
• the number of letters
• the number of spaces
• the number of syllables
• the number of punctuation marks and their distribution by type
• the distribution of words by the number of letters
• the distribution of words by the number of syllables

A complex word has three or more syllables and a long word seven or more letters, as the Spanish readability formulas count them. Any statistic can be printed in a readable form:

>>> from ests import BasicStats

>>> text = "Hay tres clases de mentiras: mentiras, malditas mentiras y estadísticas"
>>> BasicStats(text).print_stats()
     Statistic      |  Value
------------------------------
Sentences           |    1
Words               |    10
Unique words        |    8
Long words          |    5
Complex words       |    5
Simple words        |    5
Monosyllabic words  |    4
Polysyllabic words  |    6
Characters          |    71
Letters             |    60
Spaces              |    9
Syllables           |    23
Punctuation marks   |    2

More in the documentation.

The library allows counting the following readability metrics:

• Flesch reading ease with the coefficients of Szigriszt-Pazos or of Fernández Huerta
• Gutiérrez de Polini comprehensibility formula
• Crawford grade
• Legibilidad µ
• SOL grade, the SMOG index converted to Spanish
• LIX readability measure
• RIX readability measure

An interpretation layer works on top of the formulas: the band of a scale for the reading ease and for Legibilidad µ, a consensus grade as the median of the grade formulas, the school stage and the reader age of Spain, and reading time by the norms of Spanish-speaking readers.

The coefficients of the Flesch reading ease are selected by the preset argument: by default the fórmula de perspicuidad of Szigriszt-Pazos with the INFLESZ scale validated on texts for patients (general); the coefficients of Fernández Huerta with the bands of their author are available as classic.

>>> from pprint import pprint
>>> from ests import ReadabilityStats

>>> text = "Hay tres clases de mentiras: mentiras, malditas mentiras y estadísticas"
>>> rs = ReadabilityStats(text)

>>> pprint(rs.get_stats(), sort_dicts=False)
{'flesch_reading_easy': 53.545000000000016,
 'gutierrez_polini_index': 33.5,
 'crawford_grade': 5.812999999999999,
 'mu_index': 50.943396226415096,
 'sol_grade': 9.258359866374562,
 'lix': 60.0,
 'rix': 5.0,
 'consensus_grade': 9.0,
 'reading_time': 0.03597122302158273}

>>> rs.print_stats()
                   Metric                    |  Value
-------------------------------------------------------
Flesch reading ease (Szigriszt-Pazos)        |  53.55
Gutiérrez de Polini comprehensibility        |  33.50
Crawford grade                               |   5.81
Legibilidad µ                                |  50.94
SOL grade (SMOG for Spanish)                 |   9.26
LIX readability index                        |  60.00
RIX readability index                        |   5.00
Consensus grade                              |   9.00
Reading time (min)                           |   0.04

>>> rs.describe_level()
'algo difícil'

>>> rs.describe_grade()
'ESO (12-16 years)'

More in the documentation.

The library allows counting 32 lexical diversity metrics, among them:

• Type-Token Ratio and its variations: RTTR, CTTR, Herdan, Summer, Maas, Dugast
• Moving Average Type-Token Ratio and Mean Segmental Type-Token Ratio
• Measure of Textual Lexical Diversity and its moving-window variants MA-MTLD and MTLD-W
• Hypergeometric Distribution D
• Simpson's index, its reciprocal and the Gini-Simpson index
• the hapax index (Honoré's R), the measures of Yule, Herdan, Sichel, Michéa, Brunet, Dugast and Baayen
• Shannon entropy, evenness and perplexity
• the slope of Zipf's law, the Zipf-Mandelbrot fit and the exponent of Heaps' law

Any metric can be computed over windows of equal length, which is the standard way to compare texts of different lengths: the mean over the windows comes with a confidence interval.

>>> from ests import DiversityStats

>>> text = "Hay tres clases de mentiras: mentiras, malditas mentiras y estadísticas"
>>> ds = DiversityStats(text)

>>> ds.ttr, ds.mtld, ds.yule_k
(0.8, 14.000000000000004, 600.0)

>>> ds.frequency_spectrum
{1: 7, 3: 1}

>>> DiversityStats("La legibilidad de un texto depende de la longitud de sus oraciones y de sus "
...                "palabras. Las fórmulas clásicas miden esas dos magnitudes y las combinan en "
...                "un solo número. Ninguna de ellas mide la comprensión: miden la superficie "
...                "del texto.").windowed("ttr", window_len=10)
WindowStats(mean=0.875, std=0.1258305739211792, lower=0.6747754774664074, upper=1.0752245225335926, n_windows=4)

More in the documentation.

The library annotates a text with the parts of speech and the grammatical features of Universal Dependencies, as the Spanish models of spaCy give them, and counts them:

• the part of speech and fifteen features: case, definiteness, degree, gender, mood, numeral type, number, person, polarity, politeness, possessive, pronoun type, reflexive, tense and verb form
• the distribution of the words by the values of any feature, and the parse of the text word by word
• the markers of Spanish: the moods among the finite forms, the non-finite forms, ser against estar, the adverbs in -mente

>>> from ests import MorphStats

>>> ms = MorphStats("Si tuviera tiempo, leería el libro que me recomendaste ayer")

>>> ms.get_stats("mood", "tense", filter_none=True)
{'mood': {'Sub': 1, 'Cnd': 1, 'Ind': 1}, 'tense': {'Imp': 1, 'Pres': 1}}

>>> ms.tags[1]
'Mood=Sub|Number=Sing|Person=3|Tense=Imp|VerbForm=Fin'

>>> ms.explain_text("pos", "mood", filter_none=True)[1]
('tuviera', {'pos': 'VERB', 'mood': 'Sub'})

>>> MorphStats("Ella es alta pero hoy está cansada y habla lentamente").get_markers()["p_ser"]
0.5

The statistics need a spaCy model: a text is parsed with es_core_news_sm, and any other pipeline can be passed in nlp.

More in the documentation.

The library measures the dependency tree of Universal Dependencies and the constructions that the Spanish guides to clear language warn about:

• the complexity of the tree: dependency distances, depth, leaves and subtrees, valency of the finite verbs, coordination chains, clauses and subordinate clauses, modifiers per noun
• the constructions: the passive with ser and with se, the participial and the gerund clauses, the chains of de, the split predicates, the impersonal se, the words of negation, the ratio of nouns to verbs

>>> from ests import SyntaxStats

>>> text = ("La revisión de las cuentas fue realizada por el comité. "
...         "Se llevó a cabo la reforma sin que nadie hiciera mención de los problemas.")
>>> ss = SyntaxStats(text)

>>> ss.n_sents, ss.n_words
(2, 24)

>>> ss.p_passive, ss.noun_verb_ratio
(0.6666666666666666, 2.3333333333333335)

>>> ss.split_predicates
('llevó cabo', 'hiciera mención')

>>> round(ss.mean_dependency_distance, 2)
2.05

The statistics need a parse: a text is parsed with es_core_news_sm, and any other pipeline can be passed in nlp.

More in the documentation.

The library measures referential cohesion in the manner of Coh-Metrix and of its Spanish adaptation Coh-Metrix-Esp:

• the overlap of nouns, of arguments and of content words between adjacent sentences and between all pairs of sentences, binary and proportional
• givenness: pronouns, demonstratives and the content words whose lemma was already used
• temporal cohesion: the repetition of the tense and of the mood of the verbs of adjacent sentences
• the density of 255 Spanish discourse markers by class - causal, adversative, concessive, temporal, additive, conditional, reformulative - and by kind

>>> from ests import CohesionStats

>>> text = ("El informe fue aprobado por la comisión. Sin embargo, el informe no resuelve el problema. "
...         "Por lo tanto, la comisión aplazó la decisión.")
>>> cs = CohesionStats(text)

>>> cs.n_sents, cs.n_words
(3, 23)

>>> cs.noun_overlap_adjacent, round(cs.p_given, 3)
(0.5, 0.167)

>>> cs.c_connectors
{'por lo tanto': 1, 'sin embargo': 1}

>>> round(cs.connectors, 2), round(cs.connectors_causal, 2)
(86.96, 43.48)

The statistics need the annotation: a text is parsed with es_core_news_sm, and any other pipeline can be passed in nlp.

More in the documentation.

Every statistics class is also a component of a pipeline, so a text is annotated and measured in one pass and the statistics travel with the Doc:

>>> import ests
>>> import spacy

>>> nlp = spacy.load("es_core_news_sm")
>>> for factory in ("basic", "morph", "syntax"):
...     _ = nlp.add_pipe(f"ests_{factory}", name=factory, last=True)

>>> nlp.pipe_names[-3:]
['basic', 'morph', 'syntax']

>>> doc = nlp("El gato duerme en la ventana. Los niños juegan en el parque.")
>>> doc._.basic.n_words, doc._.morph.pos[:2], doc._.syntax.tree_depth
(12, ('DET', 'NOUN'), 2.0)

The factories are ests_basic, ests_readability, ests_diversity, ests_morph, ests_syntax and ests_cohesion; the name of the pipe is free and is what the extension is called.

More in the documentation.

• docs - project documentation
• ests:
  • basic_stats.py - basic text statistics
  • cohesion_stats.py - cohesion statistics
  • components.py - components of a spaCy pipeline
  • constants.py - constants of the Spanish language and of the metrics
  • diversity_stats.py - lexical diversity metrics
  • exceptions.py - library exceptions
  • extractors.py - tools for object extraction from a text
  • morph_stats.py - morphological statistics
  • readability_stats.py - readability metrics
  • syntax_stats.py - syntactic statistics
  • syllables.py - syllabification and stress
  • utils.py - helper tools
• tests - tests mirroring the package structure