temporal-normalization-spacy 1.0.0

A spaCy plugin for identifying and parsing historical data in Romanian texts

Temporal Expressions Normalization spaCy (TeNs)

Temporal Expressions Normalization spaCy (TeNs) is a powerful pipeline component for spaCy that seamlessly identifies and parses date entities in text. It leverages the Temporal Normalization Framework to recognize a wide variety of date formats using an extensive set of regular expressions (RegEx), ensuring robust and adaptable date extraction across diverse textual sources.

Unlike conventional solutions that primarily focus on well-structured date formats, TeNs excels in handling real-world text by identifying not only standard date representations but also abbreviated, informal, or even misspelled temporal expressions. This makes it particularly effective for processing noisy or unstructured data, such as historical records, user-generated content, and scanned documents with OCR inaccuracies.

Moreover, TeNs is designed to integrate seamlessly into existing NLP pipelines, allowing for enhanced temporal information processing in tasks such as event extraction, timeline construction, and knowledge graph population. By providing a flexible and accurate approach to temporal data normalization, it significantly improves the quality and reliability of date-related information extracted from text.

Table I. Types of temporal expressions which can be processed

Type of Temporal Expressions	Examples of Temporal Expressions*
dates	YMD: 1881-08-31; 1857 mai 10; etc. DMY: 09.11.1518; 1 noiembrie 1624; etc. MY: ianuarie 632; etc.
timespans	centuries: s:; sc; se.; sec; sec.; secol; secolele; secolul; sex. millenniums: mil; mil.; mileniul; mileniului; mileniile
years	77; 78; 1652; [1873]; aproximativ 1834; cca. 1420; etc.

* The values are mentioned in the reference language – Romanian language

Getting started

To integrate TeNs into spaCy pipelines you need the following:

Prerequisites

• Python 3.x
• JRE 11+
• spaCy 3.x
• py4j 0.10.9.9
• langdetect 1.0.9

1. Install python packages:

    pip install -r requirements.txt

2. Install JDK 11 or OpenJDK 11 (or newer versions)

Install

    pip install temporal-spacy

Supported languages

• Romanian

Usage

---

Importing Modules & Defining Constants

import subprocess

import spacy

from temporal_normalization.commons.print_utils import console
from temporal_normalization.index import create_normalized_component, TemporalNormalization  ## noqa: F401

LANG = "ro"
MODEL = "ro_core_news_sm"
TEXT_RO = ("Sec al II-lea a.ch. a fost o perioadă de mari schimbări. "
           "În secolul XX, tehnologia a avansat semnificativ. "
           "Sec. 21 este adesea asociat cu globalizarea rapidă.")

Adding the Component to spaCy Pipeline

    # Display a warning if the language of the text is not Romanian.
console.lang_warning(TEXT_RO, target_lang=LANG)

try:
    # Load the spaCy model if it has already been downloaded
    nlp = spacy.load(MODEL)
except OSError:
    console.warning(f'Started downloading {MODEL}...')
    # Download the Romanian model if it wasn't already downloaded
    subprocess.run(["python", "-m", "spacy", "download", MODEL])
    # Load the spaCy model
    nlp = spacy.load(MODEL)

# Add "temporal_normalization" component to the spaCy pipeline
nlp.add_pipe("temporal_normalization", last=True)

Processing Text with the Pipeline

doc = nlp(TEXT_RO)

# Display NLP-specific linguistic annotations
console.tokens_table(doc)
print()

Accessing the Parsed Temporal Expressions

# Display information about the identified and normalized dates in the text.
for entity in doc.ents:
    for edge in entity._.normalized.edges:
        print(edge.serialize())
        print()

    print("Periods:")
    for period in entity._.normalized.periods:
        print(period.serialize("\t"))
        print()

Result

First Sentence

Start time:
	Matched value: Sec al II-lea a.ch.
	Matched Type: century
	Normalized label: 2nd century BC
	DBpedia uri: https://dbpedia.org/page/2nd_century_BC
End time:
	Matched value: Sec al II-lea a.ch.
	Matched Type: century
	Normalized label: 2nd century BC
	DBpedia uri: https://dbpedia.org/page/2nd_century_BC

Periods:
	Matched value: Sec al II-lea a.ch.
	Matched Type: century
	Normalized label: 1st millennium BC
	DBpedia uri: https://dbpedia.org/page/1st_millennium_BC

	Matched value: Sec al II-lea a.ch.
	Matched Type: century
	Normalized label: 2nd century BC
	DBpedia uri: https://dbpedia.org/page/2nd_century_BC

Second Sentence

Start time:
	Matched value: secolul XX
	Matched Type: century
	Normalized label: 20th century
	DBpedia uri: https://dbpedia.org/page/20th_century
End time:
	Matched value: secolul XX
	Matched Type: century
	Normalized label: 20th century
	DBpedia uri: https://dbpedia.org/page/20th_century

Periods:
	Matched value: secolul XX
	Matched Type: century
	Normalized label: 2nd millennium
	DBpedia uri: https://dbpedia.org/page/2nd_millennium

	Matched value: secolul XX
	Matched Type: century
	Normalized label: 20th century
	DBpedia uri: https://dbpedia.org/page/20th_century

Third Sentence

Start time:
	Matched value: Sec. 21
	Matched Type: century
	Normalized label: 21st century
	DBpedia uri: https://dbpedia.org/page/21st_century
End time:
	Matched value: Sec. 21
	Matched Type: century
	Normalized label: 21st century
	DBpedia uri: https://dbpedia.org/page/21st_century

Periods:
	Matched value: Sec. 21
	Matched Type: century
	Normalized label: 3rd millennium
	DBpedia uri: https://dbpedia.org/page/3rd_millennium

	Matched value: Sec. 21
	Matched Type: century
	Normalized label: 21st century
	DBpedia uri: https://dbpedia.org/page/21st_century

Publications

ECAI 2021: The Power of Regular Expressions in Recognizing Dates and Epochs (2021)

@inproceedings{9515139,
  author={Dorobăț, Ilie Cristian and Posea, Vlad},
  booktitle={2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI)}, 
  title={The Power of Regular Expressions in Recognizing Dates and Epochs}, 
  year={2021},
  pages={1-3},
  doi={10.1109/ECAI52376.2021.9515139}}