Learn, vectorize, and annotate Construction Grammars
Computational Construction Grammar, or c2xg, is two things:
(1) A Python package for the unsupervised learning of CxG representations along with tools for vectorizing these representations for computational tasks
(2) A discovery-device grammar that learns falsifiable and replicable CxGs from observed unannotated text data
Why CxGs? Constructions are grammatical entities that allow the straight-forward quantification of linguistic structure.
pip install c2xg
Environment and Dependencies
This package is meant to run in Python 3.5 with a number of dependencies. The easiest way to maintain the necessary environment is to use Anaconda Python: https://www.continuum.io/downloads
The package can be installed within Anaconda using the command:
conda install c2xg
This makes it easier to maintain the necessary environment. The package works with the dependency versions listed below. It will likely work with older versions of some packages but has not been tested with them. For example, older versions of numexpr have been known to cause issues withs pandas and may lead to lost candidates.
C2xG has two main classes:
c2xg.Parameters loads and initializes the settings needed for running C2xG; these values are set in a file
c2xg.Grammar contains the grammar resources used across all stages of the package
Initialize the package with the following commands:
Parameters = c2xg.Parameters("filename")
The parameters class takes as input a string indicating the name of the parameters file. Now, run the API using the following template, where Parameters is an initialized c2xg.Parameter object:
All functions in the API take a c2xg.Parameters object as an argument. The c2xg.Grammar object can be passed to each function or, if not passed, loaded from file.
Each function in the API takes a Parameters object and either creates the Grammar object or loads it from the file specified in the parameters.
Umbrella function for entire learning pipeline (from learn_mwes to learn_constructions).
Individual Learning Functions
Use GenSim to create the dictionary of semantic representations needed for c2xg.
Use RDRPOS Tagger Dependency to learn a new pos-tagging model.
Use c2xg to learn a dictionary of idioms (lexical constructions).
Use c2xg to learn a constituency grammar.
Use c2xg to learn a full Construction Grammar with lexical, MWE, semantic, and constituent representations.
Prepare to use TF-IDF weighting during feature extraction.
Produce a CSV file of association measures for sequences of a given length and types of representation
Tokenize, pos-tag, mark emojis, and convert to CoNLL format.
Get indexes of representation types.
Getcandidate sequences from input files (covers MWEs, Constituents, and Constructions).
Get vector of association values for each candidate.
Get vector of CxG usage for input files.
Get examples of predicted constituents by type. (*Not stable in v 0.2)
Get examples of each predicted construction. (*Not stable in v 0.2)
(1) Begin a Python interpreter
(2) Import the package:
(3) Initialize the parameters object:
Parameters = c2xg.Parameters("filename")
(4a) Run the API, loading grammar objects from disk:
(4b) Run the API, initializing and then passing grammar objects:
Grammar = c2xg.Grammar()
This section describes the input formats for the different components.
(1) Creating Semantic Dictionary
Input: Unannotated text, one sentence per line. Tokenization and emoji identification are performed on each line.
(2) Creating Models of Grammar and Usage
Input: Annotated: CoNLL format of tab-separate fields [Word-Form, Lemma, POS, Index].
Use <s:ID> to assign ids to documents.
Input: Unannotated: Plain text with line breaks for documents / sentences as desired.
[In both cases, each line is assumed to be a "text" or the containing unit of analysis; instances can be separated by the "|" character for aggregation]
(3) Extracting Feature Vectors
Input with Meta-Data: Field:Value,Field:Value\tText
Input without Meta-Data: Plain text with line breaks (\n) for documents or sentences depending on the level of analysis.
Given a language-specific CxG, the get_vectors and learn_usage functions convert that grammar into a vector representation of texts or sentences (i.e., one unit per line in the input files). There are two modes and three quantification methods for creating vectors:
vector_scope = "Full": Constructions and lexical / POS / semantic features
vector_scope = "Lexical": Only lexical features
vector_scope = "CxG": Only construction features
relative_freq == True: Quantify using the relative frequency of the feature in given sentence or text (as negative logarithms)
relative_freq == False: Quantify using unadjusted raw frequency of the feature
use_centroid == True: Extract vectors with centriod normalization learned using learn_usage. This is functionally equivalent to TF-IDF scaling
Centroid normalization first finds the probability of a given feature in the background corpus. This is stored after running learn_usage in separate centroid_df models for the full grammar and for the lexical-only features. During extraction, if centroids are used for representation, this is converted into negative logarithms of the inverted joint probability of each feature occuring as many times as it does in a message.
*Note: Feature extraction is stable in v 0.1 but not in v 0.2
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size c2xg-0.21-py3-none-any.whl (13.6 MB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|