cellar-extractor 1.1.3

Library for extracting cellar data

Cellar extractor

This library contains two functions to get cellar case law data from eurlex.

Version

Python 3.9 onwards *

Tests

Contributors

Pranav Bapat

Piotr Lewandowski

shashankmc

gijsvd

venvis

How to install?

pip install cellar-extractor

What are the functions?

1. get_cellar
Gets all the ECLI data from the eurlex sparql endpoint and saves them in the CSV or JSON format, in-memory or as a saved file.

2. get_cellar_extra
Gets all the ECLI data from the eurlex sparql endpoint, and on top of that scrapes the eurlex websites to acquire the full text, keywords, case law directory code and eurovoc identifiers. If the user does have an eurlex account with access to the eurlex webservices, he can also pass his webservices login credentials to the method, in order to extract data about works citing work and works being cited by work. The full text is returned as a JSON file, rest of data as a CSV. Can be in-memory or as saved files.
3. get_nodes_and_edges_lists
Gets 2 list objects, one for the nodes and edges of the citations within the passed dataframe. Allows the creation of a network graph of the citations. Can only be returned in-memory.
4. filter_subject_matter
Returns a dataframe of cases only containing a certain phrase in the column containing the subject of cases.
5. Analyzer
A class whose instance(declaration) when called returns a list of the all the text contained within the operative part for each European Court of Justice (CJEU, formerly known as European Court of Justice (ECJ)) judgement (English only).
6. Writing
A class which writes the text for the operative part for each European Case law case(En-English only) into csv,json and txt files(Generated upon initialization).
the Writing class has three functions :


• to_csv() - Writes the operative part along with celex id into a csv file
• to_json() - Writes the operative part along with celex id into a json file
• to_txt() - Writes the operative part along with celex id into a txt file

What are the parameters?

1. get_cellar
Parameters:
• max_ecli: int, optional, default 100
Maximum number of ECLIs to retrieve.
• sd: date, optional, default '2022-05-01'
The start last modification date (yyyy-mm-dd).
• ed: date, optional, default current date
The end last modification date (yyyy-mm-dd).
• save_file: ['y', 'n'],optional, default 'y'
Save data in a data folder, or return in-memory.
• file_format: ['csv', 'json'],optional, default 'csv'
Returns the data as a JSON/dictionary, or as a CSV/Pandas Dataframe object.
2. get_cellar_extra
• max_ecli: int, optional, default 100
Maximum number of ECLIs to retrieve.
• sd: date, optional, default '2022-05-01'
The start last modification date (yyyy-mm-dd).
• ed: date, optional, default current date
The end last modification date (yyyy-mm-dd).
• save_file: ['y', 'n'],optional, default 'y'
Save the full text of cases as JSON file / return as a dictionary and save the rest of the data as a CSV file / return as a Pandas Dataframe object.
• threads: int ,optional, default 10
Extracting the additional data takes a lot of time. The use of multi-threading can cut down this time. Even with this, the method may take a couple of minutes for a couple of hundred cases. A maximum number of 10 recommended, as this method may also affect the device's internet connection.
• username: string, optional, default empty string
The username to the eurlex webservices.
• password: string, optional, default empty string
The password to the eurlex webservices.

3. get_nodes_and_edges_lists
• df: DataFrame object, required, default None
DataFrame of cellar metadata acquired from the get_cellar_extra method with eurlex webservice credentials passed. This method will only work on dataframes with citations data.
• only_local: boolean, optional, default False
Flag for nodes and edges generation. If set to True, the network created will only include nodes and edges between cases exclusively inside the given dataframe.
4. filter_subject_matter
• df: DataFrame object, required, default None
DataFrame of cellar metadata acquired from any of the cellar extraction methods listed above.
• phrase: string, required, default None
The phrase which has to be present in the subject matter of cases. Case insensitive.
5. Analyzer
• celex id: str, required
• Pass as a constructor upon initializing the class
6. Writing
• celex id: str, required
• Pass as a constructor upon initializing the class

Examples

import cellar_extractor as cell

Below are examples for in-file saving:

cell.get_cellar(save_file='y', max_ecli=200, sd='2022-01-01', file_format='csv')
cell.get_cellar_extra(max_ecli=100, sd='2022-01-01', threads=10)

Below are examples for in-memory saving:

df = cell.get_cellar(save_file='n', file_format='csv', sd='2022-01-01', max_ecli=1000)
df,json = cell.get_cellar_extra(save_file='n', max_ecli=100, sd='2022-01-01', threads=10)

Create a callback of the instance of the class initiated and pass a list as it's value.

import cellar_extractor as cell
instance=cell.Analyzer(celex_id:str)
output_list=instance()
print(output_list) # prints operative part of the Case as a list

The Writing Class also takes a celex id , upon initializing the class , through the means of the constructor and writes the content of its operative part into different files , depending on the function called

import cellar_extractor as cell
instance=cell.Writing(celex_id:str)
output=instance.to_csv()#for csv
output=instance.to_txt()#for txt
output=instance.to_json()#for json