Skip to main content

Corpus of Annual Reports in Japan

Project description

CoARiJ: Corpus of Annual Reports in Japan

PyPI version Build Status codecov

We organized Japanese financial reports to encourage applying NLP techniques to financial analytics.

Dataset

You can download dataset by command line tool.

pip install coarij

Please refer the usage by -- (using fire).

coarij --

Example command.

# Download raw file version dataset of 2014.
coarij download --kind F --year 2014

# Extract business.overview_of_result part of TIS.Inc (sec code=3626).
coarij parse business.overview_of_result --sec_code 3626

# Tokenize text by Janome (`janome` or `sudachi` is supported).
pip install janome
coarij tokenize --tokenizer janome

# Show tokenized result (words are separated by \t).
head -n 5 data/processed/2014/docs/S100552V_business_overview_of_result_tokenized.txt
1       【      業績    等      の      概要    】
(       1       )               業績
当      連結    会計    年度    における        我が国  経済    は      、     消費    税率    引上げ  に      伴う    駆け込み        需要    の      反動   や      海外    景気    動向    に対する        先行き  懸念    等      から   弱い    動き    も      見      られ    まし    た      が      、      企業   収益    の      改善    等      により  全体  ...

You can use Ledger to select your necessary file from overall CoARiJ dataset.

from coarij.storage import Storage


storage = Storage("your/data/directory")
ledger = storage.download_ledger()
collected = ledger.collect(edinet_code="E00021")

Raw dataset file

The corpora are separated to each financial years.

fiscal_year Raw file version (F) Text extracted version (E)
2014 .zip (9.3GB) .zip (269.9MB)
2015 .zip (9.8GB) .zip (291.1MB)
2016 .zip (10.2GB) .zip (334.7MB)
2017 .zip (9.1GB) .zip (309.4MB)
2018 .zip (10.5GB) .zip (260.9MB)

Statistics

fiscal_year number_of_reports has_csr_reports has_financial_data has_stock_data
2014 3,724 92 3,583 3,595
2015 3,870 96 3,725 3,751
2016 4,066 97 3,924 3,941
2017 3,578 89 3,441 3,472
2018 3,513 70 2,893 3,413

Content

Raw file version (--kind F)

The structure of dataset is following.

chakki_esg_financial_{year}.zip
└──{year}
     ├── documents.csv
     └── docs/

docs includes XBRL and PDF file.

  • XBRL file of annual reports (files are retrieved from [EDINET]).
  • PDF file of CSR reports (additional content).

documents.csv has metadata like following.

  • edinet_code: E0000X
  • filer_name: XXX株式会社
  • fiscal_year: 201X
  • fiscal_period: FY
  • doc_path: docs/S000000X.xbrl
  • csr_path: docs/E0000X_201X_JP_36.pdf

Text extracted version (--kind E)

Text extracted version includes txt files that match each part of an annual report.
The extracted parts are defined at edinet-python.

chakki_esg_financial_{year}_extracted.zip
└──{year}
     ├── documents.csv
     └── docs/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for coarij, version 0.2.1
Filename, size File type Python version Upload date Hashes
Filename, size coarij-0.2.1.tar.gz (10.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page