CoARiJ: Corpus of Annual Reports in Japan

We organized Japanese financial reports to encourage applying NLP techniques to financial analytics.

Dataset

The corpora are separated to each financial years.

master version.

• financial data is from 決算短信情報.
  • We use non-cosolidated data if it exist.
• stock data is from 月間相場表（内国株式）.
  • close is fiscal period end and open is 1 year before of it.

Past release

• v1.0

Statistics

File structure

Raw file version (--kind F)

The structure of dataset is following.

chakki_esg_financial_{year}.zip
└──{year}
     ├── documents.csv
     └── docs/

docs includes XBRL and PDF file.

• XBRL file of annual reports (files are retrieved from EDINET).
• PDF file of CSR reports (additional content).

documents.csv has metadata like following. Please refer the detail at Wiki.

• edinet_code: E0000X
• filer_name: XXX株式会社
• fiscal_year: 201X
• fiscal_period: FY
• doc_path: docs/S000000X.xbrl
• csr_path: docs/E0000X_201X_JP_36.pdf

Text extracted version (--kind E)

Text extracted version includes txt files that match each part of an annual report.
The extracted parts are defined at xbrr.

chakki_esg_financial_{year}_extracted.zip
└──{year}
     ├── documents.csv
     └── docs/

Tool

You can download dataset by command line tool.

pip install coarij

Please refer the usage by -- (using fire).

coarij --

Example command.

# Download raw file version dataset of 2014.
coarij download --kind F --year 2014

# Extract business.overview_of_result part of TIS.Inc (sec code=3626).
coarij extract business.overview_of_result --sec_code 3626

# Tokenize text by Janome (`janome` or `sudachi` is supported).
pip install janome
coarij tokenize --tokenizer janome

# Show tokenized result (words are separated by \t).
head -n 5 data/processed/2014/docs/S100552V_business_overview_of_result_tokenized.txt
1       【      業績    等      の      概要    】
(       1       )               業績
当      連結    会計    年度    における        我が国  経済    は      、     消費    税率    引上げ  に      伴う    駆け込み        需要    の      反動   や      海外    景気    動向    に対する        先行き  懸念    等      から   弱い    動き    も      見      られ    まし    た      が      、      企業   収益    の      改善    等      により  全体  ...

If you want to download latest dataset, please specify --version master when download the data.

• About the parsable part, please refer the xbrr.

You can use Ledger to select your necessary file from overall CoARiJ dataset.

from coarij.storage import Storage


storage = Storage("your/data/directory")
ledger = storage.get_ledger()
collected = ledger.collect(edinet_code="E00021")