TS POSTagger

TS POSTagger is a Turkish part-of-speech tagging library with a hybrid pipeline:

1. ts-tokenizer tokenizes is used to tokenize input data.
2. A bundled spaCy POS model predicts tags. No external model download is required.

The package exposes:

• a Python API: from ts_postagger import pos
• a CLI: ts-postagger

Installation

pip install ts-postagger

Requirements:

• Python >=3.11

The trained model is bundled with the package. No separate download step is required.

Quick Start

from ts_postagger import pos

tokens = pos("Defne'nin heyecanla beklediği #viyana yolculuğu bugün başladı.")

for token in tokens:
    print(token.text, token.pos)

Example output:

Defne'nin   PropN
heyecanla   Adv
beklediği   Adj
#viyana     Hashtag
yolculuğu   Noun
bugün       Adv
başladı     Verb
.           Punc

Python API

The main entrypoint is pos(text: str) -> list[TSToken].

Each returned TSToken has these fields:

Field	Description
text	Original surface form
lower	Turkish-aware lowercase form
token_type	Deterministic token class from ts-tokenizer
tag	Contextual grammatical prediction from the model
pos	Final output POS label

pos is the field you should use as the final annotation.

Minimal example

from ts_postagger import pos

text = pos("Bugün yeni ve güzel bir gün!")

for token in text:
    print(token.pos)

Convert results to dictionaries

TSToken is a dataclass, so standard dataclass helpers work:

from dataclasses import asdict

from ts_postagger import pos

tokens = pos("#YeniBilgi yayımlandı.")
rows = [asdict(token) for token in tokens]

for row in rows:
    print(row)

Example dictionary:

{
    "text": "#YeniBilgi",
    "lower": "#yenibilgi",
    "token_type": "Hashtag",
    "tag": "Noun",
    "pos": "Hashtag",
}

Empty input

from ts_postagger import pos

print(pos(""))

Output:

[]

Preserve XML lines for corpus output

XML tag lines are returned as structural tokens with token_type, tag, and pos set to "XML_Tag". Use token.text directly for those lines when writing CWB-style corpus output:

from ts_postagger import pos

tokens = pos('<text id="001" author="ts">\nBugün hava çok güzel.\n</text>')

for token in tokens:
    if token.token_type == "XML_Tag":
        print(token.text)
    else:
        print(f"{token.text}\t{token.lower}\t{token.pos}")

Output:

<text id="001" author="ts">
Bugün	bugün	Adv
hava	hava	Noun
çok	çok	Adv
güzel	güzel	Adj
.	.	Punc
</text>

Why token_type, tag, and pos are different

The library intentionally keeps multiple annotation layers.

For lexical tokens, the final output usually follows the POS model:

çalışmalar  Valid_Word  Noun  Noun
yayımlandı  Valid_Word  Verb  Verb

For structural or social-media tokens, the final output stays deterministic even when the model predicts a regular grammatical tag:

#YeniBilgi  Hashtag  Noun  Hashtag
@yeni  Mention  Noun  Mention
19.10.2026  Date  Num  Date
https://example.org  URL  Noun  URL

Meaning of each layer:

• token_type: deterministic label from the tokenizer
• tag: raw contextual prediction from the POS model
• pos: final POS output of TS POSTagger

Turkish-aware lowercasing

The lower field uses Turkish-aware lowercasing from ts-tokenizer. This eliminates problems with Python's built-in lower() function errors.

from ts_postagger import pos

tokens = pos("ISPARTA İSTANBUL")

for token in tokens:
    print(token.text, token.lower)

Output:

Isparta  ısparta
İSTANBUL  istanbul

lower is a lowercase surface form. It is not a lemma.

CLI

Installing the package also installs the ts-postagger command.

The CLI accepts either:

• a single positional text argument, or
• standard input

Default output format:

TOKEN<TAB>POS

Tag inline text

ts-postagger "Bugün yeni ve güzel bir gün!"

Example output:

Bugün	Adv
yeni	Adj
ve	Conj
güzel	Adj
bir	Det
gün	Noun
!	Punc

Lowercase only

ts-postagger -low "Bugün yeni ve güzel bir gün!"

Example output:

bugün
yeni
ve
güzel
bir
gün
!

Raw model tag

ts-postagger -tag "Bugün yeni ve güzel bir gün!"

Example output:

Bugün	Adv
yeni	Adj
ve	Conj
güzel	Adj
bir	Det
gün	Noun
!	Punc

Full output

ts-postagger -full "Bugün yeni ve güzel bir gün!"

Example output:

Bugün	bugün	Adv
yeni	yeni	Adj
ve	ve	Conj
güzel	güzel	Adj
bir	bir	Det
gün	gün	Noun
!	!	Punc

Columns:

TOKEN<TAB>LOWER<TAB>POS

Read from stdin

echo "Bugün yeni ve güzel bir gün!" | ts-postagger

For a file, pass the file content through standard input:

ts-postagger -full < test_sentence.txt

The positional argument is interpreted as text, not as a file path. XML tag lines are preserved as structural lines without POS columns, so CWB-style corpus markup can pass through the tagger:

<text id="001" author="ts">
Bugün	bugün	Adv
</text>

Run from a source checkout

python -m venv .venv
. .venv/bin/activate
python -m pip install -e .
ts-postagger -full < test_sentence.txt

You can also run the CLI module directly from the checkout:

python src/ts_postagger/cli.py -full < test_sentence.txt

Version

ts-postagger -V

Help

ts-postagger --help

Notes

• The package name for installation is ts-postagger.
• The Python import package is ts_postagger.
• The CLI command is ts-postagger.

Citation

If you use TS POSTagger in academic work, please cite the associated doctoral dissertation:

Sezer, T. (2025). Dizilerden birimlere: Bilişimsel dilbilim çerçevesinde bir birimlendirici tasarımı [Doctoral dissertation, Hacettepe University].

License

This project is licensed under the MIT License.