TS POSTagger

TS POSTagger is a Turkish part-of-speech tagging library with a hybrid pipeline:

1. ts-tokenizer produces token boundaries and deterministic token classes.
2. A bundled spaCy POS model predicts contextual grammatical tags.
3. A resolver preserves structural token types such as hashtags, mentions, URLs, dates, and punctuation when they are more informative than a generic grammatical label.

The package exposes:

• a Python API: from ts_postagger import pos
• a CLI: ts-postagger

Installation

pip install ts-postagger

Requirements:

• Python >=3.11

The trained model is bundled with the package. No separate download step is required.

Quick Start

from ts_postagger import pos

tokens = pos("Defne'nin heyecanla beklediği #viyana yolculuğu bugün başladı.")

for token in tokens:
    print(token.text, token.pos)

Example output:

Defne'nin   PropN
heyecanla   Adv
beklediği   Adj
#viyana     Hashtag
yolculuğu   Noun
bugün       Adv
başladı     Verb
.           Punc

Python API

The main entrypoint is pos(text: str) -> list[TSToken].

Each returned TSToken has these fields:

Field	Description
text	Original surface form
lower	Turkish-aware lowercase form
token_type	Deterministic token class from ts-tokenizer
tag	Contextual grammatical prediction from the model
pos	Final output POS label

pos is the field you should use as the final annotation.

Minimal example

from ts_postagger import pos

text = pos("Bugün yeni ve güzel bir gün!")

for token in text:
    print(token.pos)

Convert results to dictionaries

TSToken is a dataclass, so standard dataclass helpers work:

from dataclasses import asdict

from ts_postagger import pos

tokens = pos("#YeniBilgi yayımlandı.")
rows = [asdict(token) for token in tokens]

for row in rows:
    print(row)

Example dictionary:

{
    "text": "#YeniBilgi",
    "lower": "#yenibilgi",
    "token_type": "Hashtag",
    "tag": "Noun",
    "pos": "Hashtag",
}

Empty input

from ts_postagger import pos

print(pos(""))

Output:

[]

Why token_type, tag, and pos are different

The library intentionally keeps multiple annotation layers.

For lexical tokens, the final output usually follows the POS model:

çalışmalar  Valid_Word  Noun  Noun
yayımlandı  Valid_Word  Verb  Verb

For structural or social-media tokens, the final output stays deterministic even when the model predicts a regular grammatical tag:

#YeniBilgi  Hashtag  Noun  Hashtag
@yeni  Mention  Noun  Mention
19.10.2026  Date  Num  Date
https://example.org  URL  Noun  URL

Meaning of each layer:

• token_type: deterministic label from the tokenizer
• tag: raw contextual prediction from the POS model
• pos: final POS output of TS POSTagger

Turkish-aware lowercasing

The lower field uses Turkish-aware lowercasing from ts-tokenizer.

from ts_postagger import pos

tokens = pos("IĞDIR İSTANBUL")

for token in tokens:
    print(token.text, token.lower)

Output:

IĞDIR  ığdır
İSTANBUL  istanbul

lower is a lowercase surface form. It is not a lemma.

CLI

Installing the package also installs the ts-postagger command.

The CLI accepts either:

• a single positional text argument, or
• standard input

Default output format:

TOKEN<TAB>POS

Tag inline text

ts-postagger "Bugün yeni ve güzel bir gün!"

Example output:

Bugün	Adv
yeni	Adj
ve	Conj
güzel	Adj
bir	Det
gün	Noun
!	Punc

Lowercase only

ts-postagger -low "Bugün yeni ve güzel bir gün!"

Example output:

bugün
yeni
ve
güzel
bir
gün
!

Raw model tag

ts-postagger -tag "Bugün yeni ve güzel bir gün!"

Example output:

Bugün	Adv
yeni	Adj
ve	Conj
güzel	Adj
bir	Det
gün	Noun
!	Punc

Full output

ts-postagger --full "Bugün yeni ve güzel bir gün!"

Example output:

Bugün	bugün	Adv
yeni	yeni	Adj
ve	ve	Conj
güzel	güzel	Adj
bir	bir	Det
gün	gün	Noun
!	!	Punc

Columns:

TOKEN<TAB>LOWER<TAB>POS

Read from stdin

echo "Bugün yeni ve güzel bir gün!" | ts-postagger

Version

ts-postagger -V

Help

ts-postagger --help

Notes

• The package name for installation is ts-postagger.
• The Python import package is ts_postagger.
• The CLI command is ts-postagger.

Citation

If you use TS POSTagger in academic work, please cite the associated doctoral dissertation:

Sezer, T. (2025). Dizilerden birimlere: Bilişimsel dilbilim çerçevesinde bir birimlendirici tasarımı [Doctoral dissertation, Hacettepe University].

License

This project is licensed under the MIT License.