Skip to main content

Pointwise Hilbert窶鉄chmidt Independence Criterion (PHSIC)

Compute co-occurrence between two objects utilizing similarities.

For example, given consistent sentence pairs:

X Y
They had breakfast at the hotel. They are full now.
They had breakfast at ten. I'm full.
She had breakfast with her friends. She felt happy.
They had breakfast with their friends at the Japanese restaurant. They felt happy.
He have trouble with his homework. He cries.
I have trouble associating with others. I cry.

PHSIC can give high scores to consistent pairs in terms of the given pairs:

X Y score
They had breakfast at the hotel. They are full now. 0.1134
They had breakfast at an Italian restaurant. They are stuffed now. 0.0023
I have dinner. I have dinner again. 0.0023

Installation

$ pip install phsic

This will install phsic command to your environment:

$ phsic --help

Basic Usage

Download pre-trained wordvecs (e.g. fasttext):

$ wget https://s3-us-west-1.amazonaws.com/fasttext-vectors/crawl-300d-2M.vec.zip
$ unzip crawl-300d-2M.vec.zip

Prepare dataset:

$ TAB="$(printf '\t')"
$ cat << EOF > train.txt
They had breakfast at the hotel.${TAB}They are full now.
They had breakfast at ten.${TAB}I'm full.
She had breakfast with her friends.${TAB}She felt happy.
They had breakfast with their friends at the Japanese restaurant.${TAB}They felt happy.
He have trouble with his homework.${TAB}He cries.
I have trouble associating with others.${TAB}I cry.
EOF
$ cut -f 1 train.txt > train_X.txt
$ cut -f 2 train.txt > train_Y.txt
$ cat << EOF > test.txt
They had breakfast at the hotel.${TAB}They are full now.
They had breakfast at an Italian restaurant.${TAB}They are stuffed now.
I have dinner.${TAB}I have dinner again.
EOF
$ cut -f 1 test.txt > test_X.txt
$ cut -f 2 test.txt > test_Y.txt

Then, train and predict:

$ phsic train_X.txt train_Y.txt --kernel1 Gaussian 1.0 --encoder1 SumBov FasttextEn --emb1 crawl-300d-2M.vec --kernel2 Gaussian 1.0 --encoder2 SumBov FasttextEn --emb2 crawl-300d-2M.vec --limit_words1 10000 --limit_words2 10000 --dim1 3 --dim2 3 --out_prefix toy --out_dir out --X_test test_X.txt --Y_test test_Y.txt
$ cat toy.Gaussian-1.0-SumBov-FasttextEn.Gaussian-1.0-SumBov-FasttextEn.3.3.phsic
1.134489336180434238e-01
2.320408776101631244e-03
2.321869174772554344e-03

Citation

@InProceedings{D18-1203,
  author = 	"Yokoi, Sho
        and Kobayashi, Sosuke
        and Fukumizu, Kenji
        and Suzuki, Jun
        and Inui, Kentaro",
  title = 	"Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions",
  booktitle = 	"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing",
  year = 	"2018",
  publisher = 	"Association for Computational Linguistics",
  pages = 	"1763--1775",
  location = 	"Brussels, Belgium",
  url = 	"http://aclweb.org/anthology/D18-1203"
}

Metadata

Release files for phsic-cli 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for phsic-cli 0.1.0
File Size Uploaded Provenance Actions
phsic-cli-0.1.0.tar.gz 11.6 kB Without provenance Details for phsic-cli-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for phsic-cli 0.1.0
File Interpreter ABI Platform Actions
phsic_cli-0.1.0-py3-none-any.whl Python 3 none any Details for phsic_cli-0.1.0-py3-none-any.whl

Total release size: 50.8 kB

Release files / phsic-cli-0.1.0.tar.gz

Download URL phsic-cli-0.1.0.tar.gz
Size 11.6 kB
Tags Source
SHA-256 checksum
How to use checksums
42dace8336026d0dcfbe2f7e062fff644e4cf908f0d65abee558a10950a59096
BLAKE2b-256 checksum
How to use checksums
f9e6aa405ee91d870b99a5491f6fcf8026406029b067e2c739ecbe801bfc3104
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/0.12.5 CPython/3.6.6 Windows/10

Release files / phsic_cli-0.1.0-py3-none-any.whl

Download URL phsic_cli-0.1.0-py3-none-any.whl
Size 39.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
085579e96158ebe8e69ecb5f4252d101654c7fa65c4e8c8606d7cd6e585fa9db
BLAKE2b-256 checksum
How to use checksums
68a35c906dd2d6c40f9d4e8f047ca603824bf8428b484c707c963cde4b137d6e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/0.12.5 CPython/3.6.6 Windows/10

Security for phsic-cli 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page