wcwidth 0.1.9

Measures number of Terminal column cells of wide-character codes

Introduction

This Library is mainly for those implementing a Terminal Emulator, or programs that carefully produce output to mimick or to be interpreted by an emulator.

Problem Statement: The printible length of most strings are equal to the number of cells they occupy on the screen. However, there are categories of characters that occupy 2 cells (full-wide), and others that occupy 0 cells (zero-width).

Solution: POSIX.1-2001 and POSIX.1-2008 conforming systems provide wcwidth(3) and wcswidth(3) C functions of which this python module’s functions precisely copy. These functions return the number of cells a unicode string is expected to occupy.

Installation

The stable version of this package is maintained on pypi, install or upgrade using pip:

pip install --upgrade wcwidth

Example

Problem: given the following phrase (Japanese),

>>> text = u'コンニチハ'

Python incorrectly uses the string length of 5 codepoints rather than the printible length of 10 cells, so that when using the rjust function, the output length is wrong:

>>> print(len('コンニチハ'))
5

>>> print('コンニチハ'.rjust(11, '_'))
______コンニチハ

By defining our own “rjust” function that uses wcwidth, we can correct this:

>>> def wc_rjust(text, length, padding=' '):
...    from wcwidth import wcswidth
...    return padding * max(0, (length - wcswidth(text))) + text
...

Our Solution uses wcswidth to determine the string length correctly:

>>> from wcwidth import wcswidth
>>> print(wcswidth('コンニチハ'))
10

>>> print(wc_rjust('コンニチハ', 11, '_'))
_コンニチハ

Uses

This library is used in:

• asciimatics: Package to help people create full-screen text UIs.

• blessed: a simplified wrapper around curses.

• curtsies: Curses wrapper with a display based on compositing 2d arrays of text.

• ftfy: Fixes mojibake and other glitches in Unicode text, after the fact.

• pyte: a Simple VTXXX-compatible linux terminal emulator.

• python-prompt-toolkit: a Powerful interactive command line building library.

• termtosvg: Terminal recorder that renders sessions as SVG animations.

wcwidth, wcswidth

Use function wcwidth() to determine the length of a single unicode character, and wcswidth() to determine the length of a several, or a string of unicode characters.

Briefly, return values of function wcwidth() are:

-1

Indeterminate (not printable).

0

Does not advance the cursor, such as NULL or Combining.

2

Characters of category East Asian Wide (W) or East Asian Full-width (F) which are displayed using two terminal cells.

1

All others.

Function wcswidth() simply returns the sum of all values for each character along a string, or -1 when it occurs anywhere along a string.

More documentation is available using pydoc:

$ pydoc wcwidth

Caveats

This library attempts to determine the printable width by a fictional terminal, using the very latest Unicode specification, though some work is in progress for the ability to select a version, there is no standards conforming ability to discern what the target emulator software, version, of level of support is. Results may vary!

A crude method of determining the level of unicode support by the target emulator may be performed using the VT100 Query Cursor Position sequence.

The libc version of wcwidth(3) is often several unicode releases behind, and therefor several levels of support lower than this python library.

Developing

Install wcwidth in editable mode:

pip install -e.

Install developer requirements:

pip install -r requirements-develop.txt

Execute unit tests using tox:

tox

Use the interactive browser:

python bin/wcwidth-browser.py

This library aims to be forward-looking, portable, and most correct. The most current release of this API is based on the Unicode Standard release files:

DerivedGeneralCategory-13.0.0.txt

Date: 2019-10-21, 14:30:32 GMT © 2019 Unicode®, Inc.

EastAsianWidth-13.0.0.txt

Date: 2020-01-21, 18:14:00 GMT [KW, LI] © 2020 Unicode®, Inc.

Updating Tables

The command python setup.py update will fetch the following resources:

• http://www.unicode.org/Public/UNIDATA/EastAsianWidth.txt

• http://www.unicode.org/Public/UNIDATA/extracted/DerivedGeneralCategory.txt

And generates the table files:

• wcwidth/table_wide.py

• wcwidth/table_zero.py

History

0.1.9 2020-03-22

• Performance optimization by Avram Lubkin, PR #35.

• Updated tables to Unicode Specification 13.0.0.

0.1.8 2020-01-01

• Updated tables to Unicode Specification 12.0.0. (PR #30).

0.1.7 2016-07-01

• Updated tables to Unicode Specification 9.0.0. (PR #18).

0.1.6 2016-01-08 Production/Stable

• LICENSE file now included with distribution.

0.1.5 2015-09-13 Alpha

• Bugfix: Resolution of “combining character width” issue, most especially those that previously returned -1 now often (correctly) return 0. resolved by Philip Craig via PR #11.

• Deprecated: The module path wcwidth.table_comb is no longer available, it has been superseded by module path wcwidth.table_zero.

0.1.4 2014-11-20 Pre-Alpha

• Feature: wcswidth() now determines printable length for (most) combining characters. The developer’s tool bin/wcwidth-browser.py is improved to display combining characters when provided the --combining option (Thomas Ballinger and Leta Montopoli PR #5).

• Feature: added static analysis (prospector) to testing framework.

0.1.3 2014-10-29 Pre-Alpha

• Bugfix: 2nd parameter of wcswidth was not honored. (Thomas Ballinger, PR #4).

0.1.2 2014-10-28 Pre-Alpha

• Updated tables to Unicode Specification 7.0.0. (Thomas Ballinger, PR #3).

0.1.1 2014-05-14 Pre-Alpha

• Initial release to pypi, Based on Unicode Specification 6.3.0

This code was originally derived directly from C code of the same name, whose latest version is available at http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c:

* Markus Kuhn -- 2007-05-26 (Unicode 5.0)
*
* Permission to use, copy, modify, and distribute this software
* for any purpose and without fee is hereby granted. The author
* disclaims all warranties with regard to this software.