denial 0.0.4

Is one None not enough for you? There's more

ⓘ

The None constant built into Python is convenient for client code, but it is often insufficient when creating libraries. The fact is that this makes it impossible to distinguish situations where a value is undefined from situations where it is defined as undefined. Does that sound too abstract?

In fact, the problem of this distinction is found everywhere in library development. Sentinel objects are used to resolve it, and many modules from the standard library define their own. For example, the dataclasses library defines a special MISSING constant for such cases. This is used to separate the cases when the user has not set a default value from the case when he has set None as the default value.

However, we can't all use sentinel objects from some built-in module if we don't need the functionality of that module. Until the PEP has been adopted on this topic, it is better to use a special package containing only this primitive. Such a package is denial. It defines just such an object: None for situations where you need to distinguish None as a value from the user, and None as a designation that something is really undefined. This value should not fall "outside", into the user's space, it should remain only inside the libraries implementations. In addition, this library offers a special class that allows you to create your own sentinels.

Table of contents

• Installation
• The second None
• Your own None objects
• Type hinting
• Analogues

Installation

Install it:

pip install denial

You can also quickly try out this and other packages without having to install using instld.

The second None

This library defines an object that is proposed to be used in almost the same way as a regular None. This is how it is imported:

from denial import InnerNone

This object is equal only to itself:

print(InnerNone == InnerNone)
#> True
print(InnerNone == False)
#> False

This object is also an instance of InnerNoneType class (an analog of NoneType, however, is not inherited from this), which makes it possible to check through isinstance:

from denial import InnerNoneType

print(isinstance(InnerNone, InnerNoneType))
#> True

Like None, InnerNone (as well as all other InnerNoneType objects) always returns False when cast to bool:

print(bool(InnerNone))
#> False

ⓘ It is recommended to use the InnerNone object inside libraries where a value close to None is required, but meaning a situation where the value is not really set, rather than set as None. This object should be completely isolated from the user code space. None of the public methods of your library should return this object.

Your own None objects

If None and InnerNone are not enough for you, you can create your own similar objects by instantiating InnerNoneType:

sentinel = InnerNoneType()

This object will also be equal only to itself:

print(sentinel == sentinel)
#> True

print(sentinel == InnerNoneType())  # Comparison with another object of the same type
#> False
print(sentinel == InnerNone)  # Also comparison with another object of the same type
#> False
print(sentinel == None)  # Comparison with None
#> False
print(sentinel == 123)  # Comparison with an arbitrary object
#> False

You can also pass an integer or a string to the class constructor. An InnerNoneType object is equal to another such object with the same argument:

print(InnerNoneType(123) == InnerNoneType(123))
#> True
print(InnerNoneType('key') == InnerNoneType('key'))
#> True

print(InnerNoneType(123) == InnerNoneType(1234))
#> False
print(InnerNoneType('key') == InnerNoneType('another key'))
#> False
print(InnerNoneType(123) == InnerNoneType())
#> False
print(InnerNoneType(123) == 123)
#> False

💡 Any InnerNoneType objects can be used as keys in dictionaries.

There is an internal id inside each InnerNoneType object, which is incremented when it is created (yes, it is thread-safe!). It is according to it that two objects are checked for equality: they are equal if their ids are equal. However, if you pass your own string or integer id when creating an object, it is used for checks. In fact, there are 2 parallel identifier spaces: those that are assigned automatically and those that are passed to the constructor when creating objects. There may be collisions between these spaces, so it is recommended to use only one selected type of identification in your code, without mixing them.

Type hinting

When used in a type hint, the expression None is considered equivalent to type(None).

Official typing documentation

None is a special value for which Python type checkers make an exception, allowing it to be used as an annotation of its own type. Unfortunately, this behavior cannot be reproduced without changing the internal implementation of existing type checkers, which I would not expect until the PEP is adopted.

Therefore, it is suggested to use class InnerNoneType as a type annotation:

def function(default: int | InnerNoneType):
    ...

In case you need a universal annotation for None and InnerNoneType objects, use the Sentinel annotation:

from denial import Sentinel

variable: Sentinel = InnerNone
variable: Sentinel = InnerNoneType()
variable: Sentinel = None  # All 3 annotations are correct.

Analogues

The problem of distinguishing types of uncertainty is often faced by programmers and they solve it in a variety of ways. This problem concerns all programming languages, because it ultimately describes our knowledge, and the questions of cognition are universal for everyone.

Some programming languages are a little better thought out in this matter than Python. For example, JavaScript explicitly distinguishes between undefined and null. I think this is due to the fact that form validation is often written in JS, and it often requires such a distinction. However, this approach is not completely universal, since in the general case the number of layers of uncertainty is infinite, and here there are only 2 of them. In contrast, denial provides both features: the basic InnerNone constant for simple cases and the ability to create an unlimited number of InnerNoneType instances for complex ones. Other languages, such as AppleScript and SQL, also distinguish several different types of undefined values. A separate category includes the languages Rust, Haskell, OCaml, and Swift, which use algebraic data types.

The Python standard library uses at least 15 sentinel objects:

• _collections_abc: marker
• cgitb.UNDEF
• configparser: _UNSET
• dataclasses: _HAS_DEFAULT_FACTORY, MISSING, KW_ONLY
• datetime.timezone._Omitted
• fnmatch.translate() STAR
• functools.lru_cache.sentinel (each @lru_cache creates its own sentinel object)
• functools._NOT_FOUND
• heapq: temporary sentinel in nsmallest() and nlargest()
• inspect._sentinel
• inspect._signature_fromstr() invalid
• plistlib._undefined
• runpy._ModifiedArgv0._sentinel
• sched: _sentinel
• traceback: _sentinel

Since the language itself does not regulate this in any way, there is chaos and code duplication. Before creating this library, I used one of them, but later realized that importing a module that I don't need for anything other than sentinel is a bad idea.

Not only did I come to this conclusion, the community also tried to standardize it. A standard for sentinels was proposed in PEP-661, but at the time of writing it has still not been adopted, as there is no consensus on a number of important issues. This topic was also indirectly raised in PEP-484, as well as in PEP-695 and in PEP-696. Unfortunately, while there is no "official" solution, everyone is still forced to reinvent the wheel on their own. Some, such as Pydantic, are proactive, as if PEP-661 has already been adopted. Personally, I don't like the solution proposed in PEP-661, mainly because of the implementation examples that suggest using a global registry of all created sentinels, which can lead to memory leaks and concurrency limitations.

In addition to denial, there are many packages with sentinels in Pypi. For example, there is the sentinel library, but its API seemed to me overcomplicated for such a simple task. The sentinels package is quite simple, but in its internal implementation it also relies on the global registry and contains some other code defects. The sentinel-value package is very similar to denial, but I did not see the possibility of autogenerating sentinel ids there. Of course, there are other packages that I haven't reviewed here.

And of course, there are still different ways to implement primitive sentinels in your code in a few lines of code without using third-party packages.