Tagged information entities, built on TagSets for representation and typically an SQLTags for storage. I use these to persist and mediate knowledge, including my interactions with web sites, APIs, and third party databases.

Short summary:

• Entities: A mixin to support classes which use a .tagsets:BaseTagSets attribute to store their data.

• Entity: A base class for classes which have a .tags:TagSet attribute and a .tags_db:BaseTagSets containing other Tagsets.

• ScanData: A class to manage data obtained about SiteEntity instances, for example from an API or scanning a web page.

Classes

class Entities

A mixin to support classes which use a .tagsets:BaseTagSets attribute to store their data.

Subclasses may define the following class attributes:

• EntityClass: a subclass of Entity which represents data entities; the default is Entity which should be enough if there is no .tYPE_ZONE
• TYPE_ZONE: the type zone identifying entities in the larger BaseTagSets data; if this is not supplied it is obtained from EntityClass.TYPE_ZONE, if defined

A typical use subclasses cs.sqltags.UsesSQLTags, a subclass of this which uses an SQLTags as the storage backend.

If there is a .TYPE_ZONE, the meaning of the type zone, subname and key are as described for the ZonedTypes class.

Entities.TagsetsClass

None

Entities.__class_getitem__(index)

An Entities subclass may be indexed with a string.

If there is no cls.TYPE_ZONE the string is treated either as:

• if the string ha no dots, a TYPE_ZONE value - the Entities instance for that zone is returned
• if the string has dots, as an Entity.name and looked up with cls.by_entity_id(index).

If there is a cls.TYPE_ZONE, such as with a SiteMap, the string is treated as a ZonedTypes.type_zone_key and looked up as by indexing that zone's Entities instance.

Example using TheTVDBAPI, which has a TYPE_ZONE:

# fetch the TV series entity with id 1234
# there is a TheTVDBAPI.TYPE_ZONE
series = TheTVDBAPI['series.1234']

# fetch an arbtrary Entity
# the value of `TheTVDBAPI.TYPE_ZONE` is "tvdb"
series = Entities['tvdb.series.1234']

Example using SiteMap, the base class for site maps, and which has no .TYPE_ZONE:

smh_map = SiteMap['smh']
smh_topic = SiteMap['smh.topic.technology']
smh_article = SiteMap['smh']['article.abcd']

Entities.__getitem__(self, index: str | tuple[str, str | int] | tuple[str, str, str | int]) -> cs.tagged.Entity

self.__getitem__(index) calls self.entity(index).

Entities.__init_subclass__(**kw)

Inititialise a subclass by defining .TYPE_ZNE if already present.

Entities.as_zone(self, zone=None)

Push this Entities instance as the default mapping for zone, whose default is self.__class__.TYPE_ZONE. Yields the zone, or None if there is no

Entities.by_entity_id(entity_id: str) -> cs.tagged.Entity

Return the Entity instance corresponding to entity_id from the full tb Raise ValueError if entity_id cannot be parsed by ZonedTypes.type_parts_of. Raise KeyError if there is no Entities instance for the zone and we cannot make a default instance.

Entities.by_type_zone

<WeakValueDictionary at 0x10e799490>

Entities.class_by_type_zone

{}

Entities.default(zone: str | None = None) -> 'Entities'

Return the default Entities instance for zone. If zone is not defined it is taken from cls.TYPE_ZONE. Raise KeyError for an unregistered zone. Raise TypeError if there is no registered default and the class for zone cannot be instantiated with entcls().

Entities.entity(self, index: str | tuple[str, str | int] | tuple[str, str, str | int], zone=None) -> cs.tagged.Entity

Fetch the Entity instance for the supplied index. This underlies the __getitem__ method.

The meaning of the type zone, subname and key are as described for the ZonedTypes class.

The index may take the following forms:

• str: a string which will be split into subname and key for use in self.TYPE_ZONE
• (subname,key): a 2-tuple of the type subname and key in self.TYPE_ZONE the subname make also be a subclass of self.EntityClass
• (zone,subname,key): a 3-tuple of the type zone, subname and key The subname may also be a class (normally a subclass of Entity, usually a subclass of type(self).EntityClass); in this case the subname will be taken from type(self).TYPE_SUBNAME attribute. The key may also be an int or a uuid.UUID, in which case it will be used as str(key).

Examples:

# the Entity subclass Artist, and the Entities
# subclass MBDB which hold MusicbrainzNG information
from cs.cdrip import Artist, MBDB
mbdb = MBDB()

# Various indices obtaining the record for Jon Cleary,
# whose key is 'mbdb.artist.a417f0e5-2c14-445a-9a07-5a7ad2bdeafa'

# the subname.key as a single string
artist = mbdb['artist.a417f0e5-2c14-445a-9a07-5a7ad2bdeafa']

# the subname and key in a 2-tuple
artist = mbdb['artist', 'a417f0e5-2c14-445a-9a07-5a7ad2bdeafa']

# the record but not from the default MBDB zonne
artist = mbdb['mbdb2', 'artist', 'a417f0e5-2c14-445a-9a07-5a7ad2bdeafa']

# the preferred way to obtain it, using the entity type
artist = mbdb[Artist, 'a417f0e5-2c14-445a-9a07-5a7ad2bdeafa']

# or if you're working with UUIDs
artist_uuid = UUID('a417f0e5-2c14-445a-9a07-5a7ad2bdeafa')
artist = mbdb[Artist, artist_uuid]

Entities.find(self, *criteria, **crit_kw) -> list[cs.tagged.Entity]

Find entities in the database.

This calls self.tagsets.find() and returns the associated Entity instances.

Entities.keys(self, subname=None)

Return the keys from self.tagsets as (subname,type_key) 2-tuples suitable as indices of self. If subname is not None, restrict the keys to those with that subname.

Entities.set_as_zone(self, zone: str, if_unset=False)

Set this Entities instance as the one handling entities in zone.

Entities.zone_entity(self, zone: str) -> 'Entity'

Return the Entity entity associated with a per-type-zone key. For example, self.zone_entity('tvdb') would return the entity for tvdb.tvdb_id where tvdb_id comes from self['id.tvdb'].

class Entity(collections.abc.Mapping, cs.tagset.ZonedTypes, cs.obj.Refreshable, cs.deco.Promotable, cs.lex.FormatableMixin, cs.obj.NoAttrs)

A base class for classes which have a .tags:TagSet attribute and a .tags_db:BaseTagSets containing other Tagsets.

Usually these are considered part of a "zone" - a group of entities in a particular applicaiton domain.

The subclass may itself define its .tags instance attribute or rely on the default cached property .tags, which will return self.tags_db[self.tags_entity_key]. (self.tags_entity_key is self.tags.name by default.)

Note that this class brings its own __new__ method which can choose a subclass based on the subclass' .TYPE_SUBNAME attribute. See the __new__ docstring.

This also provides some behaviour based around updating entities based on some kind of API call; the direct values from the API call land on attributes named {zone}.{key} - the .type_zone_update(mapping) provides a convenient call for this.

Entity instances are designed as representing entities in some "zone", a set of entities in some domain or organised grouping; typical examples include entities describes by some API like MusicBrainzNG or objects presented by some website. As such, they subclass ZonedTypes, which expects the entity's .name to be of the form zone.subname.key; the zone partitions entities off into their own domain, the subname is in effect the entity's type within that domain and the key is the entity id within that type.

On this basis, entities updated with data from the zone, for example from an API call or by scraping a web page, normally update tag keys named zone.field where the field is the top level field from the data.

The ZonedTypes.__getattr__(attr) method looks first for a direct tag named attr but falls back to a tag named zone.attr. This allows entities to be tagged with the data from an API, but to be overridden by the direct tag if the API data are considered incorrect or unsuitable.

The ScanData.apply() method follows this principle, applying the scanned data to tags named zone.field.

We relate entities using attributes named field_id, which may be a single key for another entity or a list of keys.

Various derived attributes are also provided, see the __getattr__ docstring for details:

• field_ent: the related Entity named zone.field.key where key comes from the .field_id attribute
• field_ents: multiple related Entity named zone.field.key where key comes from the .field_id attribute

Entity.__delitem__(self, tag_name: str)

Remove an entry from self.tags.

Entity.__getattr__(self, attr)

Try ZonedTypes.__getattr__ (which lokks up [attr] then [f'{zone}.{attr}']) then fall back to suffix based synthetic attributes where an attribute ending in _suffix is implemented by the suffix_suffix(attr) method if it exists.

The following synthetic attibutes are implemented:

• attr0_or_none: return .attr0 or None if that does not exist
• subtype_ent: the entity with name type_zone.subtype.id or None where id comes from the .attr_id value; see the suffix_ent method.
• subtype_ents: the entities with name type_zone.subtype.id or None where each id comes from the .attr_id values; see the suffix_ents method.

Entity.__getitem__(self, tag_name: str)

Index self.tags.

Entity.__len__(self)

###################################################################### Mapping Methods, proxies to self.tags.

Entity.__setitem__(self, tag_name, value, *, verbose=False)

Set a tag value.

Entity.as_dict(self)

Proxy .as_dict() to self.tags.

Entity.entity

<functools.cached_property object at 0x10e9e5e50>

Entity.entity_

<functools.cached_property object at 0x10e9e5b50>

Entity.field_ref_type(self, field_name: str) -> str

Map a field name to its associated subtype. This default method returns the field name unchanged.

This is used by methods such as .suffix_ents() to map its reference field name (the attribute name before _ents) via this method to the zone subtype.

Entity.format_attributes

{'json': <function Entity.json at 0x10e722ac0>}

Entity.format_kwargs(self)

A format_kwargs method to support cs.lex.FormatableMixin.

Entity.get(self, tag_name: str, default=None)

Call .tags.get(tag_name).

Entity.items(self)

The tags items.

Entity.prefix_in(self, attr) -> collections.abc.Sequence[typing.Self]

Resolve in_subtype[_field to the Entity instance of subtype subtype whose field_id attribute contains self.type_key. The default field is self.type_subname.

For example, if self.name is "tvdb.episode.1234" then self.in_season would return a list of all the tvdb.season entities whose episode_id attributes referred to 1234.

Where the

Entity.print(self)

The default print() runs self.printt(). This is intended to be a nice print of important stuff.

Entity.refresh_key(self)

The unique key identifying this object for use in recursive refreshes.

Entity.refresh_last_update

<property object at 0x10e7ae6b0>

Entity.setdefault(self, key, default_value)

Set self[key]=default_value if key is not present.

Entity.suffix_ent(self, attr) -> Optional[Self]

Resolve subtype_ent to self[type_zone.subtype.id] or None if no self[subtype_id]

Entity.suffix_ents(self, attr) -> collections.abc.Sequence[typing.Self]

Resolve subtype_ents to [self[type_zone.*subtype*.id]]or()if noself[*subtype*_id]]`.

Entity.tags

<functools.cached_property object at 0x10e6c45d0>

Entity.tags_entity_key

<functools.cached_property object at 0x10e6c4bd0>

Entity.type_zone_update(self, mapping, prefix=None, *, lc_=False)

Update self with mapping, using prefix. The default prefix is self.type_zone`.

Entity.update(self, *update_a, **update_kw)

Update the tags, tupically from a mapping or keyword arguments.

Entity.values(self)

The tags values.

class ScanData

A class to manage data obtained about SiteEntity instances, for example from an API or scanning a web page.

The data for an SiteEntity can be obtained by indexing the ScanData instance with a SiteEntity instance or a (ent_cls,type_key) 2-tuple

ScanData.__getitem__(self, ent: Union[tuple, ForwardRef('Entity')])

The data for the supplied ent.

ScanData.__iter__(self)

Iteration yields (Entity,datadict) 2-tuples.

ScanData.apply(self, *refresh_ents)

Apply the scanned data to its entities.

If an entity ent is a member of refresh_ents then call ent.refresh(data=data) on the basis that the data are complete enough to consider the entity refreshed. Otherwise call ent.type_zone_update(data).

The purpose of the call to ent.refresh() is to exercise the refresh machinery. On a Refreshable object ent this marks the object as current with the new data; the data are applied with Refreshable._refresh(), the zone specific method, which typically also uses ent.type_zone_update(data).

This follows the tag name design outlined in the Entity docstring, where API/site data are stored with tags named zone.field.

ScanData.conv(self, ent: Union[tuple, ForwardRef('Entity')], mapping, key, conv=None)

Update the data for ent from mapping[key] if present. If conv is not None it should be a callable accepting the value from mapping[key] and returning a converted value to store in the entity data.

ScanData.printt(self, title=None)

Call cs.lex.printt() to print the scanned data.

ScanData.update(self, ent: Union[tuple[type, int | str], ForwardRef('Entity')], **data_kw)

Update the data for ent from data_kw.