Simple SQL based tagging and the associated `sqltags` command line script, supporting both tagged named objects and tagged timestamped log entries.
Project description
Simple SQL based tagging
and the associated sqltags
command line script,
supporting both tagged named objects and tagged timestamped log entries.
Latest release 20240723:
- Replace many raises of RuntimeError with NotImplementedError, suggestion by @dimaqq on discuss.python.org.
- Move some constants from BaseSQLTagsCommand to SQLTagsCommandsMixin where they belong, add missing USAGE_KEYWORDS entry.
- SQLTagsORM: do not define_schema() in init, instead let the ORM do that and we prepare the metanode on first use of the db - this makes it much cheaper to make an SQLTags and then not use it.
Compared to cs.fstags
and its associated fstags
command,
this is oriented towards large numbers of items
not naturally associated with filesystem objects.
My initial use case is an activity log (unnamed timestamped tag sets) but I'm also using it for ontologies (named tag sets containing metadata).
Many basic tasks can be performed with the sqltags
command line utility,
documented under the SQLTagsCommand
class below.
See the SQLTagsORM
documentation for details about how data
are stored in the database.
See the SQLTagSet
documentation for details of how various
tag value types are supported.
Class BaseSQLTagsCommand(cs.cmdutils.BaseCommand, SQLTagsCommandsMixin)
Common features for commands oriented around an SQLTags
database.
Usage summary:
Usage: basesqltags [-f db_url] subcommand [...]
-f db_url SQLAlchemy database URL or filename.
Default from $SQLTAGS_DBURL (default '~/var/sqltags.sqlite').
Subcommands:
dbshell
Start an interactive database shell.
edit criteria...
Edit the entities specified by criteria.
export [-F format] [{tag[=value]|-tag}...]
Export entities matching all the constraints.
-F format Specify the export format, either CSV or FSTAGS.
find [-o output_format] {tag[=value]|-tag}...
List entities matching all the constraints.
-o output_format
Use output_format as a Python format string to lay out
the listing.
Default: {localtime} {headline}
help [-l] [subcommand-names...]
Print help for subcommands.
This outputs the full help for the named subcommands,
or the short help for all subcommands if no names are specified.
-l Long help even if no subcommand-names provided.
import [{-u|--update}] {-|srcpath}...
Import CSV data in the format emitted by "export".
Each argument is a file path or "-", indicating standard input.
-u, --update If a named entity already exists then update its tags.
Otherwise this will be seen as a conflict
and the import aborted.
init
Initialise the database.
This includes defining the schema and making the root metanode.
log [-c category,...] [-d when] [-D strptime] {-|headline} [tags...]
Record entries into the database.
If headline is '-', read headlines from standard input.
-c categories
Specify the categories for this log entry.
The default is to recognise a leading CAT,CAT,...: prefix.
-d when
Use when, an ISO8601 date, as the log entry timestamp.
-D strptime
Read the time from the start of the headline
according to the provided strptime specification.
orm define_schema
Runs the ORM's `define_schema()` method, which creates missing tables
and entity 0 if missing.
shell
Run a command prompt via cmd.Cmd using this command's subcommands.
tag {-|entity-name} {tag[=value]|-tag}...
Tag an entity with multiple tags.
With the form "-tag", remove that tag from the direct tags.
A entity-name named "-" indicates that entity-names should
be read from the standard input.
BaseSQLTagsCommand.Options
BaseSQLTagsCommand.apply_opt(self, opt, val)
:
Apply a command line option.
BaseSQLTagsCommand.parse_categories(categories)
:
Extract "category" words from the str
categories
,
return a list of category names.
Splits on commas, strips leading and trailing whitespace, downcases.
BaseSQLTagsCommand.parse_tagset_criterion(arg, tag_based_test_class=None)
:
Parse tag criteria from argv
.
The criteria may be either:
- an integer specifying a
Tag
id - a sequence of tag criteria
BaseSQLTagsCommand.run_context(self)
:
Prepare the SQLTags
around each command invocation.
glob2like(glob: str) -> str
Convert a filename glob to an SQL LIKE pattern.
main(argv=None)
Command line mode.
Class PolyValue(PolyValue, PolyValued)
A namedtuple
for the polyvalues used in an SQLTagsORM
.
We express various types in SQL as one of 3 columns:
float_value
: forfloat
s andint
s which round trip withfloat
string_value
: forstr
structured_value
: a JSON transcription of any other type
This allows SQL indexing of basic types.
Note that because str
gets stored in string_value
this leaves us free to use "bare string" JSON to serialise
various nonJSONable types.
The SQLTagSets
class has a to_polyvalue
factory
which produces a PolyValue
suitable for the SQL rows.
NonJSONable types such as datetime
are converted to a str
but stored in the structured_value
column.
This should be overridden by subclasses as necessary.
On retrieval from the database
the tag rows are converted to Python values
by the SQLTagSets.from_polyvalue
method,
reversing the process above.
Class PolyValueColumnMixin(PolyValued)
A mixin for classes with (float_value,string_value,structured_value)
columns.
This is used by the Tags
and TagMultiValues
relations inside SQLTagsORM
.
Class PolyValued
A mixin for classes with (float_value,string_value,structured_value)
columns.
PolyValued.as_polyvalue(self)
:
Return this row's value as a PolyValue
.
PolyValued.is_valid(self)
:
Test that at most one attribute is non-None
.
PolyValued.set_polyvalue(self, pv: 'PolyValued')
:
Set all the value fields.
PolyValued.value_test(other_value)
:
Return (column,test_value)
for constructing tests against
other_value
where column
if the appropriate SQLAlchemy column
and test_value
is the comparison value for testing.
For most other_value
s the test_value
will just be other_value
,
but for certain types the test_value
will be:
NoneType
:None
, and the column will also beNone
datetime
:datetime2unixtime(other_value)
prefix2like(prefix: str, esc='\\') -> str
Convert a prefix string to an SQL LIKE pattern.
Class SQLParameters(SQLParameters)
The parameters required for constructing queries or extending queries with JOINs.
Attributes:
criterion
: the source criterion, usually anSQTCriterion
subinstancealias
: an alias of the source table for use in queriesentity_id_column
: theentities
id column,alias.id
if the alias is ofentities
,alias.entity_id
if the alias is oftags
constraint
: a filter query based onalias
Class SQLTagBasedTest(cs.tagset.TagBasedTest, SQTCriterion)
A cs.tagset.TagBasedTest
extended with a .sql_parameters
method.
SQLTagBasedTest.match_tagged_entity(self, te: cs.tagset.TagSet) -> bool
:
Match this criterion against te
.
Class SQLTagProxies
A proxy for the tags supporting Python comparison => SQLParameters
.
Example:
sqltags.tags.dotted.name.here == 'foo'
Class SQLTagProxy
An object based on a Tag
name
which produces an SQLParameters
when compared with some value.
Example:
>>> sqltags = SQLTags('sqlite://')
>>> sqltags.init()
>>> # make a SQLParameters for testing the tag 'name.thing'==5
>>> sqlp = sqltags.tags.name.thing == 5
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.float_value = :float_value_1'
>>> sqlp = sqltags.tags.name.thing == 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value = :string_value_1'
SQLTagProxy.__eq__(self, other, alias=None) -> cs.sqltags.SQLParameters
:
Return an SQL =
test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing == 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value = :string_value_1'
SQLTagProxy.__ge__(self, other)
:
Return an SQL >=
test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing >= 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value >= :string_value_1'
SQLTagProxy.__getattr__(self, sub_tag_name)
:
Magic access to dotted tag names: produce a new SQLTagProxy
from ourself.
SQLTagProxy.__gt__(self, other)
:
Return an SQL >
test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing > 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value > :string_value_1'
SQLTagProxy.__le__(self, other)
:
Return an SQL <=
test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing <= 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value <= :string_value_1'
SQLTagProxy.__lt__(self, other)
:
Return an SQL <
test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing < 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value < :string_value_1'
SQLTagProxy.__ne__(self, other, alias=None) -> cs.sqltags.SQLParameters
:
Return an SQL <>
test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing != 'foo'
>>> str(sqlp.constraint)
'tags_1.name = :name_1 AND tags_1.string_value != :string_value_1'
SQLTagProxy.by_op_text(self, op_text, other, alias=None)
:
Return an SQLParameters
based on the comparison's text representation.
Parameters:
op_text
: the comparsion operation text, one of:'='
,'<='
,'<'
,'>='
,'>'
,'~'
.other
: the other value for the comparison, used to infer the SQL column name and kept to provide the SQL value parameteralias
: optional SQLAlchemy table alias
SQLTagProxy.likeglob(self, globptn: str) -> cs.sqltags.SQLParameters
:
Return an SQL LIKE test approximating a glob as an SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing.likeglob('foo*')
>>> str(sqlp.constraint)
"tags_1.name = :name_1 AND tags_1.string_value LIKE :string_value_1 ESCAPE '\\'"
SQLTagProxy.startswith(self, prefix: str) -> cs.sqltags.SQLParameters
:
Return an SQL LIKE prefix test SQLParameters
.
Example:
>>> sqlp = SQLTags('sqlite://').tags.name.thing.startswith('foo')
>>> str(sqlp.constraint)
"tags_1.name = :name_1 AND tags_1.string_value LIKE :string_value_1 ESCAPE '\\'"
Class SQLTags(cs.tagset.BaseTagSets, cs.deco.Promotable)
A class using an SQL database to store its TagSet
s.
SQLTags.TAGSETCLASS_DEFAULT(self, *a, _sqltags=None, **kw)
:
Factory to return a suitable TagSet
subclass instance.
This produces an SQLTagSet
instance correctly associated with this SQLTags
.
SQLTags.TagSetClass(self, *, name, **kw)
:
Local implementation of TagSetClass
so that we can annotate it with a .singleton_also_by
attribute.
SQLTags.__getitem__(self, index)
:
Return an SQLTagSet
for index
(an int
or str
).
SQLTags.__setitem__(self, index, te)
:
Dummy __setitem__
which checks te
against the db by type
because the factory has already inserted it into the database.
SQLTags.db_entity(self, index)
:
Return the Entities
instance for index
or None
.
SQLTags.db_session(self, *, new=False)
:
Context manager to obtain a db session if required
(or if new
is true).
SQLTags.default_db_session
:
The current per-Thread
SQLAlchemy Session.
SQLTags.default_factory(self, name: Optional[str] = None, *, unixtime=None, tags=None, skip_refresh=False)
:
Fetch or create an SQLTagSet
for name
.
Return the SQLTagSet
.
Note that name
may be None
to create a new "log" entry.
SQLTags.find(self, *criteria, _without_tags=False, **crit_kw)
:
Generate and run a query derived from criteria
yielding SQLTagSet
instances.
Parameters:
criteria
: positional arguments which should beSQTCriterion
s or astr
suitable forSQTCriterion.from_str
_without_tags
: optional flag to return entities without tags, defaultFalse
; this can be used for a much faster scan of the entities because it omits theJOIN
against the tag tablecrit_kw
: keyword parameters are appended to the criteria as further tag equality tests
SQLTags.flush(self)
:
Flush the current session state to the database.
SQLTags.from_str(db_url: str)
:
Create an SQLTags
from db_url
.
SQLTags.get(self, index, default=None)
:
Return an SQLTagSet
matching index
, or None
if there is no such entity.
SQLTags.import_csv_file(self, f, *, update_mode=False)
:
Import CSV data from the file f
.
If update_mode
is true
named records which already exist will update from the data,
otherwise the conflict will raise a ValueError
.
SQLTags.import_tagged_entity(self, te, *, update_mode=False) -> None
:
Import the TagSet
te
.
This updates the database with the contents of the supplied TagSet
,
which has no inherent relationship to the database.
If update_mode
is true
named records which already exist will update from te
,
otherwise the conflict will raise a ValueError
.
SQLTags.infer_db_url(envvar=None, default_path=None)
:
Infer the database URL.
Parameters:
envvar
: environment variable to specify a default, default fromDBURL_ENVVAR
(SQLTAGS_DBURL
).
SQLTags.items(self, *, prefix=None)
:
Return an iterable of (tagset_name,TagSet)
.
Excludes unnamed TagSet
s.
Constrain the names to those starting with prefix
if not None
.
SQLTags.keys(self, *, prefix=None)
:
Yield all the nonNULL names.
Constrain the names to those starting with prefix
if not None
.
SQLTags.metanode
:
The metadata node.
SQLTags.startup_shutdown(self)
:
Open the ORM while the SQLTags
is open.
SQLTags.values(self, *, prefix=None)
:
Return an iterable of the named TagSet
s.
Excludes unnamed TagSet
s.
Constrain the names to those starting with prefix
if not None
.
Class SQLTagsCommand(BaseSQLTagsCommand)
sqltags
main command line utility.
Usage summary:
Usage: sqltags [-f db_url] subcommand [...]
-f db_url SQLAlchemy database URL or filename.
Default from $SQLTAGS_DBURL (default '~/var/sqltags.sqlite').
Subcommands:
dbshell
Start an interactive database shell.
edit criteria...
Edit the entities specified by criteria.
export [-F format] [{tag[=value]|-tag}...]
Export entities matching all the constraints.
-F format Specify the export format, either CSV or FSTAGS.
find [-o output_format] {tag[=value]|-tag}...
List entities matching all the constraints.
-o output_format
Use output_format as a Python format string to lay out
the listing.
Default: {localtime} {headline}
help [-l] [subcommand-names...]
Print help for subcommands.
This outputs the full help for the named subcommands,
or the short help for all subcommands if no names are specified.
-l Long help even if no subcommand-names provided.
import [{-u|--update}] {-|srcpath}...
Import CSV data in the format emitted by "export".
Each argument is a file path or "-", indicating standard input.
-u, --update If a named entity already exists then update its tags.
Otherwise this will be seen as a conflict
and the import aborted.
init
Initialise the database.
This includes defining the schema and making the root metanode.
list [entity-names...]
List entities and their tags.
log [-c category,...] [-d when] [-D strptime] {-|headline} [tags...]
Record entries into the database.
If headline is '-', read headlines from standard input.
-c categories
Specify the categories for this log entry.
The default is to recognise a leading CAT,CAT,...: prefix.
-d when
Use when, an ISO8601 date, as the log entry timestamp.
-D strptime
Read the time from the start of the headline
according to the provided strptime specification.
ls [entity-names...]
List entities and their tags.
orm define_schema
Runs the ORM's `define_schema()` method, which creates missing tables
and entity 0 if missing.
shell
Run a command prompt via cmd.Cmd using this command's subcommands.
tag {-|entity-name} {tag[=value]|-tag}...
Tag an entity with multiple tags.
With the form "-tag", remove that tag from the direct tags.
A entity-name named "-" indicates that entity-names should
be read from the standard input.
SQLTagsCommand.cmd_list(self, argv)
:
Usage: {cmd} [entity-names...]
List entities and their tags.
SQLTagsCommand.cmd_ls(self, argv)
:
Usage: {cmd} [entity-names...]
List entities and their tags.
Class SQLTagSet(cs.obj.SingletonMixin, cs.tagset.TagSet)
A singleton TagSet
attached to an SQLTags
instance.
As with the TagSet
superclass,
tag values can be any Python type.
However, because we are storing these values in an SQL database
it is necessary to provide a conversion facility
to prepare those values for storage.
The database schema is described in the SQLTagsORM
class;
in short we directly support None
, float
and str
,
int
s which round trip with float
,
and list
, tuple
and dict
whose contents transcribe to JSON.
int
s which are too large to round trip with float
are treated as an extended "bigint"
type
using the scheme described below.
Because the ORM has distinct float
and str
columns to support indexing,
there will be no plain strings in the remaining JSON blob column.
Therefore we support other types by providing functions
to convert each type to a str
and back,
and an associated "type label" which will be prefixed to the string;
the resulting string is stored in the JSON blob.
The default mechanism is based on the following class attributes and methods:
TYPE_JS_MAPPING
: a mapping of a type label string to a 3 tuple of(type,to_str,from_str)
being the extended type, a function to convert an instance tostr
and a function to convert astr
to an instance of this typeto_js_str
: a method accepting(tag_name,tag_value)
and returningtag_value
as astr
; the default implementation looks up the type oftag_value
inTYPE_JS_MAPPING
to locate the correspondingto_str
functionfrom_js_str
: a method accepting(tag_name,js)
which uses the leading type label prefix from thejs
to look up the correspondingfrom_str
function fromTYPE_JS_MAPPING
and use it on the tail ofjs
The default TYPE_JS_MAPPING
has mappings for:
"bigint"
: conversions forint
"date"
: conversions fordatetime.date
"datetime"
: conversions fordatetime.datetime
Subclasses wanting to augument the TYPE_JS_MAPPING
should prepare their own with code such as:
class SubSQLTagSet(SQLTagSet,....):
....
TYPE_JS_MAPPING=dict(SQLTagSet.TYPE_JS_MAPPING)
TYPE_JS_MAPPING.update(
typelabel=(type, to_str, from_str),
....
)
SQLTagSet.add_db_tag(self, tag_name, pv: cs.sqltags.PolyValue)
:
Add a tag to the database.
SQLTagSet.child_tagsets(self, tag_name='parent')
:
Return the child TagSet
s as defined by their parent Tag
,
by default the Tag
named 'parent'
.
SQLTagSet.db_session(self, new=False)
:
Context manager to obtain a new session if required,
just a shim for self.sqltags.db_session
.
SQLTagSet.discard_db_tag(self, tag_name: str, pv: Optional[cs.sqltags.PolyValue] = None)
:
Discard a tag from the database.
SQLTagSet.from_js_str(tag_name: str, js: str)
:
Convert the str
js
to a Tag
value.
This is the reverse of as_js_str
.
Subclasses wanting extra type support
should either:
(usual approach) provide their own TYPE_JS_MAPPING
class attribute
as described at the top of this class
or (for unusual requirements) override this method and also to_js_str
.
SQLTagSet.from_polyvalue(tag_name: str, pv: cs.sqltags.PolyValue)
:
Convert an SQL PolyValue
to a tag value.
This can be overridden by subclasses along with to_polyvalue
.
The tag_name
is provided for context
in case it should influence the normalisation.
SQLTagSet.jsonable(value)
:
Convert value
to a form which can be directly JSON serialised.
In particular this converts non-list/set/tuple Sequences to lists and non-dict Mappings to dicts.
Warning: this is not robust against cycles.
SQLTagSet.name
:
Return the .name
.
SQLTagSet.parent_tagset(self, tag_name='parent')
:
Return the parent TagSet
as defined by a Tag
,
by default the Tag
named 'parent'
.
SQLTagSet.to_js_str(tag_name: str, tag_value) -> str
:
Convert tag_value
to a str
suitable for storage in structure_value
.
This can be reversed by from_js_str
.
Subclasses wanting extra type support
should either:
(usual approach) provide their own TYPE_JS_MAPPING
class attribute
as described at the top of this class
or (for unusual requirements) override this method and also from_js_str
.
SQLTagSet.to_polyvalue(tag_name: str, tag_value) -> cs.sqltags.PolyValue
:
Normalise Tag
values for storage via SQL.
Preserve things directly expressable in JSON.
Convert other values via to_js_str
.
Return PolyValue
for use with the SQL rows.
Class SQLTagsORM(cs.sqlalchemy_utils.ORM, cs.dateutils.UNIXTimeMixin)
The ORM for an SQLTags
.
The current implementation uses 3 tables:
entities
: this has a NULLablename
andunixtime
UNIX timestamp; this is unique pername
if the name is not NULLtags
: this has anentity_id
,name
and a value stored in one of three columns:float_value
,string_value
andstructured_value
which is a JSON blob; this is unique per(entity_id,name)
tag_subvalues
: this is a broken out version oftags
whenstructured_value
is a sequence or mapping, breaking out the values one per row; this exists to support "tag contains value" lookups
Tag values are stored as follows:
None
: all 3 columns are set toNULL
float
: stored infloat_value
int
: if theint
round trips tofloat
then it is stored infloat_value
, otherwise it is stored instructured_value
with the type label"bigint"
str
: stored instring_value
list
,tuple
,dict
: stored instructured_value
; if these containers contain unJSONable content there will be trouble- other types, such as
datetime
: these are converted to strings with identifying type label prefixes and stored instructured_value
The float_value
and string_value
columns
allow us to provide indices for these kinds of tag values.
The type label scheme takes advantage of the fact that actual str
s
are stored in the string_value
column.
Because of this, there will be no actual strings in structured_value
.
Therefore, we can convert nonJSONable types to str
and store them here.
The scheme used is to provide conversion functions to convert types
to str
and back, and an associated "type label" prefix.
For example, we store a datetime
as the ISO format of the datetime
with "datetime:"
prefixed to it.
The actual conversions are kept with the SQLTagSet
class
(or any subclass).
This ORM receives the 3-tuples of SQL ready values
from that class as the PolyValue
namedtuple
and does not perform any conversion itself.
The conversion process is described in SQLTagSet
.
SQLTagsORM.declare_schema(self)
:
Define the database schema / ORM mapping.
SQLTagsORM.prepare_metanode(self, *, session)
:
Ensure row id 0, the metanode, exists.
SQLTagsORM.search(self, criteria, *, session, mode='tagged')
:
Construct a query to match Entity
rows
matching the supplied criteria
iterable.
Return an SQLAlchemy Query
.
The mode
parameter has the following values:
'id'
: the query only yields entity ids'entity'
: (default) the query yields entities without tags'tagged'
: (default) the query yields entities left outer joined with their matching tags
Note that the 'tagged'
result produces multiple rows for any
entity with multiple tags, and that this requires the caller to
fold entities with multiple tags together.
Note:
due to implementation limitations
the SQL query itself may not apply all the criteria,
so every criterion must still be applied
to the results
using its .match_entity
method.
If name
is omitted or None
the query will match log entities
otherwise the entity with the specified name
.
The criteria
should be an iterable of SQTCriterion
instances
used to construct the query.
Class SQTCriterion(cs.tagset.TagSetCriterion)
Subclass of TagSetCriterion
requiring an .sql_parameters
method
which returns an SQLParameters
providing the information required
to construct an sqlalchemy query.
It also resets .CRITERION_PARSE_CLASSES
, which will pick up
the SQL capable criterion classes below.
SQTCriterion.TAG_BASED_TEST_CLASS
SQTCriterion.from_equality(tag_name, tag_value)
:
Return an SQTCriterion
instance based on tag_name==tag_value
.
This supports SQLTags.find
's keyword parameters.
SQTCriterion.match_tagged_entity(self, te: cs.tagset.TagSet) -> bool
:
Perform the criterion test on the Python object directly.
This is used at the end of a query to implement tests which
cannot be sufficiently implemented in SQL.
If self.SQL_COMPLETE
it is not necessary to call this method.
SQTCriterion.sql_parameters(self, orm) -> cs.sqltags.SQLParameters
:
Subclasses must return am SQLParameters
instance
parameterising the SQL queries that follow.
Class SQTEntityIdTest(SQTCriterion)
A test on entity.id
.
SQTEntityIdTest.match_tagged_entity(self, te: cs.tagset.TagSet) -> bool
:
Test the TagSet
te
against self.entity_ids
.
SQTEntityIdTest.parse(s, offset=0, delim=None)
:
Parse a decimal entity id from s
.
verbose(msg, *a)
Emit message if in verbose mode.
Release Log
Release 20240723:
- Replace many raises of RuntimeError with NotImplementedError, suggestion by @dimaqq on discuss.python.org.
- Move some constants from BaseSQLTagsCommand to SQLTagsCommandsMixin where they belong, add missing USAGE_KEYWORDS entry.
- SQLTagsORM: do not define_schema() in init, instead let the ORM do that and we prepare the metanode on first use of the db - this makes it much cheaper to make an SQLTags and then not use it.
Release 20240316: Fixed release upload artifacts.
Release 20240305: SQLTags: new .from_str so that we can inherit Promotable.promote.
Release 20240201.1: Release with the "sqltags" script.
Release 20240201:
- SQLTagSet.to_polyvalue: treat sets like lists.
- SQLTags.default_factory: honour new skip_refresh parameter, apply any presupplied tags.
- Pull the cmd_* methods from BaseSQLTagsCommand into new SQLTagsCommandsMixin for reuse.
Release 20230612:
- SQLTagBasedTest.sql_parameters: fix general tag name.
- SQLTagSet: new jsonable class method to produce a JSON serialisable object - converts sets and Sequences to flat lists, etc.
Release 20230217: SQLTagsORM.search: previous changes seem to have dropped SQTCriterion support.
Release 20230212.1: Mark SQLTags as promotable.
Release 20230212:
- @promote support for SQLTags, promoting a filesystem path to a .sqlite db.
- Simpler SQLTagsORM.search comparison implementation.
- SQLTagSet: inherit format attributes from superclasses (TagSet).
- New BaseSQLTagsCommand.cmd_shell method.
- New BaseSQLTagsCommand.cmd_orm method with "define_schema" subcommand to update the db schema.
- SQLTagsORM.init: drop case_sensitive, no longer supported?
- SQLTagsORM.init: always call define_schema, it seems there are scenarios where this does some necessary sqlalchemy prep.
Release 20221228: SQLTagsCommand: update implementation of BaseCommand.run_context to use super().run_context().
Release 20220806:
- Bugfix for SQLTagsORM.search(mode='entity').
- SQLTags.find: new _without_tags=False parameter to allow fast searches omitting the entity tags.
Release 20220606:
- New SQLTagsORM.Entities.add_new_tags method, use it in SQLTags.default_factory for bulk insert.
- SQTCriterion: new .from_equality(tag_name,tag_value) factory to make an equality criterion.
- SQLTags.find: accept criteria as positional parameters instead of a single iterable, accept new keyword parameters as equality criteria.
- SQLTags.getitem: accept a slice to index the .unixtime tag.
- SQLTagsORM: also turn on echo mode if "ECHO" in $SQLTAGS_MODES.
Release 20220311: Assorted updates.
Release 20211212:
- Rename edit_many to edit_tagsets for clarity.
- Small bugfixes.
Release 20210913:
- SQLTagsCommand: rename cmd_ns to cmd_list,cmd_ls.
- SQLTagsCommand.cmd_export: accept "-F export_format" for csv or fstags export, accept no criteria to mean all tagsets.
- Encoding schema for nonJSONable types.
- Rename the TagSets abstract base class to BaseTagSets.
- BaseSQLTagsCommand.cmd_edit: implement rename.
- Many other internal small changes.
Release 20210420:
- New PolyValueMixin pulled out of Tags for common support of the (float_value,string_value,structured_value).
- SQLTagsORM: new TagSubValues relation containing broken out values for values which are sequences, to support efficient lookup if sequence values such as log entry categories.
- New BaseSQLTagsCommand.parse_categories static method to parse FOO,BAH into ['foo','bah'].
- sqltags find: change default format to "{datetime} {headline}".
- Assorted small changes.
Release 20210404:
- SQLTags.getitem: when autocreating an entity, do it in a new session so that the entity is commited to the database before any further use.
- SQLTagsCommand: new cmd_dbshell to drop you into the database.
Release 20210321: Drop logic now merged with cs.sqlalchemy_utils, use the new default session stuff.
Release 20210306.1: Docstring updates.
Release 20210306: Initial release.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.