Categories¶
A Category names the class an individual belongs to — a group of
detections that share some attribute, typically assigned by a classifier or retrieved via
re-ID (e.g. "female_fly", "male_fly", "fur_shaved", "mouse"). It is the third grouping
axis alongside Track and Identity:
| Concept | Scope | Question it answers |
|---|---|---|
Track |
within one video | which trajectory is this over time? (ephemeral) |
Identity |
across videos / sessions | which specific individual is this? (persistent) |
Category |
across individuals | which class does this belong to? (classification / re-ID) |
Where an Identity names one specific animal, a Category names a set of animals that share a
property. Many individuals map to one category.
The Category class¶
A Category has exactly two fields — the same shape as Identity and
Track:
| Field | Type | Description |
|---|---|---|
name |
str |
Human-readable class name (e.g. "female_fly"). Not required to be unique, but name is how categories are matched across separately-loaded files and merges. |
metadata |
dict[str, str] |
Arbitrary string-keyed, string-valued metadata (e.g. {"color": "#e6194b", "supercategory": "insect"}). Empty by default. |
>>> import sleap_io as sio
>>> female = sio.Category(name="female_fly", metadata={"color": "#e6194b"})
>>> male = sio.Category(name="male_fly")
>>> print(female.name)
female_fly
Like Track and Identity, Category uses object-identity equality (eq=False), so two
Category objects with the same name are distinct objects but still match by name — the key
that survives serialization and cross-file merges. Compare with matches() (default
method="name"); pass method="identity" to instead require the same Python object:
>>> import sleap_io as sio
>>> a1 = sio.Category(name="female_fly")
>>> a2 = sio.Category(name="female_fly")
>>> print(a1.matches(a2)) # same name -> same class
True
>>> print(a1.matches(a2, method="identity")) # distinct objects -> no match
False
No dedicated color
There is no color field on a Category. If a visualization color is desired, store it as a
conventional metadata entry such as metadata["color"] = "#e6194b"; it persists like any other
metadata key. Coloring by category uses the palette index into Labels.categories order
(identical to color-by-identity), not a per-category color.
Per-detection slots¶
Every detection modality — Instance, Centroid,
SegmentationMask, BoundingBox, ROI — carries a
trio of category slots, mirroring the identity trio (identity / identity_score /
identity_embedding):
| Slot | Type | Description |
|---|---|---|
category |
Category \| None |
The assigned class. |
category_score |
float \| None |
Classification / assignment confidence. |
category_embedding |
Embedding \| None |
The appearance vector the class was predicted from. |
>>> import numpy as np
>>> import sleap_io as sio
>>> skeleton = sio.Skeleton(["head", "tail"])
>>> female = sio.Category(name="female_fly")
>>> inst = sio.Instance.from_numpy(
... np.array([[0, 1], [2, 3]]),
... skeleton=skeleton,
... category=female,
... category_score=0.97,
... category_embedding=sio.Embedding(np.ones(64, dtype="float32")),
... )
>>> print(inst.category.name, inst.category_score, inst.category_embedding.dim)
female_fly 0.97 64
The trio is propagated when converting between detection modalities (e.g.
Instance.to_centroid(), Centroid.to_bbox()), exactly like the identity trio.
Promotion of the legacy category string¶
Older code set a free-form category: str class label directly on bounding boxes, centroids,
ROIs, and masks (the object-detection class, e.g. category="mouse"). That field is now the
first-class Category slot, with a str -> Category converter so existing call sites keep
working — the empty-string "unset" sentinel maps to None:
>>> import sleap_io as sio
>>> bbox = sio.UserBoundingBox(x1=0, y1=0, x2=10, y2=10, category="mouse")
>>> print(bbox.category) # promoted to a Category
Category(name="mouse")
>>> unset = sio.UserBoundingBox(x1=0, y1=0, x2=10, y2=10)
>>> print(unset.category) # "" / omitted -> None
None
The catalog: Labels.categories¶
Labels.categories is the top-level catalog of Category objects — a list,
like Labels.identities and Labels.tracks — auto-collected
in first-seen order from the detections on save:
>>> import numpy as np
>>> import sleap_io as sio
>>> skeleton = sio.Skeleton(["head", "tail"])
>>> female = sio.Category(name="female_fly")
>>> male = sio.Category(name="male_fly")
>>> video = sio.Video(filename="clip.mp4", open_backend=False)
>>> inst_f = sio.Instance.from_numpy(
... np.array([[0, 1], [2, 3]]), skeleton=skeleton, category=female
... )
>>> inst_m = sio.Instance.from_numpy(
... np.array([[4, 5], [6, 7]]), skeleton=skeleton, category=male
... )
>>> lf = sio.LabeledFrame(video=video, frame_idx=0, instances=[inst_f, inst_m])
>>> labels = sio.Labels(labeled_frames=[lf], categories=[female, male])
>>> print([c.name for c in labels.categories])
['female_fly', 'male_fly']
Save / load round-trip¶
The category catalog, the per-detection category / category_score links, and the
category_embedding appearance vectors persist to SLP in format 2.7+ (additive — older
readers ignore them and category-free files round-trip unchanged):
>>> import os
>>> import tempfile
>>> import numpy as np
>>> import sleap_io as sio
>>> skeleton = sio.Skeleton(["head", "tail"])
>>> female = sio.Category(name="female_fly")
>>> inst = sio.Instance.from_numpy(
... np.array([[0, 1], [2, 3]]),
... skeleton=skeleton,
... category=female,
... category_embedding=sio.Embedding(np.ones(64, dtype="float32")),
... )
>>> video = sio.Video(filename="clip.mp4", open_backend=False)
>>> lf = sio.LabeledFrame(video=video, frame_idx=0, instances=[inst])
>>> labels = sio.Labels(labeled_frames=[lf], categories=[female])
>>> path = os.path.join(tempfile.mkdtemp(), "cats.slp")
>>> sio.save_slp(labels, path, save_embedding_vectors=True)
>>> loaded = sio.load_slp(path)
>>> print([c.name for c in loaded.categories])
['female_fly']
>>> print(loaded[0][0].category.name, loaded[0][0].category_embedding.dim)
female_fly 64
Pass save_slp(..., save_embedding_vectors=False) to persist the category links (which
detection is which class) while skipping the large appearance vectors — the same gate used for
identity embeddings. See Formats → SLP and
Embeddings.
Merging: deduping the catalog¶
When merging files, the category catalog is deduped by a CategoryMatcher
— by default matching on name, so two files that both use "female_fly" collapse to a single
catalog entry:
>>> import numpy as np
>>> import sleap_io as sio
>>> skeleton = sio.Skeleton(["head", "tail"])
>>> def make():
... female = sio.Category(name="female_fly")
... inst = sio.Instance.from_numpy(
... np.array([[0, 1], [2, 3]]), skeleton=skeleton, category=female
... )
... video = sio.Video(filename="clip.mp4", open_backend=False)
... lf = sio.LabeledFrame(video=video, frame_idx=0, instances=[inst])
... return sio.Labels(labeled_frames=[lf], categories=[female])
>>> base, other = make(), make()
>>> _ = base.merge(other, category="name", frame="keep_both")
>>> print([c.name for c in base.categories]) # same-named categories deduped
['female_fly']
Pass category="identity" to instead require the same Python object (no name-based dedup). See
Merging.
Coloring by category¶
render_image / render_video accept color_by="category", which assigns one palette color per
category by its index in Labels.categories order (identical plumbing to color_by="identity").
Detections without a category fall back to index 0:
import sleap_io as sio
labels = sio.load_slp("classified.slp")
img = sio.render_image(labels[0], color_by="category")
sio.render_video(labels, "by_category.mp4", color_by="category")
From the CLI:
See Rendering and the CLI reference.
API reference¶
sleap_io.Category
¶
Ground-truth class membership of a detection (e.g. species, sex, condition).
Where Track is an ephemeral temporal trajectory within a single video and
Identity names a specific individual across videos, Category names the
class an individual belongs to -- a group of individuals that share some
attribute, typically assigned by classification or retrieved via re-ID (e.g.
"female_fly", "fur_shaved", "mouse"). The per-detection binding is
stored on Instance.category (and the analogous slot on the other detection
modalities), alongside an optional category_score (assignment confidence)
and category_embedding (the appearance vector it was classified from).
Attributes:
| Name | Type | Description |
|---|---|---|
name |
Human-readable name for this category (e.g., |
|
metadata |
Arbitrary string-keyed, string-valued metadata (e.g.
|
Notes
Category objects use object-identity equality (eq=False), matching
Track and Identity. Use matches() (default method="name") to
compare categories across files, where Python object identity is not
meaningful.
Methods:
| Name | Description |
|---|---|
__init__ |
Method generated by attrs for class Category. |
__repr__ |
Return a readable string representation. |
__setattr__ |
Method generated by attrs for class Category. |
matches |
Check if this category matches another category. |
Source code in sleap_io/model/category.py
@define(eq=False)
class Category:
"""Ground-truth class membership of a detection (e.g. species, sex, condition).
Where `Track` is an ephemeral temporal trajectory within a single video and
`Identity` names a specific individual across videos, `Category` names the
*class* an individual belongs to -- a group of individuals that share some
attribute, typically assigned by classification or retrieved via re-ID (e.g.
``"female_fly"``, ``"fur_shaved"``, ``"mouse"``). The per-detection binding is
stored on ``Instance.category`` (and the analogous slot on the other detection
modalities), alongside an optional ``category_score`` (assignment confidence)
and ``category_embedding`` (the appearance vector it was classified from).
Attributes:
name: Human-readable name for this category (e.g., ``"female_fly"``). Not
required to be unique, but ``name`` is how categories are matched
across separately-loaded files and merges.
metadata: Arbitrary string-keyed, string-valued metadata (e.g.
``{"color": "#e6194b", "supercategory": "insect"}``). Empty by default.
Notes:
`Category` objects use object-identity equality (``eq=False``), matching
`Track` and `Identity`. Use `matches()` (default ``method="name"``) to
compare categories across files, where Python object identity is not
meaningful.
"""
name: str = field(default="", validator=instance_of(str))
metadata: dict[str, str] = field(factory=dict, validator=instance_of(dict))
def matches(self, other: "Category", method: str = "name") -> bool:
"""Check if this category matches another category.
Args:
other: Another category to compare with.
method: Matching method:
- ``"name"`` (default): match by the `name` attribute, which
survives serialization and cross-file merges.
- ``"identity"``: match by Python object identity (same object).
Returns:
True if the categories match according to the specified method.
Raises:
ValueError: If `method` is not one of the supported values.
"""
if method == "name":
return self.name == other.name
elif method == "identity":
return self is other
else:
raise ValueError(f"Unknown matching method: {method}")
def __repr__(self) -> str:
"""Return a readable string representation."""
return f'Category(name="{self.name}")'
__annotations__ = {'name': 'str', 'metadata': 'dict[str, str]'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = True
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=False, added_eq=False, added_ordering=False, hashability=<Hashability.LEAVE_ALONE: 'leave_alone'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Ground-truth class membership of a detection (e.g. species, sex, condition).\n\nWhere `Track` is an ephemeral temporal trajectory within a single video and\n`Identity` names a specific individual across videos, `Category` names the\n*class* an individual belongs to -- a group of individuals that share some\nattribute, typically assigned by classification or retrieved via re-ID (e.g.\n``"female_fly"``, ``"fur_shaved"``, ``"mouse"``). The per-detection binding is\nstored on ``Instance.category`` (and the analogous slot on the other detection\nmodalities), alongside an optional ``category_score`` (assignment confidence)\nand ``category_embedding`` (the appearance vector it was classified from).\n\nAttributes:\n name: Human-readable name for this category (e.g., ``"female_fly"``). Not\n required to be unique, but ``name`` is how categories are matched\n across separately-loaded files and merges.\n metadata: Arbitrary string-keyed, string-valued metadata (e.g.\n ``{"color": "#e6194b", "supercategory": "insect"}``). Empty by default.\n\nNotes:\n `Category` objects use object-identity equality (``eq=False``), matching\n `Track` and `Identity`. Use `matches()` (default ``method="name"``) to\n compare categories across files, where Python object identity is not\n meaningful.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 9
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('name', 'metadata')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.category'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('name', 'metadata', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__init__(name='', metadata=NOTHING)
¶
Method generated by attrs for class Category.
__repr__()
¶
__setattr__(name, val)
¶
Method generated by attrs for class Category.
matches(other, method='name')
¶
Check if this category matches another category.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
other
|
Category
|
Another category to compare with. |
required |
method
|
str
|
Matching method:
|
'name'
|
Returns:
| Type | Description |
|---|---|
bool
|
True if the categories match according to the specified method. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in sleap_io/model/category.py
def matches(self, other: "Category", method: str = "name") -> bool:
"""Check if this category matches another category.
Args:
other: Another category to compare with.
method: Matching method:
- ``"name"`` (default): match by the `name` attribute, which
survives serialization and cross-file merges.
- ``"identity"``: match by Python object identity (same object).
Returns:
True if the categories match according to the specified method.
Raises:
ValueError: If `method` is not one of the supported values.
"""
if method == "name":
return self.name == other.name
elif method == "identity":
return self is other
else:
raise ValueError(f"Unknown matching method: {method}")