Embeddings¶

An Embedding is a per-detection appearance / re-identification feature vector. It is the bridge for multi-object tracking and re-ID workflows: a model produces one vector per detection crop, and those vectors are compared (typically by cosine similarity) to link detections to a global Identity.

Embedding is a bare value object — the vector is its only field:

Field	Type	Description
`vector`	`np.ndarray` `(D,)`	The feature vector. Floating dtype is preserved (e.g. `float32` from a network); non-floating input is cast to `float32`.

Embedding compares by value (two embeddings are equal when their vectors are element-wise equal) and is therefore unhashable. It exposes a dim property for the vector length. What the vector represents is implied by the slot it fills on the detection.

Per-detection slot¶

Every detection modality — Instance, Centroid, SegmentationMask, BoundingBox, ROI — carries a single identity_embedding: Embedding | None slot (the appearance vector used for re-identification):

>>> import numpy as np
>>> import sleap_io as sio
>>> from sleap_io import Embedding
>>> skeleton = sio.Skeleton(["head", "tail"])
>>> inst = sio.Instance.from_numpy(np.array([[0, 1], [2, 3]]), skeleton=skeleton)
>>> inst.identity_embedding = Embedding(np.ones(128, dtype="float32"))
>>> print(inst.identity_embedding.dim)
128

Because detection objects use object-identity equality, the (potentially large) embedding vector is never compared when two detections are compared. The identity_embedding is also propagated when converting between detection modalities (e.g. Instance.to_centroid()).

SLP persistence¶

Embeddings persist to SLP in format 2.5+ via the additive /embeddings group, stored as a single columnar struct-of-arrays:

/embeddings/
  vectors     (N, D) float32   # chunked so whole rows stay within a chunk, gzip compressed
  owner_type  (N,)   uint8      # OWNER_INSTANCE / CENTROID / MASK / BBOX / ROI
  owner_id    (N,)   int64      # global instance_id or per-modality list index

Row i of all three datasets describes the same detection. Keeping owner_type/owner_id as separate parallel datasets means more per-embedding attributes can be added later without re-laying-out vectors. The large float vectors live in their own chunked, gzipped dataset — never in a JSON blob or the fixed instance row layout — so the format stays additive: older readers ignore the group and embedding-free files round-trip unchanged.

All embedding vectors in a file must share the same dimensionality D (they come from one re-ID model); a mixed-D save raises ValueError.

Skipping appearance vectors on disk

Appearance vectors are large. Pass labels.save(path, save_embedding_vectors=False) to skip the /embeddings group entirely while still persisting identity links (the /identity group); the vectors stay in memory (e.g. to build identity prototypes). This is distinct from embed, which embeds video frames.

API reference¶

`sleap_io.Embedding` ¶

A per-detection appearance / re-identification embedding vector.

An Embedding wraps a single feature vector describing the visual appearance of one detection (e.g. the crop around an Instance, Centroid, SegmentationMask, or BoundingBox). What the vector represents is implied by the slot it fills on the detection -- currently identity_embedding on every detection modality, used for re-identification.

Attributes:

Name	Type	Description
`vector`		1-D feature vector of shape `(D,)`. The dtype is preserved from the input when floating (e.g. float32 from a neural network); non-floating inputs are cast to float32.

Notes

Embedding uses value equality: two embeddings are equal when their vectors are element-wise equal. It is therefore unhashable (an Embedding is only ever a field value, never a set member or dict key). Detection objects use object-identity equality, so a (potentially large) embedding vector is never compared when two detections are compared.

Methods:

Name	Description
`__eq__`	Method generated by attrs for class Embedding.
`__init__`	Method generated by attrs for class Embedding.
`__repr__`	Method generated by attrs for class Embedding.
`__setattr__`	Method generated by attrs for class Embedding.

Source code in sleap_io/model/embedding.py

@define
class Embedding:
    """A per-detection appearance / re-identification embedding vector.

    An `Embedding` wraps a single feature vector describing the visual appearance
    of one detection (e.g. the crop around an `Instance`, `Centroid`,
    `SegmentationMask`, or `BoundingBox`). What the vector represents is implied by
    the slot it fills on the detection -- currently ``identity_embedding`` on every
    detection modality, used for re-identification.

    Attributes:
        vector: 1-D feature vector of shape ``(D,)``. The dtype is preserved from
            the input when floating (e.g. float32 from a neural network);
            non-floating inputs are cast to float32.

    Notes:
        `Embedding` uses value equality: two embeddings are equal when their
        vectors are element-wise equal. It is therefore unhashable (an `Embedding`
        is only ever a field value, never a set member or dict key). Detection
        objects use object-identity equality, so a (potentially large) embedding
        vector is never compared when two detections are compared.
    """

    vector: np.ndarray = field(
        converter=_as_vector,
        eq=attrs.cmp_using(eq=np.array_equal),
        repr=lambda v: f"<{v.shape[0]}-d {v.dtype}>",
    )

    @property
    def dim(self) -> int:
        """Dimensionality ``D`` of the embedding vector."""
        return int(self.vector.shape[0])

`annotations = {'vector': 'np.ndarray'}` `class-attribute` ¶

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

`__attrs_own_setattr__ = True` `class-attribute` ¶

Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

`__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)` `class-attribute` ¶

Effective class properties as derived from parameters to attr.s() or define() decorators.

This is the same data structure that attrs uses internally to decide how to construct the final class.

Warning:

This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.

Attributes:

Name	Type	Description
`is_exception`	`bool`	Whether the class is treated as an exception class.
`is_slotted`	`bool`	Whether the class is `slotted <slotted classes>`.
`has_weakref_slot`	`bool`	Whether the class has a slot for weak references.
`is_frozen`	`bool`	Whether the class is frozen.
`kw_only`	`KeywordOnly`	Whether / how the class enforces keyword-only arguments on the `__init__` method.
`collected_fields_by_mro`	`bool`	Whether the class fields were collected by method resolution order. That is, correctly but unlike `dataclasses`.
`added_init`	`bool`	Whether the class has an attrs-generated `__init__` method.
`added_repr`	`bool`	Whether the class has an attrs-generated `__repr__` method.
`added_eq`	`bool`	Whether the class has attrs-generated equality methods.
`added_ordering`	`bool`	Whether the class has attrs-generated ordering methods.
`hashability`	`Hashability`	How `hashable <hashing>` the class is.
`added_match_args`	`bool`	Whether the class supports positional `match <match>` over its fields.
`added_str`	`bool`	Whether the class has an attrs-generated `__str__` method.
`added_pickling`	`bool`	Whether the class has attrs-generated `__getstate__` and `__setstate__` methods for `pickle`.
`on_setattr_hook`	`Callable[[Any, Attribute[Any], Any], Any] \| None`	The class's `__setattr__` hook.
`field_transformer`	`Callable[[Attribute[Any]], Attribute[Any]] \| None`	The class's `field transformers <transform-fields>`.

.. versionadded:: 25.4.0

doc = 'A per-detection appearance / re-identification embedding vector.\n\nAn `Embedding` wraps a single feature vector describing the visual appearance\nof one detection (e.g. the crop around an `Instance`, `Centroid`,\n`SegmentationMask`, or `BoundingBox`). What the vector represents is implied by\nthe slot it fills on the detection -- currently ``identity_embedding`` on every\ndetection modality, used for re-identification.\n\nAttributes:\n vector: 1-D feature vector of shape ``(D,)``. The dtype is preserved from\n the input when floating (e.g. float32 from a neural network);\n non-floating inputs are cast to float32.\n\nNotes:\n `Embedding` uses value equality: two embeddings are equal when their\n vectors are element-wise equal. It is therefore unhashable (an `Embedding`\n is only ever a field value, never a set member or dict key). Detection\n objects use object-identity equality, so a (potentially large) embedding\n vector is never compared when two detections are compared.\n' `class-attribute` ¶

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.

`firstlineno = 26` `class-attribute` ¶

int([x]) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.

int('0b100', base=0) 4

`__match_args__ = ('vector',)` `class-attribute` ¶

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

`module = 'sleap_io.model.embedding'` `class-attribute` ¶

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.

`slots = ('vector', 'weakref')` `class-attribute` ¶

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

`__static_attributes__ = ()` `class-attribute` ¶

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

`weakref` `property` ¶

list of weak references to the object

`dim` `property` ¶

Dimensionality D of the embedding vector.

`eq(other)` ¶

Method generated by attrs for class Embedding.

Source code in sleap_io/model/embedding.py

arr = np.asarray(value)
if not np.issubdtype(arr.dtype, np.floating):
    arr = arr.astype(np.float32)
if arr.ndim != 1:
    raise ValueError(
        f"Embedding vector must be 1-dimensional, got shape {arr.shape}."

`init(vector)` ¶

Method generated by attrs for class Embedding.

Source code in sleap_io/model/embedding.py

    )
return arr

`repr()` ¶

Method generated by attrs for class Embedding.

Source code in sleap_io/model/embedding.py

"""Embedding data structure for per-detection appearance / re-ID vectors."""

from __future__ import annotations

import attrs
import numpy as np
from attrs import define, field


def _as_vector(value) -> np.ndarray:
    """Coerce an input to a 1-D embedding vector, preserving floating dtype.

    Floating inputs (e.g. float32 from a neural network) keep their dtype;
    non-floating inputs are cast to float32.
    """

`setattr(name, val)` ¶

Method generated by attrs for class Embedding.

Embeddings¶

Per-detection slot¶

SLP persistence¶

API reference¶

sleap_io.Embedding ¶

__annotations__ = {'vector': 'np.ndarray'} class-attribute ¶

__attrs_own_setattr__ = True class-attribute ¶

__firstlineno__ = 26 class-attribute ¶

__match_args__ = ('vector',) class-attribute ¶

__module__ = 'sleap_io.model.embedding' class-attribute ¶

__slots__ = ('vector', '__weakref__') class-attribute ¶

__static_attributes__ = () class-attribute ¶

__weakref__ property ¶

dim property ¶

__eq__(other) ¶

__init__(vector) ¶

__repr__() ¶

__setattr__(name, val) ¶

`sleap_io.Embedding` ¶

`annotations = {'vector': 'np.ndarray'}` `class-attribute` ¶

`__attrs_own_setattr__ = True` `class-attribute` ¶

`firstlineno = 26` `class-attribute` ¶

`__match_args__ = ('vector',)` `class-attribute` ¶

`module = 'sleap_io.model.embedding'` `class-attribute` ¶

`slots = ('vector', 'weakref')` `class-attribute` ¶

`__static_attributes__ = ()` `class-attribute` ¶

`weakref` `property` ¶

`dim` `property` ¶

`eq(other)` ¶

`init(vector)` ¶

`repr()` ¶

`setattr(name, val)` ¶