Changelog¶

v0.9.2 ¶

sleap-io v0.9.2 Release Notes

Summary

sleap-io v0.9.2 is a focused patch release that fixes a major I/O scalability bug in multi-camera RecordingSession storage. Previously, an entire session — calibration, camera↔video mapping, and every frame group's inline 3D points — was serialized as one JSON string in sessions_json. On a real 3-camera, 108,000-frame project that string reached 524 MB, 78.6% of which was 3D point text, and HDF5 variable-length strings can't be read back past roughly a 0.45 GB ceiling in JS/WASM — making such files effectively unreadable in browser-based tooling. The bulk numeric data now lives in a dedicated columnar /session_data HDF5 group, dropping sessions_json to single-digit MB on that project.

The SLP format advances 2.7 → 2.8 — format 2.8 adds the /session_data group. It's additive and read-on-group-presence: pre-2.8 files load unchanged, and files with no camera sessions stay at format ≤2.7.

Highlights:

Columnar RecordingSession storage (#546) — moves per-frame 3D point data out of sessions_json into a chunked, gzip-compressed /session_data HDF5 group, referenced by row range (mirroring how 2D /points are referenced from /instances). Fully backward compatible; no public API changes. A real 108k-frame project's sessions_json shrinks from 524 MB to single-digit MB.
h5wasm identity/category link interop fix (#548) — read_identity_links and read_category_links now accept the flat-2D + field_names table encoding that sleap-io.js (via h5wasm) writes, matching every other structured-table reader. Only affects cross-language interop with sleap-io.js-written files.
Storage Representation Matrix docs (#549) — docs/formats/slp.md gains a reference table of every on-disk data structure's representation, introducing format version, and read/write support at format 2.8.

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

Improvements

Columnar `RecordingSession` storage: 3D points move out of `sessions_json` (#546)

RecordingSession (multi-camera calibration) was serialized as a single JSON string per session in the sessions_json HDF5 dataset — calibration, the camcorder_to_video_idx_map, and every frame group with its inline 3D points, all as text. On a real 3-camera, 108,000-frame project that string reached 524 MB, of which 412.7 MB (78.6%) was 3D point text. HDF5 variable-length strings can't be read back past roughly a 0.45 GB ceiling in JS/WASM, so such files were effectively unreadable in browser-based tooling (the coordinated sleap-io.js port).

The unbounded per-frame numeric payload now lives in a dedicated columnar /session_data HDF5 group, referenced by row range from a slim sessions_json — the same pattern already used for how 2D /points are referenced from /instances:

sessions_json (slim): calibration + camcorder_to_video_idx_map + session metadata + an fg_start/fg_end range into session_data/frame_groups.
/session_data/frame_groups — (frame_idx, ig_start, ig_end)
/session_data/instance_groups — (identity_idx, score, instance_3d_score, pts3d_start/end, pts3d_predicted, member_start/end)
/session_data/instance_group_members — (camera, lf, inst), the columnarized camcorder_to_lf_and_inst_idx_map
/session_data/points_3d (N,3) / pred_points_3d (N,4 = xyz+score) — chunked, gzip-compressed float matrices; NaN rows denote missing keypoints
/session_data/frame_group_meta / instance_group_meta — optional per-row JSON blobs for lossless typed metadata

On the reported dataset, sessions_json drops from 524 MB to single-digit MB.

Format version: 2.7 → 2.8, bumped only when a session actually has frame groups; session-free/single-view files stay at ≤2.7 and are byte-identical to before.

Compatibility: Fully backward and forward compatible. The reader dispatches on /session_data presence and still parses legacy inline frame_group_dicts from ≤2.7 files. An older (<2.8) reader opening a 2.8 file loads calibration from the slim sessions_json and silently ignores /session_data — no corruption, just absent 3D data. No public API changes: labels.sessions[...].frame_groups[...].instance_groups[...].points works identically; this is purely an on-disk representation change.

Handled consistently across eager I/O (incremental per-session append, bounded peak memory), lazy I/O (raw passthrough — also fixes a prior bug where lazy re-save could silently drop frame groups/3D data), and streaming/remote reads (chunked datasets over fsspec range reads, removing the vlen-string ceiling for large remote files).

Fixes

Read h5wasm flat-2D identity/category links (#548)

read_identity_links and read_category_links read /identity/links and /categories/links via direct structured field access only, with no fallback. h5wasm — the HDF5 layer sleap-io.js writes through — can't create true HDF5 compound datasets, so it stores every structured table as a flat 2-D f8 array plus a field_names attribute, the same convention already handled for points/pred_points/instances/frames, and for /session_data in #546. These two per-detection link readers were the one place not yet wired through the shared flat-2D conversion helper, so a links table written by sleap-io.js raised on read in Python.

Both readers now route through the existing shared conversion helper, which rebuilds a structured array when a field_names attribute is present and passes a genuine compound dataset through unchanged. Only cross-language interop is affected — files written by sleap-io.js (h5wasm) that are read back by Python sleap-io. Plain Python-written .slp/.pkg.slp files are read identically before and after this fix. Closes #547.

Documentation

New Storage Representation Matrix section in docs/formats/slp.md cataloging, for every on-disk data structure at format 2.8, its storage representation (compound, columnar group, EAV, plain matrix, ragged CSR, string, JSON, or attribute), the format version that introduced it, and read/write support — complementing the existing Version History and browser-compatibility sections.
Documents the write/read asymmetry: Python always writes a single native representation per structure, while compound-kind readers additionally accept the sleap-io.js h5wasm flat-2D + field_names encoding for browser interop.

Changelog

#546: feat(io): Columnar RecordingSession storage: move 3D points out of sessions_json (SLP 2.8)
#548: fix(io): read h5wasm flat-2D identity/category links (coordinated sleap-io.js port)
#549: docs(slp): add Storage Representation Matrix (format 2.8)

Full Changelog: v0.9.1...v0.9.2

v0.9.1 ¶

sleap-io v0.9.1 Release Notes

Summary

sleap-io v0.9.1 is a focused patch release pairing one new modeling concept with a major I/O performance fix. It adds first-class Category — class membership as a third grouping axis alongside Track and Identity — and eliminates a tens-of-minutes silent stall when embedding image-sequence frames into a .pkg.slp (a real 15,000-frame project drops from ~32 minutes to ~4 seconds). Two small, mechanical breaking changes ride along: .category on boxes/centroids/ROIs/masks is now a Category object rather than a bare string, and the SLP embed progress_callback gains a third phase argument.

The SLP format advances 2.6 → 2.7 — 2.7 adds the /categories group (mirroring identity). It is additive and read-on-group-presence, so older readers ignore it and pre-2.7 files load unchanged (identity-only files stay byte-identical).

Highlights:

First-class Category (#542) — a named catalog entry (name + string metadata + matches()) mirroring Identity, attached per detection via category / category_score / category_embedding, collected into Labels.categories, colored by render --color-by category, and persisted to .slp (format 2.7). Motivated by the move toward object detection, where the object class (COCO "category") is a first-class citizen.
Embedding writes are no longer slow or silent (#543) — dropping gzip on already-compressed frame bytes turns a tens-of-minutes post-progress-bar stall into seconds; a real 15,000-frame image sequence went ~32 min → ~4 s end-to-end.
ImageVideo byte-copy fast path (#543) — embedding a PNG/JPEG image sequence now copies the source bytes verbatim: lossless (no re-encode artifacts), faster (no decode/encode), and smaller (~3.9 GB → ~2.0 GB on that project).
Write-phase progress (#543) — a second Writing frames progress bar (and a "write" phase for progress_callback) so the write phase no longer looks hung.

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

Breaking Changes

v0.9.1 is a patch release with two small, mechanical breaking changes.

`.category` is now a `Category`, not a `str` (#542)

The pre-existing free-form category: str field on BoundingBox, Centroid, ROI, and SegmentationMask is promoted to category: Category | None. Construction is unchanged — a str → Category converter keeps existing code working — but code that reads .category as a string must now use .category.name (and treat None as unset).

# Construction is unchanged — a bare class-label string still works
bbox = sio.UserBoundingBox(x1=0, y1=0, x2=10, y2=10, category="mouse")

# Before (0.9.0): a plain string
bbox.category            # "mouse"

# After (0.9.1): a first-class Category (empty string -> None)
bbox.category            # Category(name="mouse")
bbox.category.name       # "mouse"

The on-disk SLP string format is unchanged — the bbox/centroid/roi_categories/mask_categories datasets still store the plain name string, so files round-trip identically. LabelImage.Info.category also remains a plain str (it belongs to the separate panoptic-mask subsystem and is unaffected).

SLP embed `progress_callback` gains a `phase` argument (#543)

The embed progress_callback is now invoked in two phases and takes a third argument:

# Before:  progress_callback(current, total) -> bool
# After:   progress_callback(current, total, phase) -> bool   # phase: "embed" | "write"

"embed" fires while frames are loaded/encoded/byte-copied into memory; "write" fires while bytes are flushed to the HDF5 file (the newly-instrumented write phase). Returning a truthy value from either phase still cancels the export.

Any caller passing a 2-argument callback to write_labels / save_slp / save_file will break and must accept the third phase argument. The SLEAP GUI is the main such caller; it is version-fenced to an older sleap-io, so this is safe until that pin is bumped.

New Features

First-class `Category`: class membership alongside `Track` and `Identity` (#542)

Category completes a trio of grouping axes for detections. Where Track links the same individual within a video and Identity links the same individual across videos, Category groups individuals of the same class — the natural home for a classifier or object-detector's class label.

Concept	Scope	Example
`Track`	same individual within a video (tracking)	`track_0`
`Identity`	same individual across videos (unique ID)	`C57BL6.cohort2.12`
`Category`	individuals of the same class (classified / detected)	`female_fly`, `fur_shaved`

Category is a direct mirror of Identity: a named catalog entry with arbitrary string metadata, matched by name (default) or object identity. Every detection modality (Instance, PredictedInstance, BoundingBox, Centroid, ROI, SegmentationMask) gains category, category_score, and category_embedding slots; InstanceGroup gains a bare category. Labels.categories is a catalog auto-collected from the detections, exactly like Labels.identities.

import numpy as np
import sleap_io as sio

# The promoted free-form class-label string still works, and is now a Category
bbox = sio.UserBoundingBox(x1=0, y1=0, x2=10, y2=10, category="mouse")
bbox.category                    # Category(name="mouse")

# Full first-class use: catalog entry + per-detection score + embedding
fly = sio.Category(name="female_fly", metadata={"sex": "F"})
inst = sio.Instance.from_numpy(
    np.array([[10.0, 20.0], [30.0, 40.0]]),
    skeleton=skeleton,
    category=fly,
    category_score=0.94,
    category_embedding=sio.Embedding(np.random.rand(128).astype("float32")),
)

labels = sio.Labels(
    labeled_frames=[sio.LabeledFrame(video=video, frame_idx=0, instances=[inst])]
)
labels.categories                # [Category(name="female_fly")]

# Categories match by name, like Identity and Track
assert fly.matches(sio.Category(name="female_fly"))

sio.save_slp(labels, "out.slp", save_embedding_vectors=True)   # format 2.7

Persistence (SLP format 2.7). Category links are written to a new self-contained /categories group (a name catalog, an EAV metadata table, and a links dataset carrying category_idx / category_score), mirroring /identity. Category embeddings are stored as parallel category_* datasets alongside the identity vectors in /embeddings, so the two embedding kinds are independent and need not share dimensionality. As with identity/re-ID embeddings, appearance vectors stay off disk unless you pass save_embedding_vectors=True (default False, mirroring embed).

Tooling. sio merge --category {name,identity} controls how the category catalog is deduplicated on merge; sio render --color-by category colors renders by class; and sio show reports category counts (--json includes n_categories, a categories[] listing, and n_instances_with_category_embedding).

sio render preds.slp -o out.mp4 --color-by category
sio merge a.slp b.slp -o merged.slp --category name
sio show preds.slp --json          # includes n_categories, categories[], ...

Because the class label was already a string field, every reader and writer that consumed it — the COCO, Ultralytics, and JABS readers/writers, the GeoJSON writer, roi.__geo_interface__, and the get_*(category=...) filters — now reads .category.name, so those formats round-trip unchanged. See docs/model/category.md.

Improvements

Embedding writes are no longer slow or silent (#543)

Embedding video frames into a .pkg.slp could spend tens of minutes silently writing to disk after the Embedding frames progress bar had already finished. The cause was gzip on encoded frames: PNG/JPEG bytes are already entropy-coded, so gzip only compressed the fixed-length zero padding while forcing chunked storage. guess_chunk then picked a tall (~100–235 row) chunk, and the row-by-row writes forced a repeated read → decompress → modify → recompress → rewrite of every chunk, compounded by the 1 MB default chunk-cache thrashing.

Encoded frames are now stored uncompressed and contiguous (only the raw-array hdf5 format still uses gzip). Measured on 15,000 frames (0.56 GB of encoded bytes):

strategy	time	file
gzip + auto-chunk (old)	>100 s	0.56 GB
uncompressed contiguous (new)	1.8 s	0.60 GB

The tradeoff is that fixed-length zero padding is no longer compressed away, so files can be modestly larger when per-frame sizes vary widely; fixed_length=False (vlen) remains available for maximum compactness.

`ImageVideo` byte-copy fast path (#543)

When embedding an image sequence of PNG/JPEG files, the source bytes are now copied verbatim — no decode/re-encode. The embedded dataset's @format follows the source (e.g. jpg) with @channel_order RGB. This is lossless (byte-identical to the source, no added JPEG artifacts), faster (the "Embedding frames" phase speeds up too, since it skips decode+encode), and smaller (it stores the compact source bytes instead of a larger re-encoded PNG). On the 15,000-frame project above, the package shrank from ~3.9 GB (re-encoded PNG) to ~2.0 GB (byte-copied JPEG). MediaVideo (mp4) sources still decode and re-encode to PNG.

sio embed mars_top.slp -o mars_top.pkg.slp
# Embedding frames: 100%|██████████| 15000/15000 [00:01<00:00, 12071 it/s]
# Writing frames:   100%|██████████| 15000/15000 [00:02<00:00,  5578 it/s]

Progress on the write phase (#543)

The HDF5 write loop now reports progress: a second Writing frames tqdm bar on the CLI (shown after Embedding frames) and a "write" phase for progress_callback — so a long write no longer looks like a hang.

Fixes

⚠️ Spilled `source_video` metadata made embedded packages unreadable (#543)

Fixed: For projects whose source_video metadata (e.g. the filename list of a large image sequence) exceeds HDF5's 64 KB attribute limit, that metadata is spilled to a source_video/json dataset (since #516, which only taught the label loader to read it). HDF5Video.__attrs_post_init__ still read the source_video/@json attribute directly, so it raised KeyError: can't locate attribute: 'json', left Video.backend as None, and made the embedded frames unreadable — breaking sio split and frame access on any such package. The backend now reads the dataset-or-attribute, mirroring the loader.

Documentation

New docs/model/category.md (class membership: Category, per-detection slots, catalog, and persistence).
docs/formats/slp.md — format-version history extended through 2.7 and the embed storage-layout change (uncompressed contiguous encoded frames, ImageVideo byte-copy).
docs/cli.md — merge --category, render --color-by category, and category reporting in show / --json.

Changelog

#542: feat(model,io,cli): first-class Category (class membership) mirroring Identity (@talmo)
#543: perf(io): fix slow/silent embed write; ImageVideo byte-copy; write-phase progress (@talmo)

Full Changelog: v0.9.0...v0.9.1

v0.9.0 ¶

sleap-io v0.9.0 Release Notes

Summary

sleap-io v0.9.0 is a feature release that grows the annotation model in two new directions and makes every representation freely interconvertible. It adds a re-identification subsystem — a global Identity catalog plus per-detection appearance Embeddings that attach to every detection modality and round-trip through .slp — and frame-spanning Event annotations, the library's first annotation with a temporal extent (behavior bouts, stimulus epochs, review flags) with an ethogram catalog and inclusive frame intervals. Alongside these, unified modality interconversion gives pose, centroid, bounding box, segmentation mask, and ROI a single shared verb set (.to_centroid() / .to_bbox() / .to_roi() / .to_mask(), plus Centroid.to_pose()) with batch LabeledFrame.convert() / Labels.convert() entry points. It rounds out with name-based skeleton symmetry inference, a sio.download() primitive and sio download CLI, machine-readable --json CLI inspection, large-project save hardening (past HDF5's 64 KB per-attribute limit), an O(N) merge speedup, and pynwb 4 compatibility.

The SLP format advances 2.4 → 2.6 — 2.5 adds the identity/embedding groups and 2.6 adds events. Both are additive and read-on-group-presence, so older readers ignore them and new files still load everywhere.

Highlights:

Re-identification subsystem (#513, #514, #515, #527, #535, #536) — new Identity (a named, cross-file ground-truth animal identity) and Embedding (a per-detection appearance vector) attach to every detection via identity / identity_score / identity_embedding, collect into Labels.identities, color renders (render --color-by identity), and persist to .slp (format 2.5). Appearance vectors stay off disk by default (save_embedding_vectors=False, mirroring embed).
Frame-spanning Event annotations (#540) — EventType / UserEvent / PredictedEvent represent anything with a temporal extent over an inclusive [start_frame, end_frame] interval, with optional subject/target participants (Track or Identity) and framewise or scalar prediction confidence. Query with labels.get_events(...) / labels.events_at(video, frame_idx); persist to format 2.6.
Unified modality interconversion (#531) — pose ⇄ centroid ⇄ bbox ⇄ mask ⇄ ROI share one verb set and batch convert() entry points, preserving track/identity/score metadata and the User/Predicted variant throughout.
Infer left/right symmetries from node names (#534) — Skeleton.infer_symmetries_by_name() suggests symmetric pairs from names like eye_L/ eye_R or left_paw/right_paw (opt-in, non-mutating).
sio.download() + sio download CLI (#528) — fetch a remote file straight to disk (http(s), cloud buckets, Google Drive) without loading it — a protocol-agnostic curl/wget replacement with streaming, atomic writes, and skip-if-exists.
Machine-readable CLI inspection (#538) — sio show --json and sio filenames --json emit structured JSON for scripting; default human-readable output is unchanged.
Reliable saving for very large projects (#517, #521, #523, #524) — .slp files no longer fail to save (or silently drop metadata) when provenance, merge history, or per-video source metadata exceeds HDF5's 64 KB per-attribute limit.
Faster merges (#537) — appending merges are now O(N) instead of O(N²) (a reported ~218s → ~0.75s on a 9k-frame merge into a ~95k-frame project).
pynwb 4 compatibility (#532) — NWB export works under pynwb 4.0 (still supports <4).

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

Breaking Changes

v0.9.0 is overwhelmingly additive. Two small, easily-migrated changes:

`Identity.color` removed — fold color into `metadata` (#535)

As part of finalizing the re-ID data model, the Identity type's dedicated color field was removed in favor of its general string metadata map. Identity.metadata is now typed dict[str, str] (string values, required for .slp persistence).

# Before (0.8.0)
ident = sio.Identity(name="mouse_A", color="#e6194b")

# After (0.9.0) — color lives in metadata
ident = sio.Identity(name="mouse_A", metadata={"color": "#e6194b"})

Only affects code that set Identity(color=...). Identity was introduced in 0.7.0 for 3D multi-view binding and was not yet persisted, so most pipelines are unaffected.

`Labels.merge()` bounds `merge_history` to 1000 records by default (#521)

To keep provenance from growing without bound (and overflowing HDF5's 64 KB attribute limit), Labels.merge() now caps provenance["merge_history"] at the most recent 1000 records by default (oldest trimmed first). Only observable if you merge more than 1000 times and inspect the full history.

# Keep the complete, unbounded merge history (previous behavior)
base.merge(other, max_merge_history=None)

DEFAULT_MERGE_HISTORY_LIMIT = 1000 is exposed on sleap_io.model.labels.

New Features

Re-identification subsystem: `Identity` + `Embedding` (#513, #514, #515, #527, #535, #536)

v0.9.0 adds a re-identification (re-ID) subsystem for tracking known animals across videos, sessions, and experiments. Two model concepts:

Identity — a named, cross-file ground-truth animal identity with arbitrary string metadata. Matched by name (default) or object identity, like Track.
Embedding — a 1-D per-detection appearance / re-ID feature vector.

Every detection modality (Instance, PredictedInstance, Instance3D, PredictedInstance3D, bounding boxes, centroids, masks, ROIs, label images) gains identity, identity_score, and identity_embedding slots; InstanceGroup.identity binds a triangulated multi-view group. Labels.identities is a catalog auto-collected from the detections.

import numpy as np
import sleap_io as sio

mouse_a = sio.Identity(name="mouse_A", metadata={"strain": "C57BL/6"})

inst = sio.Instance.from_numpy(
    np.array([[10.0, 20.0], [30.0, 40.0]]),
    skeleton=skeleton,
    identity=mouse_a,
    identity_score=0.98,
    identity_embedding=sio.Embedding(np.random.rand(128).astype("float32")),
)

# Identities match by name across files
assert mouse_a.matches(sio.Identity(name="mouse_A"))

Persistence (SLP format 2.5). Identity links are written to a new /identity group and appearance vectors to a new /embeddings group. To keep large vectors off disk, save_slp / save_file gain save_embedding_vectors which defaults to False (mirroring the embed=False default) — identity links always persist, vectors only when you ask (#536):

sio.save_slp(labels, "out.slp", save_embedding_vectors=True)   # include vectors

Tooling. sio merge --identity {name,identity} controls how the identity catalog is deduplicated on merge; sio render --color-by identity colors renders by global identity (one palette color per entry in Labels.identities); and sio show reports identity and embedding counts (--json includes n_instances_with_identity_embedding). See docs/model/embedding.md.

Frame-spanning `Event` annotations (#540)

sleap-io gains its first annotation with a temporal extent. Unlike per-frame annotations that live on a single LabeledFrame, an Event spans an inclusive [start_frame, end_frame] interval and lives on Labels.events, alongside a controlled-vocabulary catalog Labels.event_types. Use it for behavior bouts, stimulus epochs, physiological events, or review flags.

Four new classes are exported: EventType (the ethogram catalog entry), the abstract Event, and its concrete UserEvent (ground truth) and PredictedEvent (adds an optional framewise scores trace and a scalar score). Events carry optional subject/target participants (each a Track or Identity), plus name, source, and string metadata.

import numpy as np
import sleap_io as sio
from sleap_io.model.event import UserEvent, PredictedEvent

video = sio.Video(filename="session.mp4")
labels = sio.Labels(videos=[video])

# Human-annotated behavior bout over frames 100–150 (inclusive)
labels.events.append(UserEvent(type="attack", video=video, start_frame=100, end_frame=150))

# Model-predicted event with a framewise confidence trace + scalar score
labels.events.append(
    PredictedEvent(
        type="rear", video=video, start_frame=10, end_frame=12,
        scores=np.array([0.8, 0.9, 0.7]), score=0.85,
    )
)

# What is happening at frame 120? (span-covering, not per-frame)
labels.events_at(video, 120)                    # -> [UserEvent(type="attack", ...)]
labels.get_events(type="rear", predicted=True)  # -> [PredictedEvent(type="rear", ...)]

labels.save("out.slp")   # persists via SLP format 2.6

Labels.get_events(video, subject, type, frame_idx, predicted) filters the collection (frame_idx matches any event whose span covers the frame); Labels.events_at(video, frame_idx) is the "what's happening now?" convenience wrapper. Events persist to a new /event_types + /events SLP group pair (format 2.6). See docs/model/events.md.

Scope note: events persist only in .slp. Converting to NWB, Label Studio, JABS, or a DataFrame drops them — sio convert prints an explicit warning when it does.

Unified interconversion between detection modalities (#531)

Pose, centroid, bounding box, segmentation mask, and ROI now share one verb vocabulary, so you can freely move between detection representations while preserving metadata.

import sleap_io as sio

inst = labels[0].instances[0]

# Anchor a crop centroid on the thorax, fall back to center-of-mass
centroid = inst.to_centroid(method="anchor", node="thorax", fallback="center_of_mass")
box  = inst.to_bbox(padding=4, rotated=True)               # oriented, padded bbox
mask = inst.to_mask(height=H, width=W, node_radius=6, edge_radius=3)

# Batch across a frame or the whole dataset
labels.convert(to="bbox", source="mask", padding=4, inplace=True)

# Round-trip a centroid back into a single-node pose
inst2 = centroid.to_pose()

Each of Instance, Centroid, BoundingBox, SegmentationMask, and ROI gains to_centroid(), to_bbox(), to_roi(), and to_mask() (with Centroid.to_pose() / from_pose() closing the loop), and LabeledFrame.convert(to, source, ...) / Labels.convert(...) reach every cell of the matrix in one call. Conversions preserve the User/Predicted variant (carrying score on predicted paths) and propagate track, tracking_score, identity / identity_score / identity_embedding, and an instance= backref. Degenerate inputs return an empty target object (check the new is_empty property) unless you pass error_on_empty=True.

As part of this, SegmentationMask.to_polygon() now returns a PredictedROI (with score) for predicted masks instead of downcasting to UserROI. The old Centroid.to_instance / from_instance names are deprecated in favor of to_pose / from_pose.

Infer left/right symmetries from node names (#534)

Skeleton.infer_symmetries_by_name() suggests symmetric node pairs from names, so flip augmentation and QC work even for skeletons imported without symmetry metadata.

import sleap_io as sio

skel = sio.Skeleton(["nose", "eye_L", "eye_R", "ear_L", "ear_R"])
skel.infer_symmetries_by_name()                 # [(1, 2), (3, 4)]

# Non-mutating: review, then apply explicitly
skel.add_symmetries(skel.infer_symmetries_by_name())

# Name-only helper (no Skeleton needed); custom tokens supported
sio.infer_symmetry_pairs_by_name(["front_left_paw", "front_right_paw", "tail"])  # [(0, 1)]

Recognizes left/right and l/r tokens (configurable via token_pairs) as whole name segments (eye_L/eye_R, left_paw/right_paw, L1/R1), pairing only unambiguous 1:1 stems. It is opt-in and non-mutating — you apply the suggestions with add_symmetries(). Contributed by @tom21100227.

`sio.download()` and `sio download` CLI (#528)

v0.8.0 added remote reading; v0.9.0 adds a primitive to fetch a remote file straight to disk without loading it — useful for large videos or formats you load separately.

import sleap_io as sio

sio.download("https://example.com/labels.slp")       # -> ./labels.slp
sio.download("s3://bucket/run/video.mp4", "data/")    # -> data/video.mp4
sio.download(url, headers={"Authorization": "Bearer <token>"})

# Fetch-then-load for formats not loadable directly over a URL
labels = sio.load_nwb(sio.download("https://example.com/labels.nwb"))

sio download https://example.com/labels.slp
sio download s3://bucket/run/video.mp4 data/
sio download https://example.com/a.slp out.slp -H 'Authorization: Bearer <token>' -f

Supports the same schemes as the loaders — http(s), cloud buckets (s3/gs/gcs/az/ abfs, needs the [cloud] extra), and Google Drive share links — with streaming to disk, atomic writes, and idempotent skip-if-exists (overwrite=True to force). No new dependencies.

Machine-readable CLI inspection: `--json` (#538)

sio show --json prints a structured document (path/name/size/format, a stats block, and full skeletons, videos, tracks, identities, event_types, events, and provenance sections); for a standalone video it prints a video-shaped payload. sio filenames --json prints a per-video filename + provenance listing (inspection mode only).

sio show labels.slp --json
sio show labels.slp --json --lf 0        # add per-instance point detail for one frame
sio show labels.slp --frames             # per-frame listing (works with or without --json)
sio filenames labels.slp --json

The default human-readable output is unchanged when --json is omitted.

Improvements

Reliable saving for very large projects (#517, #521, #523, #524)

Hardens .slp saving against HDF5's hard 64 KB per-attribute limit (issue #516), which could make large projects fail to save or silently lose metadata:

#517 — provenance is written to a dedicated /provenance_json dataset instead of the metadata/json attribute. read_provenance falls back to the legacy attribute, so old files still load unchanged.
#521 — Labels.merge() bounds merge_history (see Breaking Changes).
#523 — oversized per-video source_video metadata spills from its attribute into a dataset (with a warning) when it exceeds 64 KB.
#524 — a new opt-in save_slp(..., preserve_unknown=True) carries unrecognized top-level HDF5 datasets/groups across a load/save cycle for forward compatibility with newer sleap-io versions.

labels = sio.load_slp("from_newer_version.slp")
sio.save_slp(labels, "out.slp", preserve_unknown=True)

These are read-backward-compatible layout changes — no SLP format-version bump.

Faster `Labels.merge()` (#537)

Appending merges no longer invalidate and rebuild the frame-lookup index on every frame; the warm index is now updated in place, turning large merges from O(N²) into O(N) (a reported ~218s → ~0.75s for a 9,000-frame merge into a ~95k-frame project). Merged output is byte-identical. Contributed by @yixi0527.

Clearer embed errors (#530)

Embedding an out-of-range frame now raises an IndexError that names the offending video (index, filename, and frame count) instead of reporting only the frame index — actionable in multi-video projects. Contributed by @gitttt-1234.

Fixes

#532 — NWB export is compatible with pynwb 4.0, which now requires num_samples on rate-based external ImageSeries. The writer sets it from the video frame count and passes it only when the installed pynwb accepts it, so pynwb < 4 is still supported.

Documentation

New docs/model/events.md (frame-spanning events) and docs/model/embedding.md (re-ID identities and embeddings).
docs/formats/slp.md — format-version history extended through 2.6.
docs/model/{poses,centroids,boxes,rois,segmentation}.md — modality interconversion verbs and examples.
docs/cli.md — download, show --json / --frames, filenames --json, and merge --identity.
docs/model/3d.md, docs/model/labels.md, docs/merging.md, docs/remote.md, docs/examples.md updated.

Known Issues

Events persist only in .slp. Converting to NWB, Label Studio, JABS, or a DataFrame drops events and event types; sio convert warns when this happens.

Changelog

Reflects net user-facing changes; intra-cycle refactors that net to no change are folded into their final form.

#513: feat(model): global re-ID Identity with cross-file name matching (@talmo)
#514: feat(model,io,cli): persist per-instance Identity, with merge and CLI support (@talmo)
#515: feat(model,io): re-ID Embedding data model + SLP persistence (@talmo)
#517: fix(io): store provenance in a /provenance_json dataset to dodge HDF5's 64 KB attribute limit (@talmo)
#521: fix(model): bound merge_history growth in Labels.merge() (default cap 1000) (@talmo)
#523: fix(io): spill oversized source_video metadata to a dataset (@talmo)
#524: feat(io): opt-in preserve_unknown carry-over of unknown HDF5 datasets on save (@talmo)
#527: feat(io,model): persist identity + re-ID embeddings across all detection modalities (@talmo)
#528: feat(io): add sio.download() + sio download CLI for fetching remote files (@talmo)
#530: fix(io): name the video in out-of-range embed errors (@gitttt-1234)
#531: feat(model): unified interconversion between detection modalities (pose ⇄ centroid ⇄ bbox ⇄ mask/ROI) (@talmo)
#532: fix(io): set ImageSeries num_samples for pynwb 4 compatibility (@talmo)
#534: feat(skeleton): infer left/right symmetries from node names (@tom21100227)
#535: refactor(model,io): finalize the re-ID identity/embedding data model (SLP format 2.5) (@talmo)
#536: feat(io): default save_embedding_vectors to False (off by default, like embed) (@talmo)
#537: perf(model): keep the frame index warm during merge for O(N) appending merges (@yixi0527)
#538: feat(cli): add --json output to show and filenames for machine-readable inspection (@talmo)
#540: feat: frame-spanning Event annotations (data model + Labels + SLP format 2.6) (@talmo)

Full Changelog: v0.8.0...v0.9.0

v0.8.0 ¶

sleap-io v0.8.0 Release Notes

Summary

sleap-io v0.8.0 is a feature release centered on getting data in and out from more places: load .slp and video directly from URLs and cloud buckets (including Google Drive share links), import full DeepLabCut projects (skeleton edges, source videos, train/test splits), and read COCO instance-segmentation datasets as SegmentationMask annotations with identity tracks. It rounds out the v0.7.0 segmentation architecture with mask provenance (to_user() / from_predicted), adds virtual on-read video cropping with a new apply-crops CLI, and ships a batch of rendering upgrades (mask overlays, track-identity coloring, grayscale and instance-less-frame fixes).

⚠️ Breaking change: Labels.merge() / Labels.match() now default to identity-based track matching instead of name-based. Pipelines that relied on collapsing same-named tracks across files must now pass track="name" explicitly. See Breaking Changes below before upgrading.

Highlights:

⚠️ Merge track matching now defaults to identity, not name (#449) — Labels.merge(other) no longer collapses tracks just because they share a name. Pass track="name" (or --track name on the CLI) to restore the old behavior. Audit downstream .merge() call sites before bumping.
Remote loading — sio.load_file / sio.load_video (and the CLI read commands) read .slp and video over http(s), s3, gs/gcs, az/abfs, and Google Drive share links via fsspec/PyAV (#439, #442, #441, #445, #501).
DeepLabCut project import — sio.load_dlc_project() reads a DLC config.yaml/project directory into Labels (skeleton edges, source videos), sio.load_dlc_splits() returns the train/test split as a LabelsSet, and sio convert imports projects from the CLI (#424, #450, #496).
COCO instance segmentation — the COCO reader imports polygon and (compressed or uncompressed) RLE segmentation as SegmentationMask annotations and can map categories to identity tracks via category_as_track=True, also exposed in sio convert (#479, #487, #496).
Segmentation mask provenance — PredictedSegmentationMask.to_user(), a from_predicted link on user masks, link-first mask merge, and .slp persistence of the provenance link (preserved across merge) (#472, #475, #478, #491).
Virtual on-read cropping — Video.crop() / CropVideoBackend present a cropped view without re-encoding, round-trip through .slp, and bake to real files via the new sio apply-crops command (#460).
Rendering upgrades — auto-drawn SegmentationMask overlays, track-identity coloring for masks/ROIs/bboxes, a grayscale-render crash fix, instance-less-frame rendering, and centroid scoping/scaling fixes (#461, #470, #466, #468, #494).
Smaller .slp files — mask RLE and ROI WKB datasets are now gzip-compressed on write (#463, #465).
Cross-platform CLI — sio no longer crashes on the default Windows (cp1252) console (#486), and sio show surfaces SegmentationMask/ROI counts (#500).
scipy<1.18 pin in the [mat]/[all] extras to keep the LEAP .mat reader working (#485).

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

Breaking Changes

⚠️ Merge track matching now defaults to `identity`, not `name` (#449, #448)

Change: TrackMatcher's default method flipped from NAME to IDENTITY. This changes the default behavior of Labels.merge(), Labels.match(), and the sio merge CLI (--track default is now identity).

Previously, merging two label files that each contained a Track(name="track_0") would treat them as the same track (collapsing by name). Now, two tracks are only matched if they are the same Python object (identity) — so the merge produces two distinct tracks unless you opt back into name matching.

import sleap_io as sio

base = sio.load_file("session_a.slp")
other = sio.load_file("session_b.slp")

# NEW default (0.8.0): identity matching — same-named tracks stay separate
base.merge(other)                    # -> distinct tracks preserved

# RESTORE 0.7.x behavior: collapse tracks that share a name
base.merge(other, track="name")

# CLI: restore name-based collapsing
sio merge a.slp b.slp -o merged.slp --track name

Migration for downstream (sleap, sleap-nn): Human-in-the-loop and re-tracking flows that relied on name-based collapse will now produce duplicate tracks. Audit every .merge() / .match() call site and add track="name" where name-collapse is intended. sio unsplit already pins track="name" internally, so its behavior is unchanged. A spatial-divergence warning (#448) now fires when name-collision merges combine tracks at incompatible locations. Documented in docs/merging.md and docs/cli.md.

COCO polygon segmentation now defaults to rasterized masks, not vector ROIs (#479)

Change: When the COCO reader encounters polygon segmentation, it now produces a single UserSegmentationMask per annotation by default (segmentation_format="mask"). In v0.7.x, polygons were read as vector UserROI objects (one per ring).

# NEW default (0.8.0): polygon -> rasterized SegmentationMask
labels = sio.load_coco("instances.json")            # masks, no ROIs

# RESTORE 0.7.x vector-ROI behavior
labels = sio.load_coco("instances.json", segmentation_format="roi")

sio convert exposes both options too: --coco-segmentation {mask,roi} selects the representation and --coco-category-as-track maps categories to identity tracks (#496).

Structural skeleton deduplication in `update` / `append` / `extend` (#447)

Change: When you add instances whose skeletons are structurally identical (same ordered node names + edges + symmetries) but are distinct Python objects, Labels now canonicalizes them to a single Skeleton and rebinds the instances to it. Point data is preserved exactly (verified point-safe); only len(labels.skeletons) changes.

s1 = sio.Skeleton(["head", "thorax", "abdomen"])
s2 = sio.Skeleton(["head", "thorax", "abdomen"])   # structurally equal, distinct object
labels = sio.Labels(labeled_frames=[lf_using_s1, lf_using_s2])
len(labels.skeletons)   # 0.8.0: 1   (0.7.x: 2)

Skeletons with a different node order are still kept distinct. Code that counted len(labels.skeletons) or held references to now-canonicalized duplicate skeletons may observe a difference. Documented in docs/model/labels.md.

Re-saving `.slp` now trusts recorded video metadata by default (#483)

Change: save_slp gained prefer_metadata=True (the default). When a video carries recorded backend_metadata (shape / grayscale / fps), the serializer now writes those recorded values instead of re-decoding the live media. This avoids opening (and OOM-ing on) large or remote media just to re-stamp metadata on save.

Impact: Re-saving a project no longer refreshes fps/shape from the live media file. To force a refresh from the live backend, pass prefer_metadata=False:

labels.save("out.slp", prefer_metadata=False)   # re-read shape/fps/grayscale from media

Preflight-hardened: earlier 0.8.0 builds could serialize a stale/inconsistent shape after a resolution-changing relink (replace_filename) or a post-load grayscale flip. Both are fixed: a relink now invalidates the stale recorded shape/grayscale/fps (#490), and metadata serialization reconciles the channel count with the grayscale flag (#495). The common open → edit → save path remains byte-identical to v0.7.x.

`scipy<1.18` upper bound added to `[mat]` and `[all]` extras (#485, #484)

Change: scipy 1.18.0 broke pymatreader mat-struct parsing (used by the LEAP .mat reader), so the mat and all extras now pin scipy<1.18. sleap-io itself does not import scipy; this only affects environments installing those extras.

Migration for downstream: packagers requesting sleap-io[all] alongside a dependency that requires scipy>=1.18 will hit a resolver conflict until pymatreader is fixed upstream.

New Features

Remote loading: URLs, cloud buckets, and Google Drive (#439, #442, #441, #445)

Load .slp files and videos directly from remote locations — no manual download step. Supported schemes: http, https, s3, gs/gcs, az/abfs, plus Google Drive share links.

import sleap_io as sio

# .slp over http(s) / cloud
labels = sio.load_file("https://example.com/predictions.slp")
labels = sio.load_file("s3://my-bucket/predictions.slp")

# Pass auth headers for protected .slp endpoints
labels = sio.load_file(
    "https://example.com/predictions.slp",
    headers={"Authorization": "Bearer <token>"},
)

# Remote video (streamed via PyAV)
video = sio.load_video("https://example.com/recording.mp4")

# Google Drive share link
labels = sio.load_file("https://drive.google.com/file/d/<id>/view")

# Clear the on-disk fsspec cache
sio.clear_remote_cache()

Hardening in #445 adds an auth probe, deterministic handle lifecycle/close, Google Drive capacity handling, and an fsspec cache. See docs/remote.md for the full guide.

The CLI's read commands (sio show, sio convert, sio render, sio export, …) also accept these URLs directly (#501):

sio show "https://example.com/predictions.slp"
sio convert "s3://my-bucket/predictions.slp" -o local.slp

Scope notes:

headers= is forwarded for .slp over HDF5. Remote media video cannot be authenticated via headers= (the av/imageio backend has no header plumbing); passing headers=/stream_mode= for a remote .mp4/.avi now raises a clear error instead of silently dropping them (#498) — use a pre-signed URL for protected media.

Only slp and video are loadable over a URL today; other formats (coco, dlc, csv, …) raise NotImplementedError over a URL — download locally first.

DeepLabCut project import (#424, #450)

Import a full DeepLabCut project — skeleton edges, source videos, and bodyparts — from its config.yaml or project directory, plus the train/test split as a LabelsSet.

import sleap_io as sio

# Whole project (config.yaml or the project directory)
labels = sio.load_dlc_project("my_project/config.yaml")
labels = sio.load_file("my_project/config.yaml")          # auto-detected

# Train/test splits as a LabelsSet
splits = sio.load_dlc_splits("my_project/config.yaml")
train, test = splits["train"], splits["test"]

sio convert now recognizes a DLC project directory / config.yaml and imports it (#496):

sio convert my_project/config.yaml -o project.slp
sio convert my_project/ -o project.slp --from dlc_project

Scope notes:

load_dlc_splits requires the labeled images to be present on disk; if they are absent it now emits a warning and returns empty splits (#492) rather than failing silently.

COCO instance-segmentation reader (#479)

The COCO reader now imports object detection/segmentation datasets: polygon and RLE segmentation become SegmentationMask annotations, and categories can be mapped to identity tracks.

import sleap_io as sio

# Polygon/RLE segmentation -> SegmentationMask (default)
labels = sio.load_coco("instances.json")

# Map COCO categories to identity tracks
labels = sio.load_coco("instances.json", category_as_track=True)

# Keep the v0.7.x vector-ROI representation instead of rasterizing
labels = sio.load_coco("instances.json", segmentation_format="roi")

Both compressed (LEB128 string-counts) and uncompressed (list-of-int) RLE, as well as polygon segmentation, are decoded (#487). sio convert exposes the identity/segmentation options too (#496):

sio convert instances.json --from coco -o out.slp \
    --coco-category-as-track --coco-segmentation roi

Segmentation mask provenance: `to_user()` and `from_predicted` (#472, #475, #478)

Predicted masks can now be "promoted" to user masks while remembering where they came from, mirroring the long-standing Instance / PredictedInstance relationship.

import numpy as np
import sleap_io as sio

pred = sio.PredictedSegmentationMask.from_numpy(arr, score=0.95)
user = pred.to_user()                 # UserSegmentationMask
user.from_predicted is pred           # True — provenance link preserved

# Link-first mask merge keeps user/predicted pairs together; the
# from_predicted link is persisted to and restored from .slp (#475).

predict → correct → consolidate workflows retain the predicted source through save/load, including when consolidation goes through Labels.merge(..., frame="auto") — the provenance link is remapped to the surviving copy on merge so it is no longer dropped (#491).

Virtual on-read video cropping + `apply-crops` CLI (#460)

Video.crop() returns a virtual cropped view backed by CropVideoBackend — frames are cropped on read, with no re-encoding. The crop round-trips through .slp (in a /video_crops table) and can later be baked into real cropped video files.

import sleap_io as sio

video = sio.load_video("recording.mp4")
cropped = video.crop(crop=(x1, y1, x2, y2))     # virtual view
cropped = video.crop(center=(cx, cy), size=(256, 256))

# Bake virtual crops into physical files and rewire references
labels.apply_crops("baked.slp")

# Materialize every virtual crop in an SLP into baked video files
sio apply-crops cropped.slp -o baked.slp --quality high
sio apply-crops cropped.slp -o baked.slp --dry-run

Note: baked crop dimensions are rounded up to a codec-friendly multiple of 16 (bottom/right padding), so e.g. a 100×100 crop bakes to 112×112. Coordinates are unchanged.

Rendering upgrades (#461, #470, #466, #468)

#461 — SegmentationMask overlays are now auto-drawn when rendering frames that contain masks, and a grayscale-render crash is fixed.
#470 — segmentation masks, ROIs, and bounding boxes are colored by track identity (matching pose coloring).
#466 — frames with no instances are rendered via the Labels path even when no skeletons exist.
#468 — render_image centroids are scoped to the rendered video (no cross-video bleed).

import sleap_io as sio

labels = sio.load_file("instances.coco.json", category_as_track=True)
# Masks auto-overlaid, colored by track identity:
sio.render_video(labels, "seg.mp4", color_by="track")

Overlay note: render_image(..., overlay=...) accepts either a single SegmentationMask/ROI/BoundingBox/LabelImage/ndarray or a list of them (#505).

Improvements

Smaller `.slp` files (#463, #465)

#465 — the roi_wkb dataset is gzip-compressed on write.
#463 — the mask_rle dataset is gzip-compressed on write.

These are write-time changes; older readers continue to read the compressed datasets transparently. ROI/mask-heavy projects (e.g. instance-segmentation imports) shrink substantially on disk.

Metadata-preferring `.slp` serialization (#483)

save_slp(prefer_metadata=True) (the new default) avoids decoding large/remote media on every save by trusting recorded backend_metadata. See Breaking Changes for the behavior change and prefer_metadata=False opt-out.

Path handling and helpful errors

#457 — transform_labels / transform_video accept str paths (not only Path).
#446 — a missing SLP metadata JSON attribute now raises a clear, actionable error instead of an opaque KeyError.
#438 — SkeletonEncoder now preserves edge-less (isolated) nodes in standalone skeleton JSON output (previously dropped). (Minor residual: standalone skeleton-JSON does not preserve isolated-node ordering; the .slp path is unaffected — see Known Issues.)

Embedded-subset shape resolution (#476)

#473/#476 — _get_effective_shape walks the source_video chain nearest-first, so embedded subsets resolve to their source video's shape and match correctly.

Mask provenance persistence and `to_user()` (#472, #475, #478)

See New Features — the from_predicted provenance link is persisted in .slp (format 2.4) and mask merge is link-first.

CLI reaches the new readers (#496, #500, #501)

#501 — sio show/convert/render/export/split/filenames accept remote URLs as input (http(s), s3, gs/gcs, az/abfs), matching the Python API.
#496 — sio convert imports DLC projects (config.yaml / project directory, --from dlc_project) and forwards COCO options (--coco-category-as-track, --coco-segmentation {mask,roi}).
#500 — sio show now reports SegmentationMask and ROI counts in the header and per-frame listing, so segmentation-only files no longer look empty.

Fixes

#467 (#466) — render instance-less frames via the Labels path when no skeletons exist.
#468 — scope render_image centroids to the rendered video.
#470 — color segmentation masks (and ROI/bbox) by track identity.
#461 — auto-draw SegmentationMask overlays and fix grayscale render crash.
#476 (#473) — resolve effective shape through the source chain so embedded subsets match.
#446 (#429) — raise a helpful error when an SLP metadata JSON attribute is missing.
#447 (#427) — structural skeleton dedup in Labels.update/append/extend (see Breaking Changes; point-safe).
#438 — preserve edge-less nodes in SkeletonEncoder JSON output.
#485 (#484) — pin scipy<1.18 in the mat/all extras to keep the LEAP .mat reader working.
#507 — sio reencode no longer deadlocks on a 0-frame input; it now defaults to .mp4 output and adds a --replace flag.
#480 — Analysis CSV / to_dataframe (all_frames) export now spans the full video length instead of stopping at the last labeled frame.
#481 — COCO reader: distinct images sharing a file_name no longer collide (each becomes its own frame), and a scored detection annotation reads as a PredictedSegmentationMask / PredictedROI.

Preflight fixes (release-hardening pass)

A pre-release adversarial audit surfaced and fixed the following before tagging:

#489 — Labels.merge() no longer silently misaligns per-node points when matched skeletons have a different node order (the structure matcher now reorders points by node name). Data-correctness fix.
#487 — the COCO reader decodes compressed (LEB128 string-counts) RLE masks instead of crashing with a TypeError.
#490 — a resolution-changing replace_filename relink invalidates stale recorded shape/grayscale/fps so save_slp(prefer_metadata=True) no longer writes a wrong shape.
#495 — metadata serialization reconciles the channel count with the grayscale flag, so a post-load grayscale flip survives a metadata save.
#491 — Labels.merge(..., frame="auto") remaps mask/instance from_predicted provenance to the surviving copy, so it is no longer dropped on save.
#486 — sio fix and sio render --help no longer crash on the default Windows (cp1252) console (stdout/stderr are reconfigured to UTF-8 at import).
#488 — sio show <dlc_project> / load_file(<dlc_project>, open_videos=…) no longer crash on an unexpected open_videos/lazy kwarg.
#493 — a degenerate COCO polygon with a valid bbox falls back to the bbox instead of being dropped.
#494 — centroid markers scale linearly with the render scale (no longer double-scaled / vanishing under preview/draft presets).
#492 — load_dlc_splits warns (instead of silently returning empty splits) when the labeled images are missing on disk.
#498 — a remote media video given unusable headers=/stream_mode= raises a clear error instead of silently dropping them.
#503 — save_coco writes track identity (attributes.object_id) for mask/ROI/bbox annotations, so a COCO round-trip preserves tracks (previously keypoint-only).
#502 — the standalone overlay helpers (draw_masks/draw_rois/draw_bboxes/draw_label_image) accept grayscale (H, W) / (H, W, 1) input instead of crashing.
#505 — render_image(..., overlay=<single object>) accepts a single SegmentationMask/ROI/BoundingBox (previously silently ignored unless wrapped in a list).
#504 — tracking_score and _instance_idx are preserved through mask/bbox/ROI conversions and resampled().
#506 — CLI/API polish: draw_centroids is exported at sio.*; the export command appears in sio --help groups; python -m sleap_io.io.cli works; and several doc corrections (palette default, video-crop note, removal of a nonexistent DeepLabCut .h5 reader claim).

Post-audit follow-ups (deferred low-severity findings)

#509 — LabeledFrame.is_user_labeled now counts user ROIs (a UserROI-only frame is correctly treated as user-labeled).
#510 — sio transform --crop/--scale/--rotate/--pad raise a clear error for a non-integer idx: prefix instead of a raw traceback.
#511 — sio convert --from dlc pointed at a DLC project errors with a pointer to --from dlc_project instead of misrouting to the single-CSV reader.
#512 — standalone skeleton JSON preserves isolated (edge-less) node order on round-trip (the .slp path was already correct).

Documentation

#458 — split the formats reference into per-format leaf pages, repoint remote cross-links, add a Guides landing page.
#454 — split spatial-annotation docs into Centroids / Boxes / ROIs / Segmentation.
#455 — content and format-reference corrections for 0.8.0.
#456 — add llms.txt, surface motion trails, rendering/CLI/nav polish.
#453 — repair non-runnable examples and guard pycon blocks.
#444 — new Remote loading guide (URLs, cloud, Google Drive, video).
#499 — add a "Breaking change in 0.8.0" callout to the COCO format reference (polygon → mask default), remove the documented-but-nonexistent sio render --crop auto / --crop-padding, and repair nine dead cross-link anchors.
#497 — execute runnable cropping.md examples (converted to pycon) so feature-doc API drift is caught by CI.

Known Issues

The pre-release audit's headline correctness, CLI, and Windows-console findings were all fixed before tagging (see Preflight fixes). The following minor items ship as-is and are tracked for follow-up:

sio show prints × (U+00D7) in video dimensions, which can render as mojibake when piped to a cp1252 consumer (the interactive console itself is now UTF-8).
Some cross-modality conversions downcast predicted → user — to_polygon/to_roi return user-variant outputs regardless of source (documented design choice).

A fuller list of low/info observations from the audit is tracked internally for post-release cleanup.

Changelog

#438: fix(io): preserve edge-less nodes in SkeletonEncoder JSON output (@talmo)
#439: feat(io): Load .slp from URLs and cloud buckets via fsspec (PR 1/3) (@talmo)
#441: feat(io): Google Drive share-link loading (PR 3/3) (@talmo)
#442: feat(video): Remote video loading via PyAV (supersedes #440) (@talmo)
#444: docs: Add Remote loading guide (URLs, cloud, Google Drive, video) (@talmo)
#445: fix(io): harden remote URL loading (auth, handle lifecycle, Drive caps, fsspec cache) (@talmo)
#446: fix(io): raise helpful error when SLP metadata JSON attribute is missing (#429) (@talmo)
#447: fix(model): structural skeleton dedup in Labels.update/append/extend (#427) (@talmo)
#448: feat(model): warn on spatially-divergent track-name merge collisions (#425) (@talmo)
#449: feat(model)!: default Labels.merge track matching to identity, not name (#425) (@talmo)
#450: feat(io): import DLC project metadata - edges, source videos, splits (#424) (@talmo)
#452: chore(release): Bump version to 0.8.0 (@talmo)
#453: fix(docs): repair non-runnable examples and guard pycon blocks (@talmo)
#454: docs: split spatial annotations into Centroids/Boxes/ROIs/Segmentation (@talmo)
#455: docs: content & format-reference corrections for 0.8.0 (@talmo)
#456: docs: add llms.txt, surface motion trails, and rendering/cli/nav polish (@talmo)
#457: fix(transform): accept str paths in transform_labels / transform_video (@talmo)
#458: docs: split formats into leaf pages, repoint remote cross-links, add Guides landing (@talmo)
#460: feat: virtual on-read video cropping (CropVideoBackend) (@talmo)
#461: fix(render): auto-draw SegmentationMask overlays and fix grayscale crash (@talmo)
#463: perf(slp): gzip-compress the mask_rle dataset when writing .slp (#464) (@talmo)
#465: perf(slp): gzip-compress the roi_wkb dataset when writing .slp (@talmo)
#466/#467: fix(render): render instance-less frames via Labels path when no skeletons exist (@talmo)
#468: fix(render): scope render_image centroids to the rendered video (@talmo)
#470: fix(render): color segmentation masks (and ROI/bbox) by track identity (@talmo)
#472: feat(model): add PredictedSegmentationMask.to_user() + from_predicted provenance (@talmo)
#475: feat(slp): persist UserSegmentationMask.from_predicted (#474 Gap 2) (@talmo)
#476: fix(model): resolve effective shape through source chain so embedded subsets match (#473) (@talmo)
#478: feat(model): mask unused_predictions + link-first mask merge (#474 Gap 1) (@talmo)
#479: feat(coco): read instance-segmentation datasets (polygon→SegmentationMask, identity tracks) (@talmo)
#480: fix(csv): span the full video in all_frames DataFrame/CSV export (@alicup29)
#481: fix(coco): robust duplicate-filename frames + scored segmentation → predicted (@talmo)
#483: feat(slp): prefer recorded video metadata over decoding when serializing (@tom21100227)
#485: fix(deps): pin scipy<1.18 to keep pymatreader (LEAP reader) working (@tom21100227)
#486: fix(cli): make stdout/stderr UTF-8 so sio fix and render --help work on cp1252 (@talmo)
#487: fix(coco): decode compressed (LEB128 string-counts) RLE masks (@talmo)
#488: fix(io): tolerate loader kwargs in load_dlc_project/load_dlc_splits (sio show DLC) (@talmo)
#489: fix(model): reorder points by node name in merge so reordered skeletons don't misalign (@talmo)
#490: fix(slp): invalidate stale shape/grayscale/fps on a resolution-changing relink (@talmo)
#491: fix(model): remap mask/instance from_predicted on merge so provenance survives save (@talmo)
#492: fix(io): warn when DLC splits resolve empty (labeled images missing) (@talmo)
#493: fix(coco): fall back to bbox when a degenerate polygon yields no geometry (@talmo)
#494: fix(render): stop double-scaling centroid markers by scale (@talmo)
#495: fix(slp): reconcile serialized channel count with the grayscale flag (@talmo)
#496: feat(cli): reach DLC-project import and COCO seg/identity options from convert (@talmo)
#497: test(docs): execute runnable cropping.md examples (@talmo)
#498: fix(video): reject (not silently drop) headers/auth kwargs for remote media video (@talmo)
#499: docs: COCO breaking-change callout, remove nonexistent render --crop docs, fix dead anchors (@talmo)
#500: feat(cli): surface SegmentationMask/ROI counts in sio show (@talmo)
#501: feat(cli): accept remote URLs as input to read commands (@talmo)
#502: fix(render): handle grayscale input in overlay helpers (@talmo)
#503: fix(coco): write track identity (attributes.object_id) for mask/ROI/bbox (@talmo)
#504: fix(model): preserve tracking_score and _instance_idx through conversions (@talmo)
#505: fix(render): accept a single object as render_image overlay= (@talmo)
#506: chore(cli/docs): export draw_centroids, group export cmd, main guard, doc fixes (@talmo)
#507: fix(cli): fix reencode deadlock at 0 frames; default .mp4 output + --replace (@talmo)
#508: ci: use sysmon coverage core on py3.13 and raise test timeout to 60m (@talmo)
#509: fix(model): count ROIs in LabeledFrame.is_user_labeled (@talmo)
#510: fix(cli): clean error for a non-integer transform index prefix (@talmo)
#511: fix(cli): clear error for --from dlc on a DLC project (@talmo)
#512: fix(io): preserve isolated-node order in standalone skeleton JSON (@talmo)

Full Changelog: v0.7.1...v0.8.0

v0.7.1 ¶

sleap-io v0.7.1 Release Notes

Summary

A focused patch release: one new feature (motion trail overlays for video rendering) and a series of fixes for data-preservation gaps in analysis HDF5 export, merge, and the DLC / COCO readers, plus a quality-of-life upgrade to Labels lookups so a foreign Video or a plain filename now resolves transparently to the canonical project video.

Highlights:

⚠️ Analysis HDF5 fix for untracked multi-animal projects — save_analysis_h5 was silently keeping only one instance per frame for untracked projects. Re-export any analysis files produced on v0.7.0 from an untracked multi-animal project.
Motion trail overlays — sio.render_image / sio.render_video (and sio render) now draw configurable per-animal motion trails over the last N frames.
Merge preserves is_negative — background (negative training) frame markers are no longer dropped by Labels.merge / LabeledFrame.merge.
Empty frames preserved — the DLC and COCO readers now retain images / rows with zero annotations as empty LabeledFrames (negative-frame friendly).
Foreign-Video lookups — Labels.find, __getitem__, numpy, extract, and the get_* family now accept a filename or a Video created outside the project; new public Labels.match_video() exposes the resolver.
CI: GitHub Actions bumped off the deprecated Node.js 20 toolchain.

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

New Features

Motion Trail Overlays (#434)

Trace each animal's recent trajectory over the last N frames behind the current frame. Trails are drawn beneath the pose and centroid layers and are off by default.

Python API:

import sleap_io as sio

labels = sio.load_file("predictions.slp")

# Render with default centroid trails
sio.render_video(
    labels,
    "trails.mp4",
    show_trails=True,
    trail_length=15,
)

# Or trail specific nodes with a custom appearance
sio.render_video(
    labels,
    "trails.mp4",
    show_trails=True,
    trail_node=["head", "thorax"],   # one trail per node
    trail_width=3.0,
    trail_alpha_fade=True,
    trail_color="#ff8800",
)

# Low-level: draw trails onto an arbitrary frame
img = sio.draw_trails(image, labels, frame_idx=120, trail_length=10)

CLI:

sio render predictions.slp --trails --trail-length 10
sio render predictions.slp --trails --trail-node head,thorax --trail-width 3
sio render predictions.slp --trails --no-trail-fade --trail-color "255,128,0"

Trails need temporal context — they're silently skipped when rendering a single LabeledFrame or a list of instances. For untracked data, trails key by instance index across frames; identity swaps will produce identity-jumping trails (use tracking for stable trails).

Bug Fixes

⚠️ Analysis HDF5 dropped instances for untracked multi-animal projects (#433)

Fixed: sio.save_analysis_h5 on an untracked multi-animal project allocated a single track slot, so every instance after the first in each frame silently overwrote slot 0. Only one animal's data survived per frame.

Root Cause: _get_occupancy_and_points sized the track axis as labels.tracks or [None] (length 1) and routed every untracked instance to that single slot.

Fix: The track axis is now sized to the largest number of instances in any frame (max_instances_per_frame), and slots are filled per frame in arbitrary order using the same logic as Labels.numpy. Synthetic track names track_0 … track_{n-1} are written in the file.

Re-export recommended. Any analysis HDF5 produced on v0.7.0 from an untracked multi-animal project should be re-exported with v0.7.1 to recover the missing instances. Files exported from fully-tracked projects are unaffected.

Behavioral notes:

Single-instance untracked projects now use track_names = ["track_0"] (previously [""]).
Tracked projects with a stray track=None instance now drop that instance (matching to_numpy) instead of overwriting slot 0.

Merge dropped the `is_negative` background-frame marker (#432, fixes #431)

Fixed: LabeledFrame.merge (every frame= strategy) and Labels.merge (non-colliding path) both ignored the is_negative flag, so background training frames lost their marker when merged.

Fix: The merged frame's is_negative is now resolved as
(self.is_negative or other.is_negative) and not has_user_pose — real user poses cancel the flag (and record a negative_flag_conflict in MergeResult.conflicts), but predicted instances do not. Documented in docs/merging.md under "Negative (background) frames".

Re-merge recommended if you ran v0.7.0 merges over labels containing negative frames.

DLC and COCO readers dropped empty / background frames (#418)

Fixed: COCO files with images that had no annotations, and DLC CSVs with rows of all-NaN keypoints, lost those entries entirely — the negative-/background-frame signal vanished from the in-memory Labels.

Fix: Both readers now create an empty LabeledFrame for every image / row, even when no instances exist.

Behavior change downstream: len(labels.labeled_frames) from these readers will increase by the number of unannotated images. Code that asserts a specific frame count against the v0.7.0 behavior should be updated. On-disk data is unaffected — re-loading recovers the frames.

Foreign-`Video` and filename lookups silently returned nothing (#436)

Fixed: Because Video uses identity comparison (@define(eq=False)), a Video created outside the project — or a plain filename — never matched anything in Labels.videos, so find, __getitem__, extract, numpy, and the get_* family silently returned [] or raised IndexError. The only workaround was to hand-walk labels.videos.

Fix: A new public method Labels.match_video(video_or_path, method="auto") canonicalizes any of Video, str, or Path to the matching project Video (or None if no match). The lookup APIs above now canonicalize their video argument through it automatically, and Labels[path] / Labels[path, frame_idx] are now valid:

labels = sio.load_file("predictions.slp")

# All of these work, even if `video` was constructed outside the project:
labels.find("video.mp4", frame_idx=10)
labels.find(sio.load_video("video.mp4"), frame_idx=10)
labels["video.mp4", 10]
labels.numpy(video="video.mp4")

# Or resolve explicitly
canonical = labels.match_video("video.mp4")

The default method="auto" runs a tiered cascade (definitive match → basename fallback). Other methods: "path", "basename", "content", "shape", "image_dedup", or any VideoMatcher instance. Ambiguous matches raise ValueError; bad argument types raise TypeError.

API Changes

New Symbols

Type	Name	Location	Description
Function	`draw_trails`	`sleap_io.rendering.overlays` (exported as `sio.draw_trails`)	Draw motion trails onto an image
Method	`Labels.match_video`	`sleap_io.model.labels`	Resolve a foreign `Video`, `str`, or `Path` to the canonical project `Video`

New Parameters

Function	New Keyword Arguments
`sio.render_image`, `sio.render_video`	`show_trails: bool = False`, `trail_length: int = 10`, `trail_node: str \| list[str] = "centroid"`, `trail_width: float = 2.0`, `trail_alpha_fade: bool = True`, `trail_alpha: float = 1.0`, `trail_color: ColorSpec \| None = None`
`sio render` (CLI)	`--trails`, `--trail-length`, `--trail-node`, `--trail-width`, `--trail-fade` / `--no-trail-fade`, `--trail-alpha`, `--trail-color`, plus `--progress` / `--no-progress`

Widened Signatures (backwards-compatible)

Function	Accepts
`Labels.find`, `Labels.numpy`, `Labels.extract`, `Labels.get_rois`, `Labels.get_masks`, `Labels.get_bboxes`, `Labels.get_centroids`, `Labels.get_label_images`	`video=` now accepts a foreign `Video`, `str`, or `Path` and resolves via `match_video`
`Labels.__getitem__`	Also accepts `str` / `Path` and `(path, frame_idx)` tuples

Modified Behavior

Symbol	Change
`Labels.merge`	Emits `ConflictResolution(conflict_type="negative_flag_conflict", ...)` when merging clears `is_negative` due to a user pose
`LabeledFrame.merge`	Mutates `is_negative` in place per the new resolution rule
`sio.load_coco`, `sio.load_dlc`	Preserve images / rows with no annotations as empty `LabeledFrame`s
`sio.save_analysis_h5`	Untracked projects size the track axis to max instances per frame; synthetic `track_0..N-1` names written

Documentation

docs/cli.md: New "Motion Trail Options" subtable under sio render and a trail example in the Quick Reference.
docs/rendering.md: "Motion trails" section and draw_trails API entry.
docs/model/labels.md: Querying section cross-references match_video and the widened lookup signatures.
docs/merging.md: "Negative (background) frames" section describes the is_negative merge rule.
docs/examples.md: "Resolving a video by path or foreign instance" walkthrough.

CI / Infrastructure

#435: Bumped actions/checkout (v4→v6), astral-sh/setup-uv (v6→v7), and codecov/codecov-action (v4→v6) across all 7 workflows. Clears GitHub's Node.js 20 deprecation warnings ahead of the June 2026 forced bump and September 2026 removal.

Changelog

#418: Preserve images with 0 instances as empty labeled frames in DLC and COCO readers (@lochhh)
#432: fix: Preserve is_negative when merging colliding and new frames (#431) (@talmo)
#433: fix: Preserve all instances in analysis HDF5 export for untracked multi-animal projects (@talmo)
#434: Add motion trail overlay to video rendering (@talmo)
#435: ci: Bump GitHub Actions off deprecated Node.js 20 (@talmo)
#436: Resolve foreign Video by path/content matching in Labels lookups (@talmo)

Full Changelog: v0.7.0...v0.7.1

v0.7.0 ¶

sleap-io v0.7.0 Release Notes

Summary

sleap-io v0.7.0 reframes the library around a single, unified annotation architecture: pose Instances are now just one of six first-class annotation types — alongside BoundingBox, LabelImage, SegmentationMask, ROI, and Centroid — all nested inside LabeledFrame. Every annotation type follows the same shape: an abstract base class with User* and Predicted* variants that carry confidence scores, a consistent tracking_score field on anything trackable, and O(1) per-frame access. Annotations no longer carry their own video/frame_idx; they derive context from their parent frame (the way Instance always has). Together these changes make sleap-io equally first-class for keypoint pose, detection, segmentation, and tracking workflows, with a backward-compatible SLP format that round-trips all of it.

Highlights:

Unified annotation architecture — abstract BoundingBox/LabelImage/SegmentationMask/ROI/Centroid bases with User*/Predicted* variants, all nested under LabeledFrame, with O(1) frame and track lookups
First-class instance segmentation — LabelImage type with streaming write, lazy read, multi-resolution metadata, batch constructors, and segmentation overlay rendering for Cellpose / StarDist / Mask R-CNN / SAM workflows
First-class detection — BoundingBox type with x1/y1/x2/y2 representation, full I/O across SLP/COCO/Ultralytics/GeoJSON/JABS, and rotated-box rendering
3D pose — Identity, Instance3D, and PredictedInstance3D for cross-session multi-camera workflows; round-trips with sleap-io.js and luc3d
New formats — Norpix .seq video, TrackMate CSV reader, h5wasm/sleap-io.js SLP, GeoJSON ROI I/O
Tracking-friendly — uniform tracking_score on every trackable type, Centroid for detection-only trackers
Performance — O(1) frame/track indices, chunked SLP v2.2 label-image storage (43× faster writes), zero-decompression label-image merge

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

Breaking Changes

v0.7.0 is a major release with intentional API breakage to unify the annotation model. Migrations are mechanical — the table below covers every break.

Annotation type construction is now via `User` / `Predicted` subclasses (#381)

BoundingBox, SegmentationMask, ROI, and LabelImage are now abstract base classes. Direct instantiation raises TypeError — instead, use the User* (ground-truth) or Predicted* (model-output, with score) variant:

# Before (0.6.x)
bbox = sio.BoundingBox(x_center=50, y_center=40, width=100, height=80)
mask = sio.SegmentationMask.from_numpy(arr)

# After (0.7.0)
bbox = sio.UserBoundingBox(x1=0, y1=0, x2=100, y2=80, category="mouse")
pred = sio.PredictedBoundingBox.from_xyxy(0, 0, 100, 80, score=0.92)
mask = sio.UserSegmentationMask.from_numpy(arr, category="cell")
pred_mask = sio.PredictedSegmentationMask.from_numpy(arr, score=0.95, score_map=heatmap)

`BoundingBox` is now `x1/y1/x2/y2` (#381)

The constructor signature changed from (x_center, y_center, width, height) to (x1, y1, x2, y2). The old fields remain as read-only computed properties.

Annotations no longer carry `video`/`frame_idx` (#411)

Centroid, BoundingBox, SegmentationMask, and LabelImage lose the video= and frame_idx= constructor kwargs entirely (ROI keeps video for static arena ROIs but loses frame_idx). Annotations now derive their spatial-temporal context from the LabeledFrame they live in, exactly the way Instance always has — eliminating an entire class of metadata-sync bugs.

# Before
c = sio.UserCentroid(x=10, y=20, video=video, frame_idx=5)
labels.add_centroid(c)

# After
lf = sio.LabeledFrame(video=video, frame_idx=5)
lf.append(sio.UserCentroid(x=10, y=20))                     # → lf.centroids
lf.append(sio.UserBoundingBox.from_xywh(0, 0, 50, 50))      # → lf.bboxes
lf.append(sio.PredictedSegmentationMask.from_numpy(arr, score=0.9))
labels = sio.Labels(labeled_frames=[lf])

# Static ROIs (no parent frame) still go on Labels directly
roi = sio.UserROI.from_bbox(0, 0, 640, 480, video=video, category="arena")
labels = sio.Labels(rois=[roi])

Labels.add_centroid(), add_bbox(), add_mask(), add_label_image(), add_roi() are removed in favor of LabeledFrame.append(). ROI.is_static is removed (a static ROI is now any ROI with no parent frame).

`Labels.centroids/.bboxes/.masks/.label_images/.rois` are now read-only properties (#402)

These are now flattened views derived from each LabeledFrame. Mutate via lf.append(ann) (frame-bound annotations) or Labels(rois=[...]) (static ROIs). Code that did labels.centroids.append(c) or labels.bboxes = [...] must be updated.

`Labels.numpy()` and `save_analysis_h5()` shape change (#368)

These now return arrays sized to the full video length instead of last_labeled_frame + 1. A 1100-frame video with the last prediction at frame 990 used to produce a 991-row array; it now produces 1100 rows with all-NaN trailing frames. Downstream code that sized arrays from arr.shape[0], sliced to last_labeled_frame, or assumed the array ended at the last prediction will need to either use len(video) directly or trim explicitly.

`ROI.centroid` renamed to `ROI.centroid_xy` (#395)

For consistency with BoundingBox.centroid_xy and Instance.centroid_xy. Update reads of roi.centroid.

`ROI.from_bbox()` and `ROI.from_xyxy()` deprecated (#381)

Use BoundingBox.from_xywh() / BoundingBox.from_xyxy() for new code. The old class methods still work but emit DeprecationWarning.

Other field/return-shape changes

Symbol	Change
`ROI.annotation_type`, `ROI.score`	Removed — use `category` and `Predicted*` variant scores instead
`SegmentationMask.annotation_type`, `SegmentationMask.score`	Removed — same migration
`get_rois()`, `get_masks()`	`annotation_type=` filter removed; filter by `category=`
`get_rois()`, `get_masks()`, `get_label_images()`	New `predicted=True/False` filter
`parse_label_file()` (ultralytics)	Returns `(instances, rois, bboxes)` 3-tuple instead of 2-tuple
`ROI(geometry=...)` validation	Sharpened: invalid geometries raise `TypeError` at construction (was deferred)
`_rasterize_geometry(Point, h, w)`	Raises `TypeError` (was returning empty mask silently)
`UserSegmentationMask.from_numpy(multi_class_arr)`	Raises `ValueError` pointing to `LabelImage.from_numpy` (was silently binarizing)
`Labels.get_frame()`, `get_track_annotations()`	Raise on lazy `Labels`; materialize first
`sio convert *.csv`	CSV files are now content-sniffed (TrackMate vs. DLC) instead of always-DLC

New Features

Unified annotation architecture (#381, #402, #411)

BoundingBox, SegmentationMask, ROI, LabelImage, Centroid, and Instance now share a single design:

Aspect	Pattern
Abstract base	`BoundingBox`, `SegmentationMask`, `ROI`, `LabelImage`, `Centroid` are `ABC`s
Variants	`User` (ground truth) and `Predicted` (model output, with `score`)
Score maps	`Predicted*` masks/label-images can carry an optional dense `score_map`
Tracking	All trackable types carry `track` and `tracking_score: float
Storage	Columnar HDF5 storage in SLP (format v2.0+); `User`/`Predicted` distinguished by an `is_predicted` column
Frame context	`video`/`frame_idx` derived from parent `LabeledFrame`

import sleap_io as sio

lf = sio.LabeledFrame(video=video, frame_idx=5)
lf.append(sio.UserBoundingBox.from_xywh(10, 10, 50, 50, category="mouse"))
lf.append(sio.PredictedSegmentationMask.from_numpy(arr, score=0.92))
lf.append(sio.PredictedCentroid(x=42, y=37, score=0.88, tracking_score=0.7))
lf.append(sio.UserInstance(...))
labels = sio.Labels(labeled_frames=[lf])

labels.get_bboxes(category="mouse")
labels.get_masks(predicted=True)         # only model predictions
labels.get_track_annotations(video, track)  # all annotation types, sorted by frame

O(1) frame and track indices (#403, #405)

Lazy-built dict indices replace O(n) linear scans across Labels. get_frame(video, frame_idx) returns a LabeledFrame instantly; get_track_annotations(video, track) returns every annotation of a track sorted by frame_idx. find(), get_centroids(), get_bboxes(), get_masks(), get_label_images(), and get_rois() automatically use the index when both video and frame_idx are provided. Rendering also benefits — per-frame centroid and track lookups now run in O(1).

lf = labels.get_frame(video, frame_idx=42)               # O(1)
anns = labels.get_track_annotations(video, track)         # O(1), sorted
labels.get_centroids(video=v, frame_idx=42)              # O(1) fast path

First-class `BoundingBox` (#373, #381)

BoundingBox, UserBoundingBox, and PredictedBoundingBox for detection and tracking workflows. Full I/O across SLP (format v1.7 → v2.0 columnar), COCO, Ultralytics, GeoJSON, and JABS. Rotated boxes, fill, and score-text rendering.

bbox = sio.UserBoundingBox.from_xywh(10, 20, 90, 60, category="mouse")
pred = sio.PredictedBoundingBox.from_xyxy(0, 0, 100, 80, score=0.92)
labels = sio.Labels(bboxes=[bbox, pred])
labels.get_bboxes(category="mouse")

First-class `LabelImage` for instance segmentation (#377, #389, #390, #392, #393, #395)

A dense integer-array annotation type where each pixel value encodes which object occupies that pixel — matching the standard output of Cellpose, StarDist, Mask R-CNN, and SAM. Includes track-centric query API, ↔ SegmentationMask conversion, lazy-loaded SLP storage, TIFF I/O with .meta.json sidecars, and COCO panoptic format support.

Batch construction:

# From (T, H, W) stack — Cellpose/StarDist style
label_images = sio.PredictedLabelImage.from_stack(
    masks, video=video, source="cellpose", create_tracks=True, score=1.0,
)

# From per-object binary masks — SAM/Mask R-CNN style
li = sio.PredictedLabelImage.from_binary_masks(
    sam_masks,                        # (N, H, W) bool
    label_ids=[5, 10],                # pin pixel values for cross-frame consistency
    tracks=[t1, t2],
    scores=[0.95, 0.87],              # per-object → Info.score
    score=0.9,                        # image-level
    video=video, frame_idx=0,
)

# Globally normalize label IDs across frames so each track has a stable pixel value
track_map = sio.normalize_label_ids(labels.label_images, by="track")

Streaming + lazy + merge for large datasets:

# Constant-memory streaming write — viable for 1000+ frame microscopy stacks
with sio.LabelImageWriter("output.slp", video=video) as writer:
    for frame_idx, mask in enumerate(cellpose_generator()):
        li = sio.PredictedLabelImage.from_numpy(mask, video=video, frame_idx=frame_idx)
        writer.add(li)

# Zero-decompression raw-chunk merge across SLP files
sio.merge_label_images(["chunk_0.slp", "chunk_1.slp"], "merged.slp")

SLP format v2.2 stores label images as a chunked (T, H, W) int32 dataset with write_direct_chunk (43× faster writes). Old format files (v1.8–v2.1) remain fully readable.

Bbox extraction:

bboxes = label_image.to_bboxes()           # single-pass O(H*W); ~9ms for 500 objects on 512x512
for bb in bboxes:
    print(bb.xyxy, bb.track, bb.category, bb.centroid_xy)

bb = mask.to_bbox()                        # SegmentationMask → BoundingBox

Multi-resolution dense annotations (#385)

SegmentationMask and LabelImage get scale and offset spatial metadata so dense annotations can express their coordinate relationship to the video frame (half-resolution segmentation, cropped masks at an offset, etc.). Predicted variants get independent score_map_scale/score_map_offset. Rendering, COCO export, and TIFF sidecar (v2) all auto-resample.

mask = sio.UserSegmentationMask.from_numpy(half_res_data, scale=(0.5, 0.5))
mask = sio.UserSegmentationMask.from_numpy(data, stride=2)              # convenience
mask = sio.UserSegmentationMask.from_numpy(crop_data, offset=(100.0, 50.0))
full_res = mask.resampled(target_height=480, target_width=640)

`Centroid` data model with uniform `tracking_score` (#397)

Centroid, UserCentroid, and PredictedCentroid for lightweight 2D/3D point detections — natively used by detection-only trackers (e.g., TrackMate spot workflows). Backfills a consistent tracking_score: float | None field onto every trackable type (BoundingBox, SegmentationMask, ROI, LabelImage.Info).

c = sio.UserCentroid(x=100.5, y=200.3, z=1.0, track=track, tracking_score=0.8)
inst = c.to_instance()                       # single-node Instance
c = sio.Centroid.from_instance(pose, method="center_of_mass")
xy = pose.centroid_xy                        # (x, y) tuple or None
labels.get_centroids(video=vid, frame_idx=0)

3D data structures (#382)

Identity (a cross-session animal identity, distinct from per-video Track), Instance3D, and PredictedInstance3D with per-keypoint confidence scores. InstanceGroup gains instance_3d and identity fields; Labels gains identities. SLP format v1.9 (only emitted when identities are present, fully backward compatible) lets 3D data round-trip between sleap-io, sleap-io.js, and luc3d without conversion.

mouse_a = sio.Identity(name="mouse_A", color="#e6194b")
inst_3d = sio.Instance3D(points=[[1, 2, 3], [4, 5, 6]], skeleton=skeleton)
group = sio.InstanceGroup(
    instance_by_camera={cam1: inst_2d_cam1, cam2: inst_2d_cam2},
    instance_3d=inst_3d,
    identity=mouse_a,
)
labels = sio.Labels(identities=[mouse_a], sessions=[session])

Segmentation overlay rendering (API + CLI) (#374)

Composite integer label images, SegmentationMask, ROI, and BoundingBox onto rendered frames — alongside pose predictions or standalone (no labels file required).

img = sio.render_image(image=frame, overlay=label_mask, overlay_alpha=0.4)
img = sio.render_image(lf, overlay=label_mask, overlay_outline=True)
sio.render_video(labels, "out.mp4", overlay=lambda idx: load_mask(idx))
sio.draw_label_image(image, labels, alpha=0.4, outline=True)

sio render predictions.slp --overlay masks.tif --overlay-alpha 0.4
sio render --images frames/ --overlay masks.tif -o output.mp4

ROI: multi-geometry, indexed lookup, GeoJSON I/O (#366, #367)

ROI gets from_multi_polygon(), explode(), persisted instance field (lazy-resolved on read), skia-based overlays for all Shapely geometry types, and O(1) (video, frame_idx) index. New sio.load_geojson() / sio.save_geojson() functions — interoperable with movement, Shapely, GeoPandas, QGIS, QuPath. No new dependencies.

roi = sio.ROI.from_multi_polygon([
    [(0, 0), (10, 0), (10, 10), (0, 10)],
    [(20, 20), (30, 20), (30, 30), (20, 30)],
])
individual = roi.explode()

sio.save_geojson(rois, "rois.geojson")
loaded = sio.load_geojson("rois.geojson")
labels = sio.load_file("rois.geojson")          # auto-detected
roi.__geo_interface__                            # GeoJSON Feature dict

New format support

Format	PR	Notes
Norpix `.seq` video	#380	StreamPix files; auto-FPS from per-frame timestamps; works with `sio show`/`sio reencode`
TrackMate CSV	#399, #412	`read_trackmate_csv()` imports spots as `PredictedCentroid`, auto-detects sibling `_edges.csv` and `.tif`; `sio convert` content-sniffs CSVs
sleap-io.js / h5wasm SLP	#378	Transparent read of flat-2D-array SLP files written by the JS port
GeoJSON ROIs	#367	See above

video = sio.load_video("recording.seq")
labels = sio.load_trackmate("experiment_spots.csv", video="experiment.tif")
labels = sio.load_slp("from_browser.slp")        # h5wasm-written → auto-converted

sio convert experiment_spots.csv -o experiment.slp     # auto-detected as trackmate
sio reencode recording.seq -o recording.mp4

Strategy-aware annotation merging (#408)

Labels.merge() strategies (auto, update_tracks, replace_predictions, keep_*) now correctly apply to all annotation modalities, not just instances. auto and update_tracks use spatial centroid-distance matching for centroids, bboxes, ROIs, masks, and label images.

labels_a.merge(labels_b, frame_strategy="auto")    # spatial matching for all modalities

`LabeledFrame.append()` dispatcher

Routes annotations to the correct list based on type — Instance, Centroid, BoundingBox, SegmentationMask, LabelImage, or ROI.

lf.append(sio.UserCentroid(x=10, y=20))
lf.append(sio.UserBoundingBox.from_xywh(0, 0, 50, 50))
lf.append(sio.PredictedSegmentationMask.from_numpy(arr, score=0.9))

Bug Fixes

Lazy-load correctness

#419 Labels.__del__ no longer forcibly closes the HDF5 file on GC, so sio.load_slp("x.slp")[0].label_images[0].data works (the file stays open via h5py refcount as long as a lazy LabelImage needs it). Explicit Labels.close() is unchanged. (@gitttt-1234)
#414 _write_labels_lazy no longer drops the video association on static ROIs during a lazy read → lazy write round-trip. (@talmo)
#407 Three index/lazy correctness bugs: _merge_annotations() no longer mutates source labels by reference, _add_annotation()/remove_predictions(clean=False) no longer leave stale indices, and get_frame/get_track_annotations raise on lazy labels instead of returning silently-wrong results. (@talmo)
#401 Labels.copy() no longer drops centroids on the lazy path; Labels.replace_videos() now updates video references on label_images. (@talmo)
#386 Labels.materialize() no longer loses label-image-to-instance associations across lazy round-trips (LabelImage.Info now carries _instance_idx). (@talmo)

Segmentation / TIFF I/O

#421 load_label_images no longer assumes multi-page TIFFs are time-stacks; auto-detects axis layout from OME-XML/ImageJ metadata, with pages_as='auto'|'time'|'classes' override. UserSegmentationMask.from_numpy raises ValueError on multi-class input instead of silently binarizing. (@gitttt-1234)
#422 TIFF "ambiguous multi-page" warning is now gated on whether a class-stack reading is even possible (every page has at most one positive value), so multi-valued integer-label time-series load silently. (@gitttt-1234)
#420 render_image(source=lf) works on segmentation-only LabeledFrames (no Instance required). (@gitttt-1234)
#387 LabelImage.from_numpy no longer auto-creates tracks (opt in via create_tracks=True); Video.exists() and Video.is_open handle directory-based ImageVideo; render_video works on labels with only spatial annotations. (@talmo)
#372 Labels._roi_index/_mask_index caches removed (returned stale results after in-place mutations); fixed AttributeError on MultiPolygon ROI Ultralytics export and added UserWarning when polygon holes are dropped. (@talmo)

Merge / clean

#408 Annotation merge now respects strategy (see "New Features"). (@talmo)
#405 Nested annotations have their .video/.track references correctly remapped on cross-Labels frame copy; clean() removes annotations whose tracks were pruned. (@talmo)
#400 merge_label_images() no longer crashes on ImageVideo-backed SLPs (filename normalized to a tuple). (@talmo)

Format-specific

#371 JABS-NWB conversion: writer auto-detects predictions format; edge-less skeletons no longer crash on save; JABS static objects (corners, food hopper, lixit) load as proper ROI MultiPoint/Point geometries with per-point scores preserved. (@talmo)
#369 Negative frames (frames marked as having no instances — important as background examples) are preserved through lazy round-trips and the dict codec. (@talmo)

Output shape

#368 Labels.numpy(), save_analysis_h5(), and the lazy numpy codec extend to len(video) instead of truncating at last_labeled_frame + 1. Breaking output shape change — see Breaking Changes. (@gitttt-1234)

Dependency

#379 Removed the skia-python<=138.0 upper pin (3.13 wheels are now upstream). (@talmo)

API Changes

New symbols

Type	Name	Purpose
Class	`UserBoundingBox`, `PredictedBoundingBox`	Detection annotations
Class	`UserSegmentationMask`, `PredictedSegmentationMask`	Raster mask annotations
Class	`UserROI`, `PredictedROI`	Vector geometry annotations
Class	`LabelImage`, `UserLabelImage`, `PredictedLabelImage`	Dense integer instance segmentation
Class	`Centroid`, `UserCentroid`, `PredictedCentroid`	Lightweight 2D/3D point annotations
Class	`Identity`, `Instance3D`, `PredictedInstance3D`	3D pose data structures
Class	`LabelImageWriter`	Constant-memory streaming SLP writer for label images
Class	`SeqVideo`	Norpix .seq video backend
Function	`sio.load_geojson()`, `sio.save_geojson()`	GeoJSON ROI I/O
Function	`sio.load_trackmate()`	TrackMate CSV reader
Function	`sio.merge_label_images()`	Zero-decompression label-image merge
Function	`sio.normalize_label_ids()`	LUT-based pixel-value remapping
Function	`sio.draw_label_image()`	Vectorized LUT label-image overlay
Method	`LabeledFrame.append()`	Type-dispatching annotation insertion
Method	`Labels.get_frame()`	O(1) frame lookup
Method	`Labels.get_track_annotations()`	O(1) track annotation lookup
Method	`LabelImage.from_stack()`, `.from_binary_masks()`, `.to_bboxes()`	Batch constructors + bbox extraction
Method	`SegmentationMask.to_bbox()`	Mask → BoundingBox
Property	`Instance.centroid_xy`, `BoundingBox.centroid_xy`, `ROI.centroid_xy`	Uniform `(x, y)` accessors
Property	`Labels.identities`	List of cross-session animal identities

New parameters

Function	New parameters
`LabeledFrame()`	`centroids`, `bboxes`, `masks`, `label_images`
`Labels()`	`identities`, `centroids` (already had bboxes/masks/rois/label_images)
`Predicted*` annotation constructors	`score`, optional `score_map` (masks/label-images)
`*` annotation constructors (`Centroid`, `BoundingBox`, `SegmentationMask`, `ROI`, `LabelImage.Info`)	`tracking_score: float
`SegmentationMask`, `LabelImage`	`scale`, `offset` (and `score_map_scale`, `score_map_offset` on Predicted variants)
`LabelImage.from_binary_masks()`	`label_ids`, `tracks`, `categories`, `names`, `scores`
`get_rois()`, `get_masks()`, `get_label_images()`	`predicted: bool
`load_label_images()`	`pages_as='auto'

Removed / renamed

Symbol	Status
`BoundingBox(x_center, y_center, width, height)`	Constructor changed to `x1/y1/x2/y2`; old fields kept as read-only properties
`BoundingBox(...)`, `SegmentationMask(...)`, `ROI(...)`, `LabelImage(...)`	Now abstract — use `User` / `Predicted`
`.video`, `.frame_idx` (annotation objects)	Removed; derived from parent `LabeledFrame` (`ROI.video` retained for static ROIs)
`Labels.add_centroid()`, `add_bbox()`, `add_mask()`, `add_label_image()`, `add_roi()`	Removed; use `lf.append(ann)`
`Labels.centroids/.bboxes/.masks/.label_images/.rois`	Now read-only flattened views
`ROI.is_static`	Removed — a static ROI is any ROI with no parent frame
`ROI.annotation_type`, `ROI.score`	Removed
`SegmentationMask.annotation_type`, `SegmentationMask.score`	Removed
`ROI.centroid`	Renamed to `ROI.centroid_xy`
`ROI.from_bbox()`, `ROI.from_xyxy()`	Deprecated; use `BoundingBox.from_xywh/from_xyxy`
`parse_label_file()` (ultralytics)	Returns 3-tuple `(instances, rois, bboxes)`
`Labels._roi_index`, `_mask_index`	Removed (caches were stale-prone)

SLP format versions

v0.7.0 introduces several format bumps. All bumps are emit-only: new sleap-io reads every old format unchanged, and old sleap-io reads new files when the new datasets are absent. Bumps only happen when the new feature is actually used.

Version	Introduced	What's new
1.6	#366	ROI `instance_idx` column
1.7	#373	First-class bounding boxes
1.8	#377	First-class label images
1.9	#382	Cross-session identities
2.0	#381	Columnar bbox storage; `is_predicted`/`score` columns
2.1	#385	Dense annotation `scale`/`offset`
2.2	#390	Chunked `(T, H, W)` int32 label-image dataset

Documentation

#362 Refreshed README — features section linking to docs, modernized uvx/uv install, new CLI/Python examples (@talmo)
#370 Corrected JABS acronym in formats docs (@gbeane)
#375 Replaced docs/model.md with six focused subpages (index, poses, labels, video, 3d, regions); 41 pycon blocks executed at build time as real REPL sessions (@talmo)
#409 Annotation architecture documentation: Centroid types, "Working with annotations in frames", "Fast lookups", merge examples; fixed several broken pycon examples (@talmo)
#415 v0.7.0 reference-correctness pass: broken labels.get(...) snippet, outdated render_video overlay docstring, analysis-h5 sleap_io_version stamp; brought docs/formats/slp.md back in sync with on-disk schema; warnings on every page where the numpy() shape change matters (@talmo)
#416 v0.7.0 feature coverage: sio render --overlay* flags, TrackMate convert, segmentation overlays, multi-resolution masks, live Identity/Instance3D examples, Centroid integration on Instance, Track-vs-Identity tip, TIFF→SLP recipe (@talmo)

Changelog

#362: docs: Refresh README to reflect current capabilities (@talmo)
#365: fix: Address ROI/SegmentationMask data model and I/O limitations (@talmo)
#366: feat: ROI instance serialization, multi-geometry, rendering, and indexed lookup (@talmo)
#367: feat: Add GeoJSON I/O for ROIs (@talmo)
#368: fix: Extend numpy output to full video length across all code paths (@gitttt-1234)
#369: fix: Preserve negative frames in lazy loading and dict codec (@talmo)
#370: docs: Correct JABS acronym (@gbeane)
#371: fix: JABS-NWB conversion bugs and static objects as ROIs (@talmo)
#372: fix: ROI lookup cache removal, Ultralytics MultiPolygon crash, and hole warning (@talmo)
#373: feat: First-class BoundingBox type with full I/O support (@talmo)
#374: feat: Segmentation overlay rendering (API + CLI) (@talmo)
#375: docs: Reorganize data model docs into focused subpages (@talmo)
#377: feat: First-class LabelImage type for instance segmentation (@talmo)
#378: feat: Support reading SLP files written by h5wasm (sleap-io.js) (@talmo)
#379: chore: Remove skia-python<=138.0 version pin (@talmo)
#380: feat: Add Norpix .seq video file format support (@talmo)
#381: feat: Predicted variants, abstract base classes, BoundingBox x1y1x2y2, columnar bbox storage (@talmo)
#382: feat: 3D data structure standardization with sleap-io.js and luc3d (@ericleonardis)
#385: feat: Dense annotation scale/offset metadata for multi-resolution support (@talmo)
#386: fix: Use _instance_idx for LabelImage.Info instance resolution in lazy mode (@talmo)
#387: fix: Label image, video, and rendering fixes for segmentation workflows (@talmo)
#389: feat: LabelImage.from_stack() and segmentation UX improvements (@talmo)
#390: feat: Streaming write, lazy read, and merge for label images (@talmo)
#392: feat: Add LabelImage.from_binary_masks() for per-object mask arrays (@talmo)
#393: feat: Add label_ids param and normalize_label_ids utility (@talmo)
#395: feat: Add bounding box extraction from LabelImage and SegmentationMask (@talmo)
#397: feat: Add Centroid data model with tracking_score backfill (@talmo)
#399: feat: Add TrackMate CSV reader (@talmo)
#400: fix: Handle ImageVideo list filenames in merge_label_images (@talmo)
#401: fix: Add missing centroids in lazy copy and label_images in replace_videos (@talmo)
#402: feat: Nest annotations in LabeledFrame (@talmo)
#403: feat: Add frame and track indices for O(1) lookups (@talmo)
#405: fix: Fix annotation handling in merge/clean + optimize rendering (@talmo)
#407: fix: Index staleness, lazy guards, and source corruption in merge (@talmo)
#408: fix: Make annotation merging strategy-aware (@talmo)
#409: docs: Update documentation for annotation architecture (PRs 401-408) (@talmo)
#411: fix: Remove video/frame_idx from annotation objects (@talmo)
#412: fix: Add TrackMate CLI convert support and format documentation (@talmo)
#414: fix(slp): preserve static ROI video association on lazy round-trip (@talmo)
#415: docs(v0.7.0): fix broken refs, schema drift, and silent shape change (@talmo)
#416: docs(v0.7.0): v0.7.0 feature coverage across cli, examples, and model (@talmo)
#419: fix: Preserve LabelImage lazy reads across Labels GC (@gitttt-1234)
#420: fix: Allow render_image(source=lf) for segmentation-only LabeledFrames (@gitttt-1234)
#421: fix: disambiguate TIFF page axes and guard SegmentationMask from silent binarization (@gitttt-1234)
#422: fix(io/tiff): suppress ambiguous-pages warning when pages are multi-valued (@gitttt-1234)

Full Changelog: v0.6.5...v0.7.0

v0.6.5 ¶

sleap-io v0.6.5 Release Notes

Summary

This release adds ROI and segmentation mask support as first-class annotation types, with full I/O across SLP, COCO, and Ultralytics formats, and fixes a numpy 2.x compatibility issue with suggestion frames.

ROI & Segmentation Masks (experimental): New ROI (vector/Shapely geometry) and SegmentationMask (raster/RLE) data model classes with SLP read/write support
COCO Detection & Segmentation I/O: Read and write bounding box, polygon, and RLE mask annotations in COCO format
Ultralytics Detection & Segmentation I/O: Auto-detect and read/write YOLO detection and segmentation label formats
NumPy 2.x Fix: Prevent TypeError when saving .slp files with numpy integer suggestion frame indices
Edge Case Fixes: Preserve ROIs/masks in lazy label copies; fix COCO bbox fallback for empty segmentation lists

Note: ROI and segmentation mask support is experimental and subject to change in future releases.

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

New Features

ROI and Segmentation Mask Data Model (#351) (experimental)

New first-class annotation types for regions of interest and segmentation masks with full SLP file I/O.

ROI represents vector geometry (bounding boxes, polygons, arbitrary shapes) backed by Shapely:

import sleap_io as sio

# Create ROIs
roi = sio.ROI.from_bbox(10, 20, 30, 40, video=video, category="mouse")
polygon = sio.ROI.from_polygon(coords, annotation_type=sio.AnnotationType.ARENA)

# Query ROIs
labels.get_rois(video=video, frame_idx=0)
labels.static_rois   # ROIs without frame index (apply to all frames)
labels.temporal_rois  # ROIs with specific frame indices

# Convert to mask
mask = roi.to_mask(height=480, width=640)

SegmentationMask represents raster masks with RLE compression:

# Create masks from numpy arrays
mask = sio.SegmentationMask.from_numpy(binary_array, video=video, frame_idx=5)

# Query masks
labels.get_masks(category="foreground")

# Convert between representations
roi = mask.to_polygon()

SLP Storage Format

ROIs and masks are stored in new optional HDF5 datasets, bumping the format to 1.5:

file.slp
├── /rois          # ROI metadata (annotation_type, video, frame_idx, track, score)
├── /roi_wkb       # WKB-encoded geometry bytes
├── /masks         # Mask metadata (height, width, annotation_type, video, frame_idx, track, score)
└── /mask_rle      # RLE-encoded mask bytes

Backward Compatibility: Old sleap-io versions can still open files with ROIs/masks (the new datasets are simply ignored). New sleap-io can open old files (no ROIs/masks loaded).

COCO Detection & Segmentation I/O (#357) (experimental)

Read and write bounding box, polygon, and RLE mask annotations in COCO format.

Reading:

# Detection-only COCO datasets (no keypoints required)
labels = sio.load_coco("instances.json")
print(labels.rois)   # Bounding boxes and polygons
print(labels.masks)  # RLE segmentation masks

Writing:

# Export ROIs as COCO bbox/polygon annotations
# Export masks as COCO RLE annotations (iscrowd=1)
sio.save_coco(labels, "output.json")

Detection-only annotations (no keypoints) are read as ROI objects
RLE segmentation annotations are read as SegmentationMask objects
COCO column-major RLE encoding is handled correctly for roundtrips
Category and score fields are preserved

Ultralytics Detection & Segmentation I/O (#358) (experimental)

Extends Ultralytics YOLO I/O to support detection and segmentation formats alongside the existing pose format.

Reading (auto-detected):

# Format is auto-detected from label file content
labels = sio.load_ultralytics("project/")
print(labels.rois)  # Bounding boxes or polygons from detection/segmentation labels

Writing:

# Export as detection format
sio.save_ultralytics(labels, "output/", task="detect")

# Export as segmentation format
sio.save_ultralytics(labels, "output/", task="segment")

# Export as pose format (default, unchanged)
sio.save_ultralytics(labels, "output/", task="pose")

Format auto-detection based on values per line:

Values	Format
5	Detection (class + bbox)
6	Detection with confidence
5 + 3k	Pose (class + bbox + keypoints)
Other even count ≥ 8	Segmentation (class + polygon)

Bug Fixes

NumPy 2.x Compatibility for Suggestion Frames (#350)

Fixed: Saving .slp files with suggestion frames generated from numpy operations raised TypeError: Object of type int64 is not JSON serializable.

Root Cause: In numpy 2.0, np.int64 is no longer a subclass of Python int. SuggestionFrame.frame_idx accepted numpy integers without coercion, which then failed during JSON serialization.

Fix: Added converter=int to SuggestionFrame.frame_idx, consistent with the existing pattern on LabeledFrame.frame_idx and FrameGroup.frame_idx. The conversion is lossless and has no effect on users already passing Python ints.

Lazy Copy Loses ROIs and Masks (#361)

Fixed: Labels.copy() on lazy-loaded labels silently dropped all ROI and segmentation mask annotations.

Root Cause: The lazy copy path constructed a new Labels object but omitted the rois and masks parameters, defaulting them to empty lists.

Fix: The lazy copy now deep-copies rois and masks alongside the other fields.

COCO Empty Segmentation List Fallback (#361)

Fixed: COCO annotations with "segmentation": [] (empty list) and a valid "bbox" failed to create a bbox ROI.

Root Cause: The condition checked segmentation is None but an empty list [] is not None, so the bbox fallback was skipped.

Fix: Changed to not segmentation which correctly handles both None and [].

API Changes

New Symbols

Type	Name	Location	Description
Class	`ROI`	`sleap_io.model.roi`	Vector geometry annotation (bbox, polygon, arbitrary shape)
Class	`SegmentationMask`	`sleap_io.model.mask`	Raster mask annotation with RLE compression
Enum	`AnnotationType`	`sleap_io.model.roi`	Annotation type classifier (DEFAULT, BOUNDING_BOX, SEGMENTATION, ARENA, ANCHOR)
Function	`detect_line_format()`	`sleap_io.io.ultralytics`	Auto-detect YOLO label file format
Function	`write_roi_label_file()`	`sleap_io.io.ultralytics`	Write detection/segmentation ROIs to YOLO format

New Parameters

Function	New Parameters
`Labels()`	`rois: list[ROI]`, `masks: list[SegmentationMask]` (default empty)
`write_labels()` (ultralytics)	`task: str = "pose"` — control output format

New Properties/Methods on `Labels`

Member	Description
`Labels.rois`	List of all ROI annotations
`Labels.masks`	List of all segmentation mask annotations
`Labels.static_rois`	ROIs without frame index (apply to all frames)
`Labels.temporal_rois`	ROIs with specific frame indices
`Labels.get_rois()`	Query ROIs by video, frame, category
`Labels.get_masks()`	Query masks by video, frame, category

Modified Behavior

Symbol	Change
`parse_label_file()` (ultralytics)	Now returns `tuple[list[Instance], list[ROI]]` (internal API)
`parse_coco_json()`	Accepts detection-only COCO datasets (no keypoints required)
SLP format_id	Bumped to 1.5 when ROIs or masks are present

New Dependency

Package	Version	Purpose
`shapely`	`>=2.0`	Geometry operations for ROI (WKB serialization, polygon operations)

Documentation

SLP Format Spec (#356): New sections documenting ROI and segmentation mask HDF5 datasets, format version 1.5, WKB geometry storage, and RLE mask encoding
Format Capabilities Table (#356): Updated with ROIs/Masks column showing support across SLP, COCO, and Ultralytics formats

Changelog

#350: fix: Add converter=int to SuggestionFrame.frame_idx for numpy 2.x compatibility (@alicup29)
#351: feat: Add ROI and SegmentationMask data model with SLP I/O (@talmo)
#356: docs: Document ROI and segmentation mask support in SLP format (@talmo)
#357: feat(coco): Add ROI and segmentation mask I/O for COCO format (@talmo)
#358: feat(ultralytics): Add detection and segmentation format I/O (@talmo)
#361: fix: Preserve ROIs/masks in lazy copy and fix COCO empty segmentation fallback (@talmo)

Full Changelog: v0.6.4...v0.6.5

v0.6.4 ¶

sleap-io v0.6.4 Release Notes

Summary

This release adds a sio export command for analysis-ready data export, fixes an out-of-memory crash in video rendering, and adds support for newer DeepLabCut file formats.

Analysis Export: New sio export command for exporting pose data to CSV and HDF5 with frame padding, range selection, and multi-video batch export
Memory-Efficient Rendering: sio render now streams frames to disk instead of accumulating them in memory, preventing OOM crashes on long videos
DLC v2 Support: Automatically loads CSV files from newer DeepLabCut versions that use multi-column image paths
Pandas 3.0 Compatibility: Internal test fix for compatibility with Pandas 3.0's new StringDtype

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

New Features

`sio export` Command for Analysis-Ready Data Export (#347)

A new dedicated CLI command for exporting pose tracking data to analysis-friendly formats. Unlike sio convert (which converts between pose tracking formats), sio export is optimized for downstream analysis workflows with options for frame padding, range selection, and batch export.

Basic Usage

# Export to CSV (default: frames-wide format with NaN padding for missing frames)
sio export labels.slp -o tracks.csv

# Export to HDF5 analysis format
sio export labels.slp -o analysis.h5

# Export a specific frame range
sio export labels.slp -o tracks.csv --start 100 --end 500

# Export all videos in a multi-video file
sio export labels.slp -o tracks.csv -v all
# Creates: tracks.video0.csv, tracks.video1.csv, ...

CSV Format Options

Five CSV layout formats are available via --csv-format:

Format	Description
`frames` (default)	One row per frame, columns for all coordinates
`instances`	One row per instance per frame
`points`	One row per keypoint per instance per frame
`sleap`	SLEAP-native column naming
`dlc`	DeepLabCut-compatible multi-header format

# Export in DLC-compatible format
sio export labels.slp -o tracks.csv --csv-format dlc --scorer MyModel

# Export point-level data with scores
sio export labels.slp -o tracks.csv --csv-format points --include-scores

Frame Padding

By default, sio export pads missing frames with NaN values for continuous time-series output. This ensures every frame in the range has a row, which is important for analysis tools that expect uniform sampling.

# Disable padding (only export frames with instances)
sio export labels.slp -o tracks.csv --no-empty-frames

Memory-Efficient Chunked Writing

For large datasets, use chunked CSV writing to limit memory usage:

sio export labels.slp -o tracks.csv --chunk-size 5000

Python API

import sleap_io as sio

# Save with frame padding and range selection
sio.save_csv(
    "tracks.csv",
    labels,
    include_empty=True,     # Pad missing frames with NaN
    start_frame=100,
    end_frame=500,
    chunk_size=5000,        # Memory-efficient chunked writing
)

Analysis HDF5 via `sio convert`

HDF5 analysis export is now also available through sio convert:

sio convert labels.slp -o analysis.h5
sio convert labels.slp -o analysis.h5 --h5-dim-order standard

Bug Fixes

Out-of-Memory Crash in Video Rendering (#348)

Fixed: render_video() and sio render would crash with an out-of-memory error on long or high-resolution videos.

Root Cause: All rendered frames were accumulated in a Python list before being written to disk. For a 1080p 10-minute video at 30fps (~18,000 frames at ~6 MB each), this required ~36 GB of memory.

Fix: Frames are now streamed directly to the VideoWriter as they are rendered. Peak memory is now constant (~5 MB) regardless of video length -- a 91% reduction in memory usage.

Scenario	Before	After
50 frames at 640x480	60.7 MB	5.3 MB
1080p, 10 min @ 30fps	~36 GB (OOM)	~12 MB

No changes to the API or CLI -- sio render and render_video() work exactly as before, just without crashing.

Newer DeepLabCut File Format Support (#343)

Fixed: Loading CSV files produced by newer versions of DeepLabCut returned 0 labeled frames.

Root Cause: Newer DLC versions (DeepLabCut PR #1584) changed how image paths are stored in CSV files. Instead of a single column with the full path (labeled-data/video/img000.png), the path is split across three columns as a pandas MultiIndex.

Fix: The DLC loader now auto-detects the newer format and handles it transparently. All three DLC variants (single-animal, multi-animal, multi-animal with unique tracking) are supported in both old and new formats.

# Works automatically with both old and new DLC files
labels = sio.load_file("dlc_project/CollectedData.csv")

Pandas 3.0 Test Compatibility (#344)

Fixed: The test_polars_pandas_equivalence test failed on Pandas 3.0 due to the new StringDtype not being compatible with np.issubdtype(). The comparison logic was simplified to use pd.testing.assert_frame_equal. This is an internal test-only fix with no impact on user-facing behavior.

API Changes

New Symbols

Type	Name	Location	Description
Function	`export()`	`sleap_io.io.cli`	CLI command for analysis-ready data export

New Parameters

Function	New Parameters
`save_csv()`	`include_empty`, `start_frame`, `end_frame`, `chunk_size`, `video_id`
`to_dataframe()`	`all_frames`, `start_frame`, `end_frame`

Modified Behavior

Symbol	Change
`sio convert`	Now supports `analysis_h5` output format (`.h5`/`.hdf5` extensions)
`sio convert`	New `--h5-dim-order` and `--min-occupancy` options for HDF5 output
`save_csv(include_empty=...)`	Previously accepted but ignored; now properly wired through to pad missing frames

CLI Changes

Command	Change
`sio export`	New command for analysis-ready CSV and HDF5 export
`sio convert`	Added `analysis_h5` output support with `--h5-dim-order` and `--min-occupancy` options

Coordination Notes

For CSV Import/Export

sleap-io now loads CSV files from both older and newer DLC versions automatically
Export to DLC-compatible format: sio export labels.slp -o tracks.csv --csv-format dlc

For Analysis Pipelines

The new sio export command is designed for feeding pose data into downstream analysis (e.g., behavior classification, kinematic analysis)
Frame padding (--empty-frames) ensures continuous time series suitable for tools expecting uniform sampling
Chunked writing (--chunk-size) enables export of very large datasets without memory issues

Changelog

#343: fix(io): Add support for loading newer DLC files (@lochhh)
#344: fix(tests): Simplify equivalence check between Polars and Pandas DataFrames (@lochhh)
#347: feat(cli): Add sio export command for analysis-ready data export (@talmo)
#348: fix(rendering): Stream frames to VideoWriter to prevent OOM (@talmo)

Full Changelog: v0.6.3...v0.6.4

v0.6.3 ¶

sleap-io v0.6.3 Release Notes

Summary

This release adds negative frame support for training data and fixes critical bugs in the CLI commands that were causing embedded image data loss.

Negative Frames: Mark frames as containing no instances (pure background) for better model training
Embedded Image Preservation: CLI commands (sio fix, sio convert, sio merge, sio unsplit) now preserve embedded images by default
Smart Skeleton Consolidation: The --consolidate-skeletons flag now reassigns compatible instances instead of deleting them

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

New Features

Negative Frames Support (#341)

A new feature to mark frames as explicitly containing no instances (pure background). This is valuable for training pose estimation models as it helps models learn what backgrounds look like without any animals present.

Key distinction: Negative frames are different from "empty frames" (frames where instances were deleted). Negative frames represent intentional user annotation indicating "there is nothing to label here."

Creating Negative Frames

import sleap_io as sio

# Create a labeled frame marked as negative
lf = sio.LabeledFrame(
    video=video,
    frame_idx=42,
    instances=[],  # No instances
    is_negative=True  # Explicitly marked as negative
)

# Add to labels
labels.append(lf)

Checking Frame Status

# Check if a frame is explicitly negative
if lf.is_negative:
    print(f"Frame {lf.frame_idx} is a negative/background frame")

# Check if a frame represents user annotation (instances OR negative)
if lf.is_user_labeled:
    print(f"Frame {lf.frame_idx} was intentionally annotated by user")

Accessing Negative Frames

# Get all negative frames from a Labels object
negative_frames = labels.negative_frames
print(f"Found {len(negative_frames)} negative frames")

# Negative frames are included in user_labeled_frames for training
training_frames = labels.user_labeled_frames
# Includes: frames with user instances + negative frames

Cleaning Labels

# clean() preserves negative frames when removing empty frames
labels.clean(frames=True)
# Empty non-negative frames are removed
# Negative frames are preserved even though they have no instances

HDF5 Storage Format

Negative frames are stored in a new optional HDF5 dataset:

file.slp
├── /frames
├── /instances
├── /points
├── /pred_points
└── /negative_frames   # NEW - optional dataset

The dataset stores (video_id, frame_idx) tuples using sparse video IDs for consistency with the /frames dataset.

Backwards Compatibility:

Old sleap-io versions can open new files (negative frame info simply won't be loaded)
New sleap-io can open old files (all frames default to is_negative=False)

Bug Fixes

Embedded Images Now Preserved in CLI Commands (#340)

Fixed: CLI commands (sio fix, sio convert, sio merge, sio unsplit) were silently stripping embedded images from regular .slp files that had embedded videos via HDF5 datasets.

Symptom: Opening an SLP file with embedded images, making changes, and saving would cause:

Warnings about missing image file paths during save
Complete data loss upon reopening (images could not be loaded)
Significantly decreased file size (embedded data removed)

Root Cause: The embedded detection logic only recognized .pkg.slp files, not regular .slp files with embedded videos.

Fix: All affected commands now use embed=None by default, which preserves whatever embedding state exists in the input file.

# These commands now preserve embedded images by default:
sio fix labels.slp -o fixed.slp
sio convert embedded.slp -o converted.slp
sio merge file1.slp file2.slp -o merged.slp
sio unsplit train.slp val.slp -o combined.slp

Smart Skeleton Consolidation (#340)

Fixed: The --consolidate-skeletons flag in sio fix was deleting all instances from non-primary skeletons, even when those skeletons were structurally identical.

Before: All instances from duplicate skeletons were deleted.

After:

Compatible skeletons (same nodes): Instances are reassigned to the most frequent skeleton
Incompatible skeletons (different structure): Instances are deleted (with warning)

# Skeleton consolidation with smart reassignment:
sio fix labels.slp --consolidate-skeletons -o fixed.slp
# Output: "Reassigned 258 instances from 2 compatible skeleton(s)."

Related Fix: SLEAP Issue #2546

This release resolves the root cause of SLEAP issue #2546 where embedded images were being stripped after save operations in the SLEAP GUI.

API Changes

New Symbols

Type	Name	Location	Description
Attribute	`LabeledFrame.is_negative`	`sleap_io.model.labeled_frame`	`bool` - If `True`, frame is marked as containing no instances (background)
Property	`LabeledFrame.is_user_labeled`	`sleap_io.model.labeled_frame`	`bool` - Returns `True` if frame has user instances OR is negative
Property	`Labels.negative_frames`	`sleap_io.model.labels`	`list[LabeledFrame]` - All frames marked as negative
Function	`write_negative_frames()`	`sleap_io.io.slp`	Writes negative frame markers to SLP file
Function	`read_negative_frames()`	`sleap_io.io.slp`	Reads negative frame markers from SLP file

Modified Behavior

Symbol	Change
`Labels.user_labeled_frames`	Now includes negative frames (frames with `is_negative=True`)
`Labels.clean(frames=True)`	Preserves negative frames when removing empty frames
`sio fix`	Uses `embed=None` by default (preserves embedded images)
`sio convert`	Uses `embed=None` for SLP→SLP conversions
`sio merge`	Uses `embed=None` by default
`sio unsplit`	Uses `embed=None` by default
`--consolidate-skeletons`	Reassigns compatible instances instead of deleting

Coordination Notes

For SLEAP

Upgrading to sleap-io ≥0.6.3 will resolve issue #2546 (embedded images stripped after save)
The SLEAP GUI will need updates to allow users to mark frames as negative
Negative frames should be included when exporting training data

For sleap-nn

Training data pipelines should include negative frames from labels.user_labeled_frames
Data loaders may need updates to handle frames with zero instances (use for background classification)

Changelog

#340: fix(cli): Preserve embedded images and reassign instances in sio fix (@talmo)
#341: feat(model): Add support for negative frames (pure background) (@talmo)

Full Changelog: v0.6.2...v0.6.3

v0.6.2 ¶

sleap-io v0.6.2 Release Notes

Summary

This release introduces content-based video matching - a major enhancement that uses pose annotations and image pixels to match videos even when file paths differ completely. Key highlights:

Pose-Based Video Matching: Videos are automatically matched when they share identical pose annotations, enabling reliable cross-platform merges
Image-Based Video Matching: Optional pixel comparison for matching videos without annotations
New Labels.match() API: Inspect matching results without merging - ideal for evaluation workflows
Video Color Mode Control: New Labels.set_video_color_mode() method and sio fix --video-color CLI option

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

# Or with pip
pip install --upgrade "sleap-io[all]"

See installation docs for more options.

New Features

Content-Based Video Matching (#336)

A major enhancement to the AUTO video matching algorithm that adds two new matching signals based on video content rather than file paths. This dramatically improves merge reliability when file paths differ between systems.

Pose-Based Matching (Default)

Videos with identical pose annotations on common frames are now automatically matched, even when file paths differ completely. This is the key enabler for cross-platform merges.

How it works:

Find common frame indices between two videos (frames that both have annotations)
For each common frame, compare pose coordinates between all instance pairs
If at least one instance pair has exactly identical coordinates (0 difference, with NaN handling)
And this holds for at least 3 frames (configurable), the videos are considered a match

Why this is reliable: Pose coordinates are stored as floats with high precision. The probability of two unrelated videos having identical pose coordinates on multiple frames is essentially zero - this is a strong cryptographic-like signal without requiring any hashing.

Example - Cross-Platform Merge:

import sleap_io as sio

# Labels from Linux workstation (paths like /mnt/data/fly.mp4)
linux_labels = sio.load_slp("annotations_linux.pkg.slp")

# Labels from Windows workstation (paths like X:\data\fly.mp4)
windows_labels = sio.load_slp("annotations_windows.pkg.slp")

# Automatically matches videos by pose content!
linux_labels.merge(windows_labels)  # video="auto" (default)

When this helps:

Cross-platform merges (Linux ↔ Windows ↔ macOS)
Embedded videos in .pkg.slp files where original files aren't accessible
Videos that have been moved or renamed
Combining predictions from different machines

Image-Based Matching (Opt-in)

For videos without annotation overlap, you can enable pixel-based content matching. This compares actual frame pixels to identify identical videos.

How it works:

Find common embedded frame indices between videos
Decode frames and convert to grayscale float (0-1 scale)
Compute mean absolute pixel difference between frames
If difference is below threshold (default: 0.05 ≈ 13/255 pixel difference) on enough frames, match

Example:

from sleap_io.model.matching import VideoMatcher

# Enable image comparison for videos without pose overlap
base.merge(other, video=VideoMatcher(
    method="auto",
    compare_images=True,                 # Enable pixel comparison
    image_similarity_threshold=0.05,     # Max mean pixel diff (0-1 scale)
    content_frames=3,                    # Require 3 matching frames
))

Threshold guidance:

0.05 (default): ~13/255 pixel difference - good for reencoded videos
0.01: Very strict (~3/255 pixels) - for exact copies
0.1: Lenient (~26/255 pixels) - for videos with minor processing differences

Note: Image matching requires decoding video frames, which is computationally expensive. It's disabled by default and runs only when pose matching doesn't find a match.

Controlling Content Matching

The VideoMatcher class provides fine-grained control:

from sleap_io.model.matching import VideoMatcher

matcher = VideoMatcher(
    method="auto",
    # Pose matching options
    content_frames=3,              # Min matching frames required (default: 3)
    compare_predictions="auto",    # Include predictions: "auto", True, False

    # Image matching options (disabled by default)
    compare_images=False,          # Enable pixel comparison
    image_similarity_threshold=0.05,  # Max mean pixel diff (0-1)
)

# Use with merge
base.merge(other, video=matcher)

# Use with match (new API)
result = base.match(other, video=matcher)

compare_predictions modes:

"auto" (default): Include predictions only if video has NO user instances (100% predictions)
True: Always include predictions in pose comparison
False: Only compare user-labeled instances (strictest)

AUTO Matching Cascade (Updated)

The video="auto" algorithm now includes content matching in its cascade:

Step	Check	Result
1	Shape incompatible (frames, H, W differ)	Reject
2	Provenance conflict (different `original_video`, verifiable)	Reject
3	Same physical file (`os.path.samefile`)	Match
4	Exact path string match	Match
5	Unique basename/parent suffix match	Match
6	Pose matching (identical annotations)	Match ← NEW
7	Image matching (pixel similarity, if enabled)	Match ← NEW
8	No match found	Add as new

`Labels.match()` API - Matching Without Merging (#337, #338)

A new API for inspecting matching results without actually merging datasets. This is particularly useful for evaluation workflows where you need to align predictions with ground truth before computing metrics.

import sleap_io as sio

gt_labels = sio.load_slp("ground_truth.slp")
pred_labels = sio.load_slp("predictions.slp")

# Match predictions to ground truth (doesn't modify either dataset)
result = gt_labels.match(pred_labels)

# Check matching results
print(result.summary())
# Videos: 2/2 matched
# Skeletons: 1/1 matched
# Tracks: 0/0 matched

# Inspect specific matches
if not result.all_videos_matched:
    for video in result.unmatched_videos:
        print(f"Warning: Could not match {video.filename}")

# Iterate through matched videos
for pred_video, gt_video in result.video_map.items():
    if gt_video is not None:
        print(f"{pred_video.filename} -> {gt_video.filename}")

The MatchResult object provides:

Property	Type	Description
`video_map`	`dict[Video, Video \| None]`	Maps other's videos to self's videos
`skeleton_map`	`dict[Skeleton, Skeleton \| None]`	Maps skeletons
`track_map`	`dict[Track, Track \| None]`	Maps tracks
`unmatched_videos`	`list[Video]`	Videos with no match
`all_videos_matched`	`bool`	True if all videos matched
`n_videos_matched`	`int`	Count of matched videos
`summary()`	`str`	Human-readable summary

MatchResult is exported directly from sleap_io:

from sleap_io import MatchResult

Video Color Mode Control (#335)

New API to batch-configure video color mode (grayscale/RGB) for all videos in a dataset. This is useful when auto-detection fails due to compression artifacts or videos with very similar color channels.

Python API:

import sleap_io as sio

labels = sio.load_file("labels.slp")

# Force grayscale (single-channel) output
labels.set_video_color_mode("grayscale")

# Force RGB (three-channel) output
labels.set_video_color_mode("rgb")

# Reset to auto-detection (default)
labels.set_video_color_mode("auto")

labels.save("labels_fixed.slp")

CLI:

# Fix video color mode issues
sio fix labels.slp -o fixed.slp --video-color grayscale
sio fix labels.slp -o fixed.slp --video-color rgb
sio fix labels.slp -o fixed.slp --video-color auto

The sio fix command now also reports video color mode analysis, showing mismatches between settings and actual video channels.

Bug Fixes

HDF5 Dataset Matching (#336)

Fixed bug where different videos embedded in the same .pkg.slp file could incorrectly match each other. The matcher now correctly checks the HDF5 dataset path to distinguish videos.

Before: Video.matches_path() compared only source_filename, but all embedded videos in the same pkg.slp share the same source_filename.

After: Different HDF5 datasets are correctly recognized as different videos.

Provenance Conflict Logic (#336)

Fixed original_videos_conflict() to allow fall-through when source files can't be verified on disk. Previously, embedded videos with different original_video paths were rejected even when the files couldn't be checked. Now content-based matching can proceed for embedded videos.

Video Color Mode Propagation (#335)

Fixed issue where setting video color mode on embedded videos (.pkg.slp) would be lost when unembedding. The setting now propagates through the entire source_video chain.

Documentation

Merging Guide - Matching Without Merging (#338)

Added new section to the merging documentation explaining the Labels.match() API with comprehensive examples for evaluation workflows.

API Changes

New Symbols

Type	Name	Location
Method	`Labels.match()`	`sleap_io.model.labels`
Method	`Labels.set_video_color_mode()`	`sleap_io.model.labels`
Class	`MatchResult`	`sleap_io.model.matching`

New CLI Options

Command	Option	Description
`sio fix`	`--video-color {grayscale,rgb,auto}`	Set video color mode for all videos

VideoMatcher Enhancements

New parameters for content-based video matching:

Parameter	Default	Description
`content_frames`	3	Minimum matching frames for pose/image confirmation
`compare_predictions`	`"auto"`	Include predictions in matching: `"auto"`, `True`, `False`
`compare_images`	`False`	Enable pixel-based matching (expensive)
`image_similarity_threshold`	0.05	Max mean pixel difference (0-1 scale)

Changelog

#335: feat: Add video color mode toggling (@talmo)
#336: fix(matching): Add pose-based video matching and fix HDF5 dataset matching for cross-platform merges (@talmo)
#337: feat(matching): Add Labels.match() API for matching without merging (@talmo)
#338: docs(merging): Add documentation for Labels.match() API (@talmo)

Full Changelog: v0.6.1...v0.6.2

v0.6.1 ¶

sleap-io v0.6.1 Release Notes

Summary

This release completes the CLI vision from issue #209 with 8 new commands, bringing the total to 14 CLI commands. Major additions include video transformation with automatic coordinate adjustment (sio transform), flexible label merging (sio merge), and video reencoding for reliable seeking (sio reencode).

Highlights:

8 New CLI Commands: merge, unsplit, fix, embed, unembed, trim, reencode, transform
New I/O Formats: CSV and SLEAP Analysis HDF5 format support
Video FPS Support: Full round-trip FPS preservation through loading, saving, and reencoding
Python 3.10+ Required: Minimum version bumped from 3.8 to 3.10
Performance: 23x faster pkg.slp saves, 2.7x faster embedded video loading

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx [email protected] show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

# Or with pip
pip install --upgrade "sleap-io[all]"

See installation docs for more options.

Breaking Changes

Python 3.10+ Required (#322)

The minimum Python version has been bumped from 3.8 to 3.10 to align with optional dependencies (PyAV, Polars) and enable modern type hint syntax.

Action: Upgrade to Python 3.10 or later if you haven't already.

New CLI Commands

`sio transform` - Coordinate-Aware Video Transformations (#326)

Apply geometric transformations to videos while automatically adjusting landmark coordinates to maintain alignment.

# Crop to region of interest
sio transform labels.slp -o cropped.slp --crop 100,100,500,500

# Scale video and coordinates
sio transform labels.slp -o scaled.slp --scale 0.5

# Multiple transformations
sio transform labels.slp -o output.slp --crop 0,0,512,512 --rotate 90 --flip horizontal

# Per-video parameters via YAML config
sio transform labels.slp -o output.slp --config transforms.yaml

# Preview mode (show transformed dimensions without processing)
sio transform labels.slp --crop 100,100,400,400 --dry-run

Supported transformations: --crop, --scale, --rotate, --pad, --flip

`sio merge` - Flexible Labels Merging (#317)

Merge multiple SLEAP files with full control over matching strategies.

# Basic merge
sio merge base.slp predictions.slp -o merged.slp

# Replace old predictions with new ones (keep manual labels)
sio merge project.slp new_preds.slp -o updated.slp --frame replace_predictions

# Merge with explicit matching strategies
sio merge base.slp other.slp -o out.slp --video path --track name

`sio unsplit` - Merge Split Files (#313)

Reverse sio split by merging train/val/test files back into one.

# Merge from directory
sio unsplit splits/ -o merged.slp

# Merge specific files
sio unsplit train.slp val.slp test.slp -o merged.slp

`sio fix` - Labels File Maintenance (#314)

Detect and repair common issues in SLEAP labels files.

# Show issues without fixing
sio fix labels.slp --dry-run

# Fix with safe defaults
sio fix labels.slp -o fixed.slp

# Fix specific issues
sio fix labels.slp -o fixed.slp --remove-empty-frames --consolidate-skeletons

# Update video paths
sio fix labels.slp -o fixed.slp --prefix /old/path /new/path

Detects: Duplicate videos, unused skeletons, empty frames, path issues.

`sio embed` / `sio unembed` - Granular Frame Embedding (#315)

Fine-grained control over frame embedding in package files.

# Embed only user-labeled frames
sio embed labels.slp -o labels.pkg.slp --user

# Embed user + predictions
sio embed labels.slp -o labels.pkg.slp --user --predictions

# Embed everything
sio embed labels.slp -o labels.pkg.slp --all

# Restore external video references
sio unembed labels.pkg.slp -o labels.slp

`sio trim` - Clip Videos + Labels (#316)

Trim videos and labels to specific frame ranges.

# Trim labels and video
sio trim labels.slp -o clipped.slp --start 100 --end 500

# Trim standalone video
sio trim video.mp4 -o clip.mp4 --start 0 --end 1000

`sio reencode` - Reliable Video Seeking (#319)

Reencode videos with frequent keyframes for frame-accurate seeking.

# Reencode video for reliable seeking
sio reencode video.mp4 -o reencoded.mp4

# Reencode all videos in a labels file
sio reencode labels.slp -o fixed_labels.slp

# Custom keyframe interval (default: every frame)
sio reencode video.mp4 -o out.mp4 --keyframe-interval 10

Use case: Fixes videos where seeking to frame N returns frame N±1, which corrupts annotations.

New I/O Formats

CSV Format (#308)

import sleap_io as sio

# Load CSV (multiple formats supported)
labels = sio.load_csv("poses.csv", format="sleap")
labels = sio.load_csv("dlc_output.csv", format="dlc")

# Save to CSV
sio.save_csv(labels, "output.csv", format="points")

Formats: sleap, dlc, points, instances, frames

CLI:

sio convert labels.slp -o poses.csv
sio convert poses.csv -o labels.slp --from csv

SLEAP Analysis HDF5 Format (#309)

Read/write SLEAP's Analysis HDF5 format for MATLAB interoperability.

import sleap_io as sio

# Load analysis file
labels = sio.load_analysis("analysis.h5")

# Save with MATLAB-compatible axis ordering
sio.save_analysis(labels, "analysis.h5", preset="matlab")

# Custom axis ordering
sio.save_analysis(labels, "analysis.h5", axis_order=("time", "nodes", "coordinates", "tracks"))

New Features

Video FPS Support (#307)

FPS is now a first-class property on Video objects with automatic extraction and round-trip preservation.

import sleap_io as sio

labels = sio.load_file("labels.slp")
video = labels.videos[0]

# Access FPS
print(video.fps)  # e.g., 30.0

# Set FPS (useful for image sequences)
video.fps = 25.0

# Convert frame to timestamp
timestamp = video.frame_to_time(100)  # Returns time in seconds

Enhanced `sio show` (#310, #311, #330)

# Clearer instance counts (user vs predicted)
sio show labels.slp

# View video encoding info for standalone videos
sio show video.mp4
# Output includes: codec, pixel format, FPS, bitrate, GOP size

# Inspect video provenance chain
sio filenames labels.slp --all
sio filenames labels.slp --original
sio filenames labels.slp --source

Automatic Embedded Video Preservation (#328)

CLI commands now automatically preserve embedded videos when converting between .pkg.slp files:

# Embedded frames are preserved automatically (no --embed needed)
sio convert input.pkg.slp -o output.pkg.slp
sio merge base.pkg.slp other.pkg.slp -o merged.pkg.slp
sio fix input.pkg.slp -o fixed.pkg.slp

Performance Improvements

23x Faster pkg.slp Saves (#327)

Saving embedded videos is now dramatically faster by copying raw encoded bytes directly when formats match.

Operation	Before	After	Speedup
Save pkg.slp with embedded video	23s	1s	23x

2.7x Faster Embedded Video Loading (#310)

Loading .pkg.slp files with many embedded videos is now significantly faster by avoiding repeated file opens.

Bug Fixes

Fix Video Matching for Embedded Videos (#323)

Fixed "Frame index out of range" errors when merging .pkg.slp files with sio unsplit --embed. Videos are now compared by HDF5 dataset identity, not just filename.

Fix Rendering Crop Offset (#331)

RenderContext.world_to_canvas() now returns correct coordinates when using the crop parameter.

Fix Embedded Video Data Loss (#332)

Fixed critical bug where loading .pkg.slp files with open_videos=False would lose all embedded frames on save. Embedded videos are now detected and preserved via backend metadata.

Fix x264 Coordinate Alignment (#333)

Frames are now padded on bottom/right edges only (instead of scaling) when encoding x264 videos with dimensions not divisible by 16, preserving keypoint coordinate accuracy.

Improvements

Modernized Type Hints (#325)

All type hints updated to Python 3.10+ syntax:

Union[X, Y] → X | Y
Optional[X] → X | None
List, Dict → list, dict

Video.original_video Refactor (#312)

Video.original_video is now a computed property that traverses the source_video chain, eliminating redundant HDF5 storage while maintaining backward compatibility.

Documentation (#318, #320, #321, #324)

Comprehensive SLP file format reference (~400 lines)
Per-PR isolated docs previews
Cleaner CLI docs navigation

Changelog

#307: feat: Add FPS video property support (@talmo)
#308: feat: Add formal CSV I/O support (@talmo)
#309: feat: Add Analysis HDF5 format I/O support (@talmo)
#310: feat: Enhance sio show command and fix load_slp performance for embedded videos (@talmo)
#311: feat: Enhance sio filenames command and fix Video.filename consistency (@talmo)
#312: refactor: Make original_video a computed property from source_video chain (@talmo)
#313: feat: Add sio unsplit command to merge split labels files (@talmo)
#314: feat: Add sio fix command for labels file maintenance (@talmo)
#315: feat: Add sio embed/unembed commands for granular frame embedding (@talmo)
#316: feat: Add sio trim command for clipping videos + labels (@talmo)
#317: feat: Add sio merge command for flexible labels merging (@talmo)
#318: docs: Add comprehensive SLP file format reference (@talmo)
#319: feat: Add sio reencode command for reliable video seeking (@talmo)
#320: feat: Add PR-local docs preview deployment (@talmo)
#321: feat: Enhance docs preview with changed page links and markdown sources (@talmo)
#322: chore: Bump minimum Python version from 3.8 to 3.10 (@talmo)
#323: fix: Compare HDF5 datasets in video matching to prevent wrong video assignment (@talmo)
#324: docs: Improve CLI docs navigation structure (@talmo)
#325: refactor: Modernize type hints to Python 3.10+ syntax (@talmo)
#326: feat: Add sio transform CLI command for coordinate-aware video transformations (@talmo)
#327: perf: Add fast path for embedded video saving in pkg.slp files (@talmo)
#328: feat(cli): Preserve embedded videos by default in pkg.slp to pkg.slp operations (@talmo)
#330: feat(cli): Show video encoding info in sio show for standalone videos (@talmo)
#331: fix(rendering): Pass crop_offset to RenderContext in callbacks (@talmo)
#332: fix(slp): Preserve embedded videos when loaded with open_videos=False (@talmo)
#333: fix(video_writing): Pad frames to macro_block_size=16 for x264, bottom/right only (@talmo)
#334: chore: Bump version to 0.6.1 (@talmo)

Full Changelog: v0.6.0...v0.6.1

v0.6.0 ¶

Summary

This release transforms sleap-io into a comprehensive pose data toolkit with three major new capabilities: a CLI overhaul with 4 new commands, a high-performance pose rendering module, and an in-memory codecs package for seamless data analysis workflows. Additionally, lazy loading delivers ~90x faster SLP file operations for large prediction files.

Highlights:

CLI Overhaul: 6 commands, 166 tests, comprehensive documentation - a full-featured command-line tool
Rendering: Publication-ready pose videos at ~50 FPS with skia-python
Codecs: Convert Labels to/from Dict, NumPy, and DataFrame (pandas/polars)
Lazy Loading: ~90x faster loading for large SLP files

Thanks to @tom21100227 for contributing the standard color palette (#301)!

Breaking Changes

Simplified Merge API (#300)

The Labels.merge() API has been redesigned for safety and simplicity.

Parameter names simplified:

Old (0.5.x)	New (0.6.0)
`skeleton_matcher=`	`skeleton=`
`video_matcher=`	`video=`
`track_matcher=`	`track=`
`frame_strategy=`	`frame=`
`instance_matcher=`	`instance=`

Default frame strategy renamed: "smart" → "auto"

String arguments now accepted: No imports needed for simple cases.

# Old API (0.5.x)
from sleap_io.model.matching import VideoMatcher, VideoMatchMethod
base.merge(predictions, video_matcher=VideoMatcher(method=VideoMatchMethod.PATH), frame_strategy="smart")

# New API (0.6.0) - simple
base.merge(predictions)  # uses auto defaults

# New API (0.6.0) - explicit
base.merge(predictions, video="path", frame="auto")

Removed Unused APIs (#302)

The following unused APIs were removed during a post-merge audit:

Removed	Reason
`FrameMatcher` class	Never used - frames are uniquely identified by `(video, frame_idx)`
`SOURCE_VIDEO_MATCHER` constant	Identical to `BASENAME_VIDEO_MATCHER`
`VideoNotFoundError` exception	Defined but never raised

Code importing these will need to remove the imports. These were dead code with no production usage.

Performance Improvements

Lazy Loading for SLP Files (#296)

Load large prediction files almost instantly with the new lazy=True parameter. Object creation is deferred until needed, enabling fast workflows for analysis and CLI operations.

Scenario	Eager	Lazy	Speedup
Load only	0.47s	0.005s	~90x
Load + numpy()	0.86s	0.38s	~2x
Load + to_dataframe()	0.13s	0.09s	~1.4x
`sio show` CLI	0.84s	0.36s	~2.3x

Benchmarks on 18,000 frames with ~40,000 instances.

import sleap_io as sio

# Fast loading for analysis workflows
labels = sio.load_slp("predictions.slp", lazy=True)
print(labels.is_lazy)  # True

# Fast stats (O(1) - no iteration needed)
print(labels.n_pred_instances)  # Instant count

# Fast numpy/DataFrame export - no Instance objects created
arr = labels.numpy()
df = labels.to_dataframe(format="points")

# Materialization for modification
eager = labels.materialize()
eager.append(new_frame)  # Now works

CLI: sio show uses lazy loading by default for SLP files.

sio show predictions.slp           # Fast (lazy)
sio show predictions.slp --no-lazy # Force eager

Impact: Enables instant CLI startup and interactive workflows with large prediction files.

New Features

CLI: New Commands (#280, #285, #286, #288)

The sleap-io CLI receives a major upgrade with 4 new commands and comprehensive documentation at io.sleap.ai.

`sio convert` - Format Conversion

Convert between 9+ pose data formats with automatic format detection.

# Basic conversion (formats inferred from extensions)
sio convert labels.slp -o labels.nwb

# Explicit format for ambiguous inputs
sio convert annotations.json -o labels.slp --from coco

# Embed frames in output
sio convert labels.slp -o labels.pkg.slp --embed user

Supported formats: slp, nwb, coco, labelstudio, alphatracker, jabs, dlc, ultralytics, leap

`sio split` - Dataset Splitting

Create reproducible train/val/test splits for machine learning workflows.

# Default 80/20 train/val split
sio split labels.slp -o splits/

# Three-way split with seed for reproducibility
sio split labels.slp -o splits/ --train 0.7 --val 0.15 --test 0.15 --seed 42

# Embed user-labeled frames for portable training data
sio split labels.slp -o splits/ --embed user --seed 42

Output: train.slp, val.slp, test.slp (or .pkg.slp with --embed)

`sio filenames` - Video Path Management

Inspect and update video paths when moving projects between systems.

# Inspection mode - list all video paths
sio filenames labels.slp

# Update mode - replace prefixes (cross-platform)
sio filenames labels.slp -o fixed.slp --prefix /old/path /new/path

`sio render` - Pose Visualization

Render publication-ready videos and images with pose overlays.

# Video rendering
sio render predictions.slp -o output.mp4
sio render predictions.slp --preset preview  # Fast 0.25x

# Single frame
sio render predictions.slp --frame 42

# Styling
sio render predictions.slp --color-by track --palette tableau10

# Render without source video (solid background)
sio render predictions.slp --background black

CLI: Improvements (#279, #281, #292, #298, #303)

Enhanced sio show:

Video index parameter: sio show labels.slp -v 2
Standalone video file inspection: sio show recording.mp4
Full absolute paths for easy copy-paste
Plugin status in --version output
Solarized theme for clean appearance

Consistent input handling: All commands now accept input files both as positional arguments AND via -i/--input:

# Both forms work identically for all commands
sio show labels.slp
sio show -i labels.slp

sio convert labels.slp -o out.nwb
sio convert -i labels.slp -o out.nwb

Additional improvements:

-h works as alias for --help on all commands
Color/palette discovery: sio render --list-colors, sio render --list-palettes
Clear error messages for conflicting inputs
166 CLI tests for comprehensive coverage

Pose Rendering Module (#288)

New sleap_io.rendering module for high-performance pose visualization using skia-python (~50 FPS for 1024x1024 frames).

import sleap_io as sio

# Render video with pose overlays
sio.render_video(labels, "output.mp4")

# Quick preview at reduced resolution
labels.render("preview.mp4", preset="preview")

# Single frame with custom styling
sio.render_image(
    labeled_frame,
    "frame.png",
    color_by="track",
    palette="tableau10",
    marker_shape="diamond"
)

# Render to numpy array
img = sio.render_image(labeled_frame)

Capabilities:

Feature	Options
Color schemes	`track`, `instance`, `node`, `auto`
Palettes	9 built-in + 200+ via colorcet; `standard` default (MATLAB colors)
Marker shapes	circle, square, diamond, triangle, cross
Quality presets	`preview` (0.25x), `draft` (0.5x), `final` (1.0x)
Background	video frame, solid color, or transparent

Advanced features:

Cropping with pixel or normalized coordinates
Custom callbacks for overlays (labels, frame info, etc.)
Progress tracking and cancellation

Impact: Publication-ready pose videos without requiring the SLEAP GUI.

Codecs Package for In-Memory Serialization (#290)

New sleap_io.codecs package for flexible conversion between Labels and various in-memory representations.

Three codecs:

Codec	Methods	Use Case
Dictionary	`to_dict()`, `from_dict()`	JSON serialization, web APIs
NumPy	`numpy()`, `from_numpy()`	ML pipelines, signal processing
DataFrame	`to_dataframe()`, `from_dataframe()`	Tabular analysis, export

DataFrame formats:

# One row per point (most normalized)
df = labels.to_dataframe(format="points")

# One row per instance (ML-ready)
df = labels.to_dataframe(format="instances")

# One row per frame (time-series)
df = labels.to_dataframe(format="frames")

# Hierarchical columns (NWB-compatible)
df = labels.to_dataframe(format="multi_index")

Backend support:

# Pandas (default)
df = labels.to_dataframe(backend="pandas")

# Native polars (faster for large datasets)
df = labels.to_dataframe(backend="polars")

# Streaming for memory efficiency
for chunk in labels.to_dataframe_iter(chunk_size=10000):
    process(chunk)

Impact: Seamless integration with pandas/polars/numpy analysis pipelines.

Labels.copy() Method (#289)

Deep copy Labels with control over video backend behavior.

# Default: preserves each video's current open_backend setting
labels_copy = labels.copy()

# Prevent file handles (useful for batch processing)
labels_copy = labels.copy(open_videos=False)

# Force all videos to auto-open
labels_copy = labels.copy(open_videos=True)

Non-mutating save: Save operations no longer mutate the original Labels by default.

# Original labels are NOT modified (default, safer)
labels.save("output.pkg.slp", embed="user")

# With embed_inplace=True: original labels ARE modified (faster)
labels.save("output.pkg.slp", embed="user", embed_inplace=True)

NWB Multisubjects Support (#273)

Export multi-animal pose data to NWB with proper subject linkage using the ndx-multisubjects extension.

from sleap_io.io.nwb_annotations import save_labels

# Basic multi-subject export
save_labels(labels, "output.nwb", use_multisubjects=True)

# With detailed subject metadata
subjects_metadata = [
    {"sex": "M", "species": "Mus musculus", "age": "P30D"},
    {"sex": "F", "species": "Mus musculus", "age": "P45D"},
]
save_labels(
    labels,
    "output.nwb",
    use_multisubjects=True,
    subjects_metadata=subjects_metadata
)

Impact: Proper multi-animal NWB export for neuroscience workflows.

replace_predictions Merge Strategy (#278)

New merge strategy for re-running inference while preserving manual corrections.

# Load project with existing predictions
project = sio.load_file("project.slp")

# Run new inference
new_preds = sio.load_file("new_predictions.slp")

# Replace old predictions, keep all manual labels
project.merge(new_preds, frame="replace_predictions")

Behavior:

Keeps all user instances from base
Removes all predictions from base
Adds only predictions from other (ignores user instances from other)
No spatial matching (clean replacement)

Safe AUTO Video Matching (#300)

Redesigned video matching algorithm for Labels.merge() that prevents silent data corruption. False positives (matching wrong videos) corrupt data irreversibly; false negatives (adding as new) are easily recoverable.

The AUTO cascade:

Step	Check	Result
1-2	Shape rejection	Different (frames, H, W) → reject
3	Provenance conflict	Different `original_video` → reject
4	Physical file identity	`os.path.samefile()` → match
5	Exact path string	Sanitized paths equal → match
6	Leaf uniqueness	Minimal unique suffixes match → match
7	Fallback	Add as new video

Key scenarios:

PKG.SLP predictions → external video: Works via provenance chain traversal
Cross-platform paths (Windows ↔ Linux): Works via leaf path uniqueness
Same basename, different content (fly.mp4 with 1000 vs 500 frames): Rejected by shape mismatch

New helper: Labels.add_video() prevents duplicate video addition.

Progress Callback for Frame Embedding (#283)

Optional callback for GUI applications during frame embedding operations.

def my_progress(current, total):
    print(f"Embedding frame {current}/{total}")
    return True  # Return False to cancel

sio.save_file(
    labels,
    "output.pkg.slp",
    embed="user",
    progress_callback=my_progress
)

Features:

1-based indexing for intuitive display
Cancellation support via ExportCancelled exception
Automatic tqdm disabling when callback provided

Impact: GUI integration with progress bars and cancellation support.

Bug Fixes

Fix Empty Embedded Video References (#282)

Videos without labeled frames are now properly converted to embedded references when exporting package files (.pkg.slp).

Problem: Videos with no labels retained external paths, causing "missing files" errors on other machines.

Solution: All videos are converted to embedded references by default. Use embed_all_videos=False for selective embedding.

Impact: Package files work correctly across machines even when some videos have no labeled frames.

Fix Video Deep Copy Losing Provenance (#302)

Video.__deepcopy__() now preserves the original_video attribute, fixing a critical bug where the provenance chain would break during merge operations.

Impact: Merge operations now correctly track video provenance through the entire chain.

Improvements

imageio-ffmpeg as Core Dependency (#287)

imageio-ffmpeg is now a core dependency, so video operations work out of the box.

# Video operations now work immediately
uvx sleap-io convert labels.slp -o out.pkg.slp --embed user

# No more "no video backend" errors
sio show labels.slp -v  # Works without extra installs

Impact: Zero-config video support for all users.

Enhanced Merge Provenance Tracking (#299)

Merge operations now record additional metadata for better audit trails:

# After merging predictions.slp into labels.slp
labels.provenance["merge_history"][-1]
# {
#     "timestamp": "2025-01-07T14:30:00.123456",
#     "source_filename": "predictions.slp",
#     "target_filename": "labels.slp",
#     "sleap_io_version": "0.6.0",
#     "source_labels": {"n_frames": 100, ...},
#     "result": {"frames_merged": 100, "instances_added": 500}
# }

New fields: source_filename, target_filename, sleap_io_version

Impact: Better data lineage tracking for reproducibility and auditing.

Changelog

#273: Add NWB Multisubjects support (@talmo)
#278: Add replace_predictions merge strategy and rewrite merging docs (@talmo)
#279: Add CLI theming and enhanced version info (@talmo)
#280: Add CLI convert command for format conversion (@talmo)
#281: Redesign CLI cat video display with defensive metadata handling (@talmo)
#282: Fix empty embedded video references for package export (@talmo)
#283: Add progress_callback support for frame embedding (@talmo)
#284: Add CLI documentation (@talmo)
#285: Add CLI split command for train/val/test splits (@talmo)
#286: Add CLI filenames command for inspecting/updating video paths (@talmo)
#287: Add imageio-ffmpeg as core dependency for video support (@talmo)
#288: Add skia-python rendering module for pose visualization (@talmo)
#289: Add Labels.copy() method with open_videos parameter (@talmo)
#290: Add codecs package for in-memory serialization (@talmo)
#292: Enhance CLI show command with video index, full paths, and standalone video display (@talmo)
#293: Add comprehensive installation documentation page (@talmo)
#294: Bump version to 0.6.0 (@talmo)
#296: Add lazy loading for SLP files (@talmo)
#297: Update documentation for v0.6.0 release (@talmo)
#298: Standardize CLI patterns and add render enhancements (@talmo)
#299: Add source/target filenames and version to merge provenance (@talmo)
#300: Implement safe AUTO video matching algorithm for merges (@talmo)
#301: Add standard palette with MATLAB default colors (@tom21100227)
#302: Fix Video.deepcopy() and remove dead code from matching module (@talmo)
#303: Standardize CLI to support both positional and -i flag input (@talmo)
#304: Add missing documentation for v0.6.0 features (@talmo)
#305: Add CI summary job to support docs-only PRs (@talmo)
#306: Update version examples in install.md to 0.6.0 (@talmo)

Full Changelog: v0.5.8...v0.6.0

v0.5.8 ¶

sleap-io v0.5.8

🎯 Summary

This release delivers a dramatic performance improvement with 2000x faster imports through lazy loading, makes imageio-ffmpeg optional to reduce installation size, and includes multiple critical bug fixes for video indexing and matching in SLP files. The v0.5.8 release focuses on reducing friction for users while improving reliability for complex video handling scenarios.

⚡ Performance Improvements

Implement Lazy Loading for 2000x Faster Imports (#270)

Dramatically reduced import time using the lazy-loader library (SPEC 1 standard used by NumPy, SciPy, scikit-image).

Performance Results:

Metric	Before	After	Improvement
Import time	4.38s	0.0022s	1991x faster
Target	<500ms	2.2ms	227x better than target

What's deferred:

pandas (2.24s) - loads only when load_dlc() is called
PyAV (0.67s) - loads only when video is opened
NWB tools (0.53s) - loads only when load_nwb() is called
All format modules - load on first use

# Before: 4.38s to import
# After: 0.0022s to import
import sleap_io

# Functions available immediately (lazy loading is transparent)
labels = sleap_io.load_slp("file.slp")

# First call to load_dlc() imports pandas (one-time ~2s cost)
labels = sleap_io.load_dlc("file.csv")

# Subsequent calls are instant (pandas already cached)
labels = sleap_io.load_dlc("file2.csv")

Key Features:

✅ Zero API changes - Users import and use sleap-io exactly as before
✅ Battle-tested - Uses lazy-loader library from SPEC 1
✅ Type-safe - Works with mypy/pyright
✅ Test coverage - EAGER_IMPORT=1 fixture ensures tests catch missing imports
✅ Lower memory footprint - ~30-40% reduction

Impact: Instant CLI startup and dramatically improved user experience, especially for quick scripts and interactive workflows.

✨ New Features

Make imageio-ffmpeg Optional and Enhance Backend Plugin System (#272)

Made imageio-ffmpeg an optional dependency and added new introspection APIs for better discoverability.

New Optional Dependency Groups:

pip install sleap-io[ffmpeg]  # Recommended for video support
pip install sleap-io[all]     # All backends
pip install sleap-io          # Minimal (no video backends)

New Public API Functions:

import sleap_io as sio

# Check what's available
print(sio.get_available_video_backends())
# Output: ['FFMPEG', 'pyav']

print(sio.get_available_image_backends())
# Output: ['opencv', 'imageio']

# Get installation help
print(sio.get_installation_instructions("opencv"))
# Output: pip install sleap-io[opencv]

Enhanced Error Messages:

Before:

ImportError: No video plugins found. Install opencv-python, imageio-ffmpeg, or av.

After:

ImportError: No video backend plugins are installed.

Available options:
  opencv (fastest):        pip install sleap-io[opencv]
  FFMPEG (most reliable):  pip install sleap-io[ffmpeg]
  pyav (balanced):         pip install sleap-io[pyav]
  all backends:            pip install sleap-io[all]

For more information, see: https://io.sleap.ai

Additional Features:

Smart warnings when preferred backend is not available
Automatic fallback to auto-detection
Updated documentation with new installation options

Impact: Reduces installation footprint and provides better guidance for users setting up video backends.

🐛 Bug Fixes

Fix Video Matching to Prioritize source_filename for HDF5 Backends (#275)

Fixed video matching for .pkg.slp files where multiple videos share the same HDF5 file path but reference different source videos.

Problem: When merging Labels with HDF5 video backends (embedded videos), Video.matches_path() would incorrectly match different videos just because they came from the same HDF5 file.

Solution: For HDF5 backends, matching now prioritizes:

source_filename (the original video path before embedding)
Falls back to dataset name if source_filename is None
Returns False if neither is available (avoids false positives)

# After fix: Correct matching for embedded videos
labels1 = sio.load_slp("project1.pkg.slp")
labels2 = sio.load_slp("project2.pkg.slp")

# Videos now match by original source filename, not HDF5 path
labels1.merge(labels2)  # ✅ Correct video matching

Impact: Critical fix for workflows involving merged predictions with embedded videos.

Fix Video ID Mapping for Sequential IDs with Sparse Dataset Names (#274)

Fixed loading of SLP files exported from larger .pkg.slp files where embedded video datasets have sparse names but sequential frame video IDs.

Problem: When SLP files are exported (e.g., via "Export Labeled Clip..."), the embedded video datasets retain sparse naming (e.g., video51/video, video49/video) but frame video IDs may be sequential (0, 1, 2, 3). This caused incorrect video-frame associations.

Solution: Added detection logic to determine if frame video IDs are sequential list indices or sparse embedded IDs, and apply the appropriate mapping.

Impact: Fixes data integrity issues when working with exported clips from larger projects.

Fix Sparse Video Indexing While Writing SLP Files (#268)

Fixed preservation of sparse video indexing when writing and re-reading SLP files with embedded videos.

Problem: When saving labels with sparse video indices (e.g., videos indexed as 0, 5, 10, 15, 20), the video IDs were incorrectly mapped to sequential indices, causing data loss or misalignment on reload.

Solution: Extract original video IDs from HDF5 dataset names and use them when writing frame data.

Impact: Ensures data integrity for round-trip operations with sparse video indices.

Fix Sparse Video Indexing Bug in read_labels() (#266)

Fixed loading of .slp files with sparse video indices from old SLEAP versions.

Problem: Old SLEAP versions (format_id < 2.0) could create files where video IDs in the frames dataset were sparse (e.g., 0, 15, 29, 47, ...), causing IndexError when loading.

Solution: Build a video_id_to_index mapping from sparse video IDs to sequential list indices when loading.

import sleap_io as sio

# Now works correctly
labels = sio.load_slp("legacy_file.slp")  # ✅ No more IndexError
assert len(labels.videos) == 5
assert len(labels) == 10

Impact: Restores compatibility with legacy SLEAP files that have non-sequential video IDs.

Fix KeyError When backend_metadata Lacks Filename Key (#267)

Fixed loading of SLP files where backend_metadata is missing the "filename" key, particularly when upgrading from SLEAP v1.4 to v1.5+.

Solution: Added fallback chain to handle legacy files and ensure "filename" is always present when writing.

Impact: Improves compatibility with older SLEAP project files.

💡 Why These Changes Matter

The v0.5.8 release significantly enhances sleap-io's performance, reliability, and ease of use:

Instant Imports: 2000x faster import times make sleap-io feel snappy for scripts, notebooks, and CLI tools
Flexible Installation: Optional video backends let users install only what they need, reducing dependencies and installation size
Better Discoverability: New introspection APIs and improved error messages help users configure their environment
Data Integrity: Five bug fixes for video indexing and matching ensure reliable handling of complex video scenarios
Legacy Compatibility: Improved support for older SLEAP file formats and upgrade paths

This release demonstrates sleap-io's commitment to developer experience, reliability, and backwards compatibility for pose tracking research workflows.

📋 Changelog

#266: Fix sparse video indexing bug in read_labels() (@talmo)
#267: Fix KeyError When backend_metadata Lacks Filename Key (@alicup29)
#268: Fix sparse video indexing while writing slp files (@gitttt-1234)
#269: Bump version from 0.5.7 to 0.5.8 (@talmo)
#270: Implement lazy loading to improve import performance (@talmo)
#271: Add investigation skill for empirical experimentation (@talmo)
#272: Make imageio-ffmpeg optional and enhance backend plugin system (@talmo)
#274: Fix video ID mapping for sequential IDs with sparse dataset names (@gitttt-1234)
#275: Fix video matching to prioritize source_filename for HDF5 backends (@gitttt-1234)

Full Changelog: v0.5.7...v0.5.8

v0.5.7 ¶

sleap-io v0.5.7

🎯 Summary

This release delivers major format compatibility improvements, critical video matching fixes, and enhanced developer tooling. The v0.5.7 release adds COCO format export for seamless integration with mmpose and other COCO-compatible tools, fixes critical video matching bugs that affected multi-video projects, resolves path expansion issues in video existence checking, and modernizes coverage testing to match Codecov's branch detection capabilities.

✨ New Features

Add COCO Format Export Functionality (#260)

Implemented comprehensive COCO format export capabilities, enabling seamless integration with mmpose, CVAT, and other COCO-compatible pose estimation tools.

Key Features:

Export Functions:

encode_keypoints(): Convert numpy points to COCO keypoint format
convert_labels(): Transform Labels to COCO JSON structure
write_labels(): Save COCO JSON annotation files
save_coco(): Main API function for easy access
save_file(): Updated to auto-detect and handle COCO format

COCO Standard Compliance:

Bounding boxes: Automatically computed from visible keypoints in [x, y, width, height] format
Area field: Computed from bounding box dimensions
iscrowd field: Set to 0 for all annotations (standard requirement)
Keypoints: Flat list format [x1, y1, v1, x2, y2, v2, ...]
Skeleton edges: 1-based indexing as per COCO spec
Visibility encoding: Support for both binary (0/1) and ternary (0/1/2)

Advanced Features:

Multiple skeletons/categories support
Tracking via attributes.object_id (CVAT-compatible)
Custom image filename generation
NaN coordinate handling for unlabeled keypoints
Roundtrip conversion (read → write → read)

import sleap_io as sio

# Load SLEAP labels
labels = sio.load_slp("annotations.slp")

# Export to COCO format
sio.save_coco(labels, "annotations.json")

# Or use save_file with auto-detection
sio.save_file(labels, "annotations.json")  # Auto-detects COCO from .json extension

# COCO files are now compatible with mmpose, CVAT, and other tools

mmpose Compatibility:

This implementation was validated against mmpose's BaseCocoStyleDataset and AP10KDataset to ensure full compatibility:

✅ Required fields: All mmpose-required fields present (bbox, keypoints, area, iscrowd)
✅ Bbox format: Correct COCO format [x, y, width, height] computed from visible keypoints
✅ Skeleton indexing: 1-based edge indices as expected
✅ Validation: Handles edge cases (zero keypoints, all NaN points)
✅ Tested: Comprehensive test suite with 55 tests including mmpose-specific scenarios

Testing:

55 total tests in tests/io/test_coco.py
New tests for bbox, area, and iscrowd fields
Edge case testing (NaN points, zero keypoints)
Roundtrip conversion verification
Integration tests via main API
All tests pass with no regressions in the full test suite (415 I/O tests)

Files Modified:

sleap_io/io/coco.py: Complete COCO export implementation (+808 lines)
sleap_io/io/main.py: Integrate COCO export into save_file()
sleap_io/__init__.py: Export save_coco function
tests/io/test_coco.py: Comprehensive test suite with 55 tests

Impact: Enables seamless integration with the broader pose estimation ecosystem, allowing researchers to export SLEAP annotations to mmpose, CVAT, and other COCO-compatible tools. This significantly expands sleap-io's interoperability and supports diverse research workflows.

🐛 Bug Fixes

Fix AUTO Video Matching to Prefer Basename Over Content Matches (#261)

Resolved a critical bug where multiple videos with identical shapes would incorrectly match by content instead of basename during merge operations.

Problem: When merging predictions back into a project with multiple videos that have identical shapes (common in experimental setups), the AUTO video matcher would incorrectly match videos:

Before: Predictions for video_b.mp4 would match to video_a.mp4 (first video with same shape)
After: Predictions correctly match to video_b.mp4 (same basename)

Root Cause: The merge loop's "first match wins" behavior combined with AUTO's content fallback would break on the first content match, even when a better basename match existed later in the list.

Solution: Two-part fix:

Updated AUTO method (matching.py) to try matching in order of specificity:
- Strict path match (exact resolved paths)
- Lenient path match (basenames)
- Content match (shape + backend) - only as last resort
Added smart matching logic (labels.py) for AUTO method in merge loop:
- Collects all potential matches across all videos
- Categorizes by quality (strict path > basename > content-only)
- Picks best match instead of "first match wins"

from sleap_io import Labels

# Project with multiple videos of same shape
labels = Labels.load("project.slp")
predictions = Labels.load("predictions.slp")

# Merge now correctly matches by basename, not just content
labels.merge(predictions)  # ✅ Correctly matches video_b.mp4 → video_b.mp4

Testing:

✅ New regression test added: test_merge_auto_video_matching_with_identical_shapes
✅ All 54 matching tests pass (no regressions)
✅ All 14 merging integration tests pass (no regressions)

Files Modified:

sleap_io/io/matching.py: Updated AUTO matching priority
sleap_io/model/labels.py: Added smart matching logic for merge loop
tests/model/test_labels.py: Added regression test

Impact: Critical fix for multi-video projects with identical video dimensions, ensuring predictions are merged with the correct video files. This is essential for experimental setups where multiple videos have the same resolution and frame rate. Fixes issue #255.

Update Video.filename When VideoBackend.filename Is Expanded (#263)

Fixed a bug where Video.exists() would incorrectly return False even when the backend successfully opened a video file with an expanded path.

Problem: When opening a video with a relative path or user path (e.g., ~/video.mp4), the backend would expand it to an absolute path, but Video.filename would retain the original unexpanded path. This caused Video.exists() to check the wrong path and return False even though the video was successfully loaded.

from sleap_io import Video

# Before fix
video = Video.from_filename("~/data/video.mp4")
# Backend opens: /Users/name/data/video.mp4
# Video.filename remains: ~/data/video.mp4
video.exists()  # ❌ False (checking wrong path)

# After fix
video = Video.from_filename("~/data/video.mp4")
# Backend opens: /Users/name/data/video.mp4
# Video.filename updated to: /Users/name/data/video.mp4
video.exists()  # ✅ True (checking correct path)

Solution: Updated Video.from_filename() to synchronize Video.filename with the backend's expanded filename after opening, ensuring consistency between the video object and its backend.

Files Modified:

sleap_io/model/video.py: Update Video.filename after backend initialization
tests/model/test_video.py: Added test for path expansion consistency

Impact: Ensures Video.exists() and other path-dependent operations work correctly when using relative paths, user paths, or symlinks. Fixes issue #262.

🔧 Improvements

Improve Coverage Testing to Detect Partial Lines Matching Codecov (#264)

Modernized coverage testing workflow to detect both missed and partial lines, matching what Codecov shows in PR reviews.

Problem: The old coverage annotate approach could only detect missed lines, but not partially covered lines (executed code with missing branch coverage). This meant development tools couldn't see the same gaps that Codecov highlights in yellow during PR reviews.

Solution: Switched from coverage annotate parsing to coverage.xml parsing with branch data, enabling detection of both missed and partial coverage.

Key Changes:

1. Updated Coverage Script (scripts/cov_summary.py)

Before: Parsed .py,cover files from coverage annotate (missed lines only)
After: Parses coverage.xml with branch data (missed + partial lines)
Detects partial lines from XML condition-coverage attribute
Supports multiple output formats: text, markdown, json, gh-annotations
Can filter to PR-changed lines using gh pr diff

2. Created Coverage Skill (.claude/skills/coverage/)

Comprehensive 200+ line guide for coverage analysis and improvement
Includes bundled copy of cov_summary.py script
Step-by-step workflow with real examples
Auto-discovered by Claude Code when working on coverage tasks
Replaces old .claude/commands/coverage.md command

3. Configuration Updates

pyproject.toml: Added relative_files = true for cross-OS path stability
CI workflow: Added PR summary step showing coverage table for changed files

Example Output:

Before (text only, missed lines):

sleap_io/io/leap.py: 105,109,148

After (markdown table, missed + partial):

| File | Missed | Partial |
| --- | --- | --- |
| io/leap.py | — | 105,109,148 |
| io/coco.py | 122-124,518 | 82,84,87,121,279 |

This shows io/leap.py has full line coverage but incomplete branch coverage, while io/coco.py has both gaps.

Why This Matters:

Coverage XML includes branch information (condition-coverage="50% (1/2)"), allowing detection of:

Missed: Lines with 0 hits
Partial: Lines with hits > 0 but incomplete branch coverage

Now development tools see exactly what Codecov shows, making targeted test improvement possible.

Design Decisions:

Why XML instead of JSON?

Codecov definitely supports XML (Cobertura format)
XML is proven to work with existing CI setup
Both formats contain the same branch information

Why include script in skill?

Makes skill self-contained
Follows Claude Skills pattern for bundling resources
Agent can reference it without context pollution

Files Modified:

scripts/cov_summary.py: Rewritten to parse XML instead of annotate files (+669, -361)
.claude/skills/coverage/: New coverage analysis skill
pyproject.toml: Added relative_files = true
.github/workflows/test.yml: Added PR coverage summary step

Impact: Enables more precise test coverage improvement by showing exactly which branches aren't covered, matching Codecov's analysis. This improves development workflow and helps maintain high code quality.

💡 Why These Changes Matter

The v0.5.7 release significantly enhances sleap-io's ecosystem integration, reliability, and developer experience:

Ecosystem Expansion: COCO format export enables seamless integration with mmpose, CVAT, and the broader pose estimation ecosystem, expanding research workflows and tool compatibility
Data Integrity: Fixed video matching ensures predictions merge with correct videos in multi-video projects, preventing silent data corruption in experimental pipelines
Path Reliability: Video filename synchronization fixes path-dependent operations when using relative paths or symlinks
Developer Tooling: Improved coverage testing matches Codecov's branch detection, enabling more precise test improvement and maintaining code quality

This release demonstrates sleap-io's continued commitment to interoperability, reliability, and developer productivity for pose tracking research workflows.

📋 Changelog

#260: Add COCO format export functionality (@talmo)
#261: Fix AUTO video matching to prefer basename over content matches (@talmo)
#263: Update Video.filename accordingly if the underlying VideoBackend has its .filename expanded (@sibocw)
#264: Improve coverage testing to detect partial lines matching Codecov (@talmo)
#265: Bump version from 0.5.6 to 0.5.7 (@talmo)

Closed Issues:

#255: Video matching in AUTO mode matches by content when basename matches exist
#262: Video.exists() returns False even when backend opens video with expanded path

Full Changelog: v0.5.6...v0.5.7

v0.5.6 ¶

🎯 Summary

This release delivers significant performance improvements, new CLI capabilities, and critical bug fixes. The v0.5.6 release achieves 52.3% faster SLP file loading through strategic optimization, introduces a command-line interface for quick dataset inspection, and fixes a critical skeleton decoding bug that could cause edge/symmetry mismatches in SLP files.

⚡ Performance Improvements

Optimize SLP Loading Performance (#259)

Achieved 52.3% faster SLP file loading through two complementary optimizations: dtype caching and HDF5 direct loading. These changes eliminate redundant array operations that were being repeated millions of times during file loading.

Performance Results on 4.2M Frame File:

Load time: 142.94s → 68.20s (52.3% faster, 74.75s saved)
Per-frame: 0.034 ms → 0.016 ms (over 2x faster)

Optimizations Implemented:

1. Dtype Caching (56.61s saved, 39.6% improvement)

Problem: _get_dtype() was creating numpy structured dtype objects from scratch for every single instance, called millions of times during file loading.

Solution: Cache the dtype at the class level using cls.__dict__, handling inheritance correctly for PredictedPointsArray subclasses.

Impact: _get_dtype() time reduced from 30.38s to 2.24s (92.6% reduction)

2. HDF5 Direct Loading (18.13s saved, 12.7% improvement)

Problem: Redundant data transformations:

Get HDF5 structured array (x, y, score, visible, complete)
Extract x, y and column_stack into (N, 2) array ← 8.5s wasted
Create instance via from_array conversion ← ~10s overhead
Copy score, visible, complete fields back ← redundant

Solution: Build PointsArray structures directly from HDF5 data in one operation using a new _points_from_hdf5_data() helper function.

Impact: Eliminated column_stack overhead (8.5s) and bypassed from_array conversion (~10s)

import sleap_io as sio

# Load large SLP file - now 52% faster!
labels = sio.load_slp("large_file.slp")  # 142.94s → 68.20s

Files Modified:

sleap_io/model/instance.py: Added dtype caching to PointsArray._get_dtype() and PredictedPointsArray._get_dtype()
sleap_io/io/slp.py: Added _points_from_hdf5_data() helper and updated read_instances()

API Changes: None - These are internal optimizations with zero API impact and full backward compatibility.

Performance Breakdown:

Component	Before	After V1	After V2	Savings
Total Time	142.94s	86.33s	68.20s	-74.75s
_get_dtype	30.38s	2.24s	2.24s	-28.14s ✅
column_stack	8.50s	8.50s	0.00s	-8.50s ✅
from_array overhead	~19.00s	~19.00s	~9.00s	-10.00s ✅
Field copying	~2.00s	~2.00s	0.00s	-2.00s ✅

Optimization Methodology:

Initial Profiling - Used pyinstrument to identify _get_dtype() consuming 30.38s (21.3% of total time)
Root Cause Analysis - Identified dtype recreation and redundant data copying
Phased Implementation - Applied dtype caching first (39.6% improvement), then direct loading (additional 21.0% improvement)
Validation - Verified correctness through successful 4.2M frame load with profiling at each stage

Design Decisions:

Why Class-Level Caching?

Used cls.__dict__ to ensure each class maintains its own cached dtype
PredictedPointsArray inherits from PointsArray but has different dtype fields
Thread-safe: worst case is redundant creation of identical dtype objects

Why Direct Loading?

The Instance.__attrs_post_init__() method has an optimization check that skips conversion when passed a fully-formed PointsArray
By building PointsArray directly from HDF5, we leverage this existing fast path
Eliminates intermediate array allocations and copying

Remaining Bottlenecks (68.20s):

Instance object creation (~30s) - attrs overhead, validation
Array allocations (~20s) - memory operations
LabeledFrame construction (~7s) - creating frame objects
Miscellaneous (~11s)

Further optimization would require architectural changes (HDF5 format redesign, lazy loading, alternative object models).

Impact: Dramatically improves user experience when working with large datasets, reducing wait times by over 50% for common loading operations. Essential for interactive workflows and large-scale analyses.

✨ New Features

Initial Command-Line Interface for Dataset Inspection (#256)

Introduced a Click-based CLI command sio cat for quick read-only inspection of SLEAP labels and videos without opening Python. This is an initial implementation providing core inspection capabilities with room for future enhancements.

Key Capabilities:

Minimal text summary for .slp files (counts of videos, frames, instances, skeletons)
Detailed labeled frame inspection with --lf N option
Skeleton structure visualization with --skeleton option
Video metadata inspection support
Rich-click integration for improved help output

# Summary of labels
sio cat tests/data/slp/typical.slp

# Detailed info for labeled frame 0
sio cat --lf 0 tests/data/slp/typical.slp

# Skeleton nodes and edges
sio cat --skeleton tests/data/slp/typical.slp

# From development environment using uv
uv run -m sleap_io.io.cli cat --skeleton tests/data/slp/typical.slp
uvx --from . sio cat --lf 0 tests/data/slp/typical.slp

CLI Options:

--lf N: Show details for labeled frame N (0-based index)
--skeleton: Show skeleton node names and edges
--open-videos / --no-open-videos: Control whether to open video backends (default: no)

Example Output:

$ sio cat data.slp
File: data.slp
Type: labels
Videos: 1
Labeled frames: 150
Instances: 300
Skeletons: 1

$ sio cat --skeleton data.slp
Skeleton: mouse
Nodes: 5
  - head
  - thorax
  - abdomen
  - left_ear
  - right_ear
Edges: 4
  head → thorax
  thorax → abdomen
  head → left_ear
  head → right_ear

Design Decisions:

Read-Only, Minimal CLI

Focused on discoverability and quick inspection
Supports uv/uvx workflows without heavy dependencies
No modification operations (write operations remain in Python API)

Default No-Open-Videos

Avoids opening video backends by default for portability and CI stability
Users can opt in with --open-videos when video metadata is needed

Rich-Click Integration

Improves help UX with minimal overhead
Consistent styling and markdown support in help text

Testing: Comprehensive test suite in tests/io/test_cli.py using click.testing.CliRunner covers:

Summary output verification
Labeled frame details
Out-of-range handling
Non-label input (video files)
Skeleton printing

Impact: Enables quick dataset inspection from command line, supporting rapid prototyping and debugging workflows. Particularly useful with uv/uvx for inspecting files without setting up a full Python environment. Future enhancements will expand CLI capabilities based on community feedback.

🐛 Bug Fixes

Fix py/id Resolution Bug in SLP Skeleton Decoder (#257)

Resolved a critical bug where py/id references in edge types were treated as direct edge type values instead of references to previously defined edge types.

Problem: When a symmetry edge (EdgeType=2) was defined before a regular edge (EdgeType=1) in the metadata, edges and symmetries were swapped:

First py/reduce creates EdgeType(2) and assigns it py/id=1
Second py/reduce creates EdgeType(1) and assigns it py/id=2
Buggy behavior: py/id=1 was treated as EdgeType(1) ❌
Correct behavior: py/id=1 should resolve to EdgeType(2) ✅

This affected real .slp files where edge types were defined in non-standard order, causing incorrect skeleton structure.

Solution: Implemented single-pass processing in SkeletonSLPDecoder.decode() that:

Builds a py/id → edge_type_value mapping as py/reduce objects are encountered
Resolves py/id references by looking up the mapping
Falls back to treating py/id as direct edge type value for backward compatibility with files that don't use py/reduce

import sleap_io as sio

# Load .slp file with non-standard edge type ordering
labels = sio.load_slp("data.slp")

# Skeleton edges and symmetries now correctly decoded ✅
skeleton = labels.skeletons[0]
print(f"Edges: {len(skeleton.edges)}")       # Correct count
print(f"Symmetries: {len(skeleton.symmetries)}")  # Correct count

Files Modified:

sleap_io/io/skeleton.py: Fixed py/id resolution logic (+18, -1)
tests/io/test_skeleton_io.py: Added test for bug (+109)

Example Impact on Real File:

Before Fix:

2 edges (wrong)
3 symmetries (wrong)

After Fix:

3 edges: nose→left, nose→right, nose→tailstart ✓
1 symmetry: left↔right ✓

Testing:

✅ New test: test_slp_decoder_edge_type_pyid_resolution passes
✅ All 56 skeleton I/O tests pass
✅ All 83 SLP tests pass
✅ Real .slp file verified to load correctly with fix

Impact: Critical fix ensuring edge types are correctly decoded regardless of definition order in the metadata. Prevents skeleton structure corruption in files with non-standard edge type ordering, which could silently break downstream analysis.

💡 Why These Changes Matter

The v0.5.6 release significantly enhances sleap-io's performance, usability, and data integrity:

Dramatic Performance Gains: 52.3% faster SLP loading makes working with large datasets substantially more efficient, reducing wait times from minutes to seconds for multi-million frame files
Improved Developer Experience: New CLI enables quick dataset inspection without Python scripting, supporting rapid prototyping and debugging workflows
Data Integrity: Skeleton decoder fix prevents silent corruption of edge/symmetry definitions, ensuring accurate skeletal structure for downstream analysis
Zero Breaking Changes: All improvements maintain full backward compatibility with existing code and file formats

This release demonstrates sleap-io's continued commitment to performance optimization, developer ergonomics, and data reliability for pose tracking research workflows.

📋 Changelog

#256: A commandline interface for sleap-io (@mshooter)
#257: Fix py/id resolution bug in SLP skeleton decoder (@talmo)
#258: Bump version to 0.5.6 (@talmo)
#259: Optimize SLP loading performance (@talmo)

Full Changelog: v0.5.5...v0.5.6

v0.5.5 ¶

🎯 Summary

This release focuses on critical data integrity improvements, format compatibility enhancements, and infrastructure robustness. The v0.5.5 release fixes critical bugs in NaN coordinate handling and color channel ordering, improves cross-platform compatibility with case-insensitive file extensions, adds metadata preservation for suggestion frames, and stabilizes the dependency ecosystem with PyAV version pinning.

🐛 Bug Fixes

Fixed NaN Handling in Label Studio Writer (#254)

Resolved a critical bug where NaN coordinates in pose data caused JSON serialization failures when exporting to Label Studio format.

Problem: When writing labels with missing or occluded keypoints (represented as NaN coordinates) to Label Studio JSON format, simplejson.dump() would fail with:

ValueError: Out of range float values are not JSON compliant: np.float64(nan)

This prevented users from exporting datasets with partial annotations to Label Studio for further refinement.

Solution: Modified convert_labels() in sleap_io/io/labelstudio.py to filter out points with NaN coordinates before JSON serialization. This approach:

Prevents JSON serialization errors
Aligns with Label Studio's expected format (numeric coordinates only)
Maintains consistency with the Label Studio reader (which already skips NaN points)
Matches how other backends (Ultralytics, JABS) handle missing points

import sleap_io as sio
import numpy as np

# Create labels with NaN coordinates (occluded keypoints)
labels = sio.load_slp("data.slp")

# Export to Label Studio now works without errors
sio.save_labelstudio(labels, "output.json")  # ✅ Success
# NaN points are omitted from JSON, matching Label Studio's format

Impact: Enables seamless export of datasets with partial annotations to Label Studio, fixing issue #246 and supporting common annotation workflows where not all keypoints are visible in every frame.

Fixed RGB/BGR Color Channel Ordering in .pkg.slp Embedded Frames (#250)

Addressed critical color channel inconsistencies when embedding and decoding frames in .pkg.slp files, ensuring accurate color representation regardless of encoding/decoding plugin combinations.

Problem: When embedding frames into SLP files, the choice of image encoding plugin (OpenCV vs imageio) was hardcoded based on sys.modules, and there was no tracking of which channel order (RGB vs BGR) was used during encoding. This caused color mismatches when:

Frames were encoded with OpenCV (BGR) but decoded with imageio (RGB)
Frames were encoded with imageio (RGB) but decoded with OpenCV (BGR)
Users switched between different video backends

Solution: Implemented a comprehensive fix with multiple components:

Bumped SLP format version to 1.4 - Added channel order metadata support
Added dedicated image plugin system - Separate from video plugins, supports only "opencv" and "imageio"
Store channel order in metadata - Track whether frames were encoded as RGB or BGR
Automatic channel correction - Auto-flip channels when encoding/decoding plugins differ
User-controllable defaults - New API functions to set preferred image plugin globally

import sleap_io as sio

# Set global default for all embedding operations
sio.set_default_image_plugin("opencv")

# Or specify per-save
labels = sio.load_slp("data.slp")
sio.save_slp(labels, "output.pkg.slp", embed="all", plugin="imageio")

# Check current default
print(sio.get_default_image_plugin())  # "opencv"

# Embedded frames now maintain accurate colors regardless of:
# - Which plugin was used for encoding
# - Which plugin is used for decoding
# - Legacy files (format < 1.4) default to BGR for safety

Key Features:

✅ RGB/BGR consistency - Automatic channel flipping when needed
✅ Backwards compatible - Legacy files (format < 1.4) default to BGR
✅ User control - Plugin parameter at all API levels + global defaults
✅ Clean separation - Image plugins separate from video plugins
✅ Well documented - Complete format version history

Design Decisions:

Why separate image plugin system?

Image encoding/decoding has different requirements than video streaming
Only OpenCV and imageio support image encode/decode (PyAV doesn't)
Clearer API semantics and avoids special-casing PyAV mappings

Why store channel order instead of plugin name?

More explicit and universally understood (RGB/BGR)
Plugin implementations could change, but color order is fundamental
Enables automatic correction regardless of plugin details

Why default to BGR for old files?

Most embedded images before this change used OpenCV (BGR)
Safest backwards-compatible default
Minimizes color errors in existing workflows

Impact: Critical fix ensuring color accuracy in embedded frames, essential for computer vision applications that depend on correct color channels. Fixes a long-standing issue that could silently corrupt color information.

Fixed Case-Insensitive File Extension Handling (#247)

Resolved cross-platform compatibility issues where video files with uppercase extensions were rejected.

Problem: Video files with uppercase extensions (e.g., .MP4, .AVI, .MOV) were not recognized by sleap-io, causing ValueError: Unknown video file type errors. This was particularly problematic on Windows systems where uppercase extensions are common.

import sleap_io as sio

video = sio.load_video("video.MP4")
# ❌ ValueError: Unknown video file type: "video.MP4"

Solution: Implemented case-insensitive extension checking across the entire codebase by converting filenames to lowercase before matching against supported extensions.

import sleap_io as sio

# Now works with any case combination
video = sio.load_video("video.MP4")  # ✅ Works
video = sio.load_video("video.mp4")  # ✅ Works
video = sio.load_video("video.Mp4")  # ✅ Works

Impact: Improved cross-platform compatibility, especially for Windows users where uppercase extensions are default. Eliminates a common source of confusion and error messages.

Improved Docs Workflow Robustness and Handle Race Conditions (#253)

Fixed critical documentation deployment failures and added PR preview support.

Problem: The documentation deployment workflow was failing with:

error: failed to push some refs to 'https://github.com/talmolab/sleap-io'

This occurred when:

gh-pages branch had diverged from local state
Race conditions when PRs were merged (both PR commit and merge commit triggered builds)
No way to preview docs changes before merging

Solution: Implemented multiple robustness improvements:

Separate build and push operations - Split mike deploy from git push for better control
Retry logic with rebase - Up to 3 attempts with 2s delay for gh-pages pushes
Concurrency groups - Prevent race conditions with queue-based execution
PR preview support - PRs now deploy to dev version for preview

concurrency:
  group: docs-deployment
  cancel-in-progress: false

Features Added:

✅ Automatic retry on push failures
✅ PR documentation previews (deploy to dev version)
✅ Race condition prevention
✅ Better error recovery

Impact: More reliable CI/CD for documentation, enabling docs changes to be previewed in PRs and preventing deployment failures from blocking releases.

✨ New Features

Add Metadata Support to SuggestionFrame (#251)

Introduced a flexible metadata system for SuggestionFrame objects to preserve arbitrary metadata during I/O operations.

Feature: Added metadata dictionary attribute to SuggestionFrame (similar to Video.backend_metadata) to store metadata that isn't explicitly represented in the data model.

Primary Use Case: Preserves the group key when reading/writing SLP files, which was previously being discarded.

from sleap_io import Video, SuggestionFrame

video = Video.from_filename("video.mp4")

# Create suggestion with group metadata
suggestion = SuggestionFrame(
    video=video,
    frame_idx=42,
    metadata={"group": 1}
)

# Metadata is preserved when saving/loading
labels.suggestions.append(suggestion)
sio.save_slp(labels, "output.slp")

# Load and verify
loaded = sio.load_slp("output.slp")
print(loaded.suggestions[0].metadata["group"])  # 1 ✅

Key Capabilities:

Flexible catch-all pattern for arbitrary metadata
Follows existing Video.backend_metadata pattern for consistency
Fully backward compatible (defaults to empty dict)
Round-trip preservation in SLP format

Backward Compatibility:

SLP files without group metadata default to group=0
Existing code continues to work unchanged
Factory default (empty dict) for new instances

Benefits:

🎯 Preserves metadata that was previously being lost
🔧 Extensible pattern for future metadata additions
↔️ Maintains backward compatibility
📏 Follows established codebase patterns

Impact: Prevents data loss during I/O operations and enables preservation of workflow-specific metadata throughout the annotation pipeline.

🔧 Improvements

Reorganized Examples Documentation (#252)

Restructured the examples.md documentation to improve organization, reduce redundancy, and provide a more logical learning progression.

New Section Organization:

Basics - Fundamental operations for creating and working with labels
Format conversion - All format-related operations including NWB and YOLO
Editing labels data - Modifying existing label data structures
Exporting labels - Creating derived datasets and files
Video operations - Supporting video operations

Content Improvements:

Added "Convert to Ultralytics YOLO format" example
Consolidated NWB examples into format conversion section
Reduced redundancy between dataset splits examples
Fixed broken anchor links
Improved flow from basics → conversion → editing → exporting

Impact: Better user experience when learning sleap-io, with clearer organization and more discoverable examples.

Pin PyAV to <16.0.0 and Reorganize Dependencies (#248)

Stabilized the dependency ecosystem and modernized the package structure following PEP 621 and PEP 735 standards.

Problem: PyAV 16.0.0 release failed to upload to PyPI due to storage limitations (see PyAV Issue #2028), causing installation failures for users attempting to install the PyAV backend.

Solution:

Pinned PyAV version - Added av<16.0.0 constraint to avoid broken release
Renamed optional dependency group - Changed from [av] to [pyav] for clarity
Reorganized dependency groups following modern standards:
- Moved opencv, pyav, mat, and all to [project.optional-dependencies] (PEP 621)
- Moved dev to [dependency-groups] (PEP 735)

Installation Syntax Changes:

# Old syntax (deprecated)
pip install sleap-io[av]

# New syntax (recommended)
pip install sleap-io[pyav]    # PyAV backend
pip install sleap-io[opencv]  # OpenCV backend
pip install sleap-io[mat]     # MATLAB file support
pip install sleap-io[all]     # All optional dependencies

# Development
uv sync --all-extras          # All optional deps + dev dependencies
uv sync --group dev           # Dev dependencies only (PEP 735)

Breaking Change (Installation Only):

Users installing the PyAV backend must now use sleap-io[pyav] instead of sleap-io[av]
No changes to actual API or code imports

Design Decisions:

Why rename av to pyav?

The package name is av, so having an optional dependency group also called [av] creates ambiguity
Using [pyav] makes it clear we're referring to the PyAV library/extra

Why move dev to [dependency-groups]?

PEP 621 ([project.optional-dependencies]): For end-user installable extras
PEP 735 ([dependency-groups]): For development-time dependencies
Clearer intent and better tool support (e.g., uv)

Impact: Prevents installation failures from broken PyAV release, modernizes package structure, and improves clarity of dependency organization.

💡 Why These Changes Matter

The v0.5.5 release significantly enhances sleap-io's reliability, data integrity, and cross-platform compatibility:

Data Integrity: NaN handling and color channel fixes prevent silent data corruption and serialization errors, ensuring accurate pose data analysis
Format Compatibility: SLP format v1.4 with channel order tracking, case-insensitive extensions, and metadata preservation maintain data fidelity across platforms and workflows
Dependency Stability: PyAV pinning prevents installation issues and ensures consistent user experience
Cross-Platform Support: Case-insensitive file extensions eliminate Windows-specific errors
Metadata Preservation: SuggestionFrame metadata prevents information loss during I/O operations
Infrastructure Robustness: Improved CI/CD ensures reliable documentation and development workflows

This release demonstrates sleap-io's continued commitment to data quality, reliability, and usability across diverse research workflows and computing environments.

📋 Changelog

#247: Update file extension handling (@gitttt-1234)
#248: Pin PyAV to <16.0.0 and rename optional dependency group (@talmo)
#249: Bump version to 0.5.5 (@talmo)
#250: Fix RGB/BGR color channel ordering in .pkg.slp embedded frames (@talmo)
#251: Add metadata support to SuggestionFrame for preserving group information (@talmo)
#252: Reorganize examples documentation for better clarity (@talmo)
#253: Improve docs workflow robustness and handle race conditions (@talmo)
#254: Fix NaN handling in Label Studio writer by filtering out invalid points (@talmo)

Closed Issues:

#246: save_labelstudio does not recognize nan values

Full Changelog: v0.5.4...v0.5.5

v0.5.4 ¶

🎯 Summary

This release focuses on improving data manipulation workflows with enhanced merging capabilities and critical bug fixes. The v0.5.4 release introduces refined merge strategies for handling predictions, improved skeleton loading for standalone files, and fixes for data extraction and coordinate comparison operations that enhance the reliability of sleap-io for annotation workflows.

✨ New Features

Updated Smart Merge Strategy (#244)

Enhanced the smart merge strategy to better handle prediction conflicts by preferring newer predictions when both instances are predicted.

Previous Behavior: When merging two predicted instances, the instance with the higher score was selected.

New Behavior: When both instances are predictions, the newer prediction is always chosen, ensuring the most recent model outputs are preserved.

# Smart merge now prefers newer predictions
merged = labels1.merge(labels2, strategy="smart")
# If both sources have predictions for the same frame,
# the prediction from labels2 (newer) is kept

Impact: This change improves iterative prediction workflows where models are refined over time, ensuring the latest model outputs are retained during merging operations.

New Update Tracks Merge Strategy (#244)

Introduced a new update_tracks merge strategy for LabeledFrame.merge() that allows updating track assignments and tracking scores while preserving the original pose annotations.

Key capabilities:

Update track assignments without modifying pose coordinates
Preserve original annotations while updating tracking metadata
Useful for refining tracking results after manual review

# Update track assignments while keeping pose data
frame_merged = frame1.merge(
    frame2,
    merge_strategy="update_tracks"
)
# Pose coordinates from frame1 are preserved
# Track IDs and scores are updated from frame2

Use Cases:

Correcting track identity switches without re-annotating poses
Updating tracking confidence scores after manual verification
Refining multi-animal tracking results iteratively

🐛 Bug Fixes

Fixed Standalone Skeleton Loading (#238)

Resolved an issue where standalone skeleton JSON files using the nx_graph format would fail to load correctly.

Problem: Standalone skeleton files (like those in tmp/skeletons/) were loading with 0 nodes and 0 edges because the SkeletonDecoder only looked for links and nodes at the top level, missing the nx_graph.links and nx_graph.nodes structure.

Solution: Enhanced SkeletonDecoder to handle both formats:

nx_graph format (standalone skeleton files)
Direct format (training config embedded skeletons)

# Standalone skeleton files now load correctly
skeleton = sio.load_skeleton("mice_hc.json")[0]
print(len(skeleton.nodes))  # 5 ✅ (was 0 before)
print(len(skeleton.edges))  # 4 ✅ (was 0 before)

Impact: All skeleton files in the SLEAP skeleton repository now load correctly, ensuring seamless skeleton sharing and reuse across projects.

Fixed Labels.extract() to Copy Suggestion Frames (#243)

Fixed a bug where Labels.extract() did not copy suggestion frames, leading to incomplete datasets when exporting subsets.

Problem: When extracting a subset of labels, suggestion frames (frames recommended for annotation) were not included in the new Labels object, causing unexpected behavior during export to SLEAP projects.

Solution: Updated Labels.extract() and Labels.__getitem__() to properly copy suggestion frames associated with extracted labeled frames.

# Extract subset now includes suggestion frames
subset = labels.extract([0, 10, 20])  
# subset.suggestions now contains relevant suggestion frames

Impact: Ensures data integrity when creating training/validation splits or exporting subsets for annotation, fixing issue #242.

Fixed Duplication Handling in Labels.merge() (#240)

Addressed an edge case where frame indices were incorrectly mapped when merging videos with duplicate images present in both datasets.

Problem: When merging ImageVideo objects with overlapping duplicate frames, the frame index mapping could be incorrect, leading to misaligned annotations.

Solution: Improved the deduplication logic to correctly handle frame indices even when an image appears as a duplicate in both source datasets.

Impact: Ensures accurate frame alignment when merging datasets with complex image sequences, particularly for CVAT-exported data with duplicate frames. Fixes issue #239.

Fixed Instance.same_pose_as() with NaN Coordinates (#237)

Resolved a critical bug where identical instances containing NaN coordinates would incorrectly return False when compared.

Problem: The distance-based comparison failed for NaN values because nan <= tolerance is always False, causing identical instances to be considered different.

Solution: Introduced a two-mode comparison system:

Exact comparison (new default): tolerance=None uses np.array_equal(equal_nan=True)
Tolerance-based comparison: When tolerance is specified, validates NaN patterns match before comparing non-NaN values

from sleap_io import Instance, Skeleton
import numpy as np

skeleton = Skeleton(["head", "thorax", "tail"])

# Instances with NaN coordinates
inst1 = Instance.from_numpy([[1, 2], [np.nan, np.nan], [5, 6]], skeleton=skeleton)
inst2 = Instance.from_numpy([[1, 2], [np.nan, np.nan], [5, 6]], skeleton=skeleton)

# Exact comparison (new default behavior)
assert inst1.same_pose_as(inst2)  # ✅ True (was False before)

# Tolerance-based comparison still works
assert inst1.same_pose_as(inst2, tolerance=1.0)  # ✅ True

API Change: Default tolerance parameter changed from 5.0 to None for more intuitive exact comparison behavior. To maintain previous behavior, explicitly specify tolerance=5.0.

Impact: Ensures correct pose comparison when working with partial annotations or occluded keypoints, critical for deduplication and tracking workflows.

💡 Why These Changes Matter

The v0.5.4 release enhances sleap-io's robustness and usability for real-world annotation workflows:

Improved Merge Strategies: Better handling of predictions and tracking data enables more sophisticated human-in-the-loop workflows
Skeleton Compatibility: Fixed standalone skeleton loading ensures seamless skeleton sharing across the SLEAP ecosystem
Data Integrity: Fixed extraction and deduplication bugs prevent data loss during dataset manipulation
Reliable Comparisons: Proper NaN handling in pose comparisons ensures accurate deduplication and tracking operations

This release demonstrates sleap-io's continued focus on reliability and usability for diverse pose tracking research workflows.

📋 Changelog

#237: Fix Instance.same_pose_as() bug with NaN coordinates (@talmo)
#238: Fix standalone skeleton loading and add mice_hc test (@talmo)
#240: Improve duplication handling in Labels.merge() (@talmo)
#243: Fix Extract Function in Labels Object to Copy Suggestion Frames (@talmo)
#244: Update merge strategy (@talmo)

Full Changelog: v0.5.3...v0.5.4

v0.5.3 ¶

🎯 Summary

This release focuses on critical bug fixes and compatibility improvements that enhance the reliability and accuracy of sleap-io across different data formats and video backends. The v0.5.3 release addresses several important issues discovered by users, including coordinate system inconsistencies in legacy files, single-node skeleton loading problems, and video color channel bugs that affected data integrity.

🐛 Bug Fixes

Fixed Legacy SLP Coordinate System (#231)

Resolved a critical coordinate system inconsistency in legacy SLP files (FORMAT_ID < 1.1) that caused a 0.5 pixel offset.

Problem: Legacy SLP files used a coordinate system where the top-left corner of pixels was at (0, 0), while modern files (>=1.1) place pixel centers at (0, 0). This difference caused coordinates from legacy files to be incorrectly positioned.

Solution: Added automatic coordinate adjustment during loading for legacy files:

Detects files with FORMAT_ID < 1.1
Applies -0.5 pixel offset to convert from corner-based to center-based coordinates
Ensures consistent coordinate interpretation across all SLP file versions

# Legacy files now load with corrected coordinates
labels = sio.load_file("legacy_file.slp")
# Point that was at pixel corner (0, 0) is now at pixel center (-0.5, -0.5)

Impact: Ensures accurate pose data analysis when working with datasets created in older versions of SLEAP.

Fixed Single Node Skeleton Decoding (#233)

Resolved a critical bug where single-node skeletons with no edges would fail to load from training configuration files.

Problem: The SkeletonDecoder only processed nodes that appeared in the links section, but single-node skeletons have no edges/links, causing empty skeletons to be returned instead of the expected single node.

Solution: Enhanced the skeleton decoder to also process nodes directly defined in the nodes array, ensuring single-node skeletons are properly loaded.

Use Cases: This fix enables several important workflows:

Plant phenotyping with single-point tracking
Minimal animal tracking for small organisms
Custom pose models with single keypoints
Loading skeleton definitions from existing training configs

# Single-node skeletons now load correctly
skeleton = sio.load_skeleton("single_node_training_config.json")[0]
print(len(skeleton.nodes))  # 1 ✅ (was 0 before)

Fixed OpenCV BGR to RGB Color Conversion (#230)

Addressed two critical issues in the OpenCV video backend that affected color accuracy and reader initialization.

Issues Fixed:

Color Channel Bug: BGR to RGB conversion was happening in the wrong place, causing incorrect color representation
Reader Initialization: When keep_open=True, the OpenCV reader wasn't properly initialized, causing errors on first frame read

Impact: Ensures consistent and accurate color representation across all video backends (OpenCV, FFMPEG, PyAV), critical for computer vision applications that depend on correct color channels.

# OpenCV backend now returns correct RGB frames
video = MediaVideo("video.mp4", plugin="opencv")
frame = video.get_frame(0)  # Returns RGB frame, not BGR

Fixed NWB Backwards Compatibility and Added Append Mode (#234)

Restored broken imports from the v0.5.2 NWB API reorganization and added append functionality.

Compatibility Fix: Restored these previously broken imports:

from sleap_io.io.nwb import append_nwb_data  # ✅ Now works again
sleap_io.io.nwb.append_nwb_data             # ✅ Now works again

New Feature: Added append parameter to NWB saving functions:

# Save predictions to new file
sleap_io.save_nwb(labels, "predictions.nwb")

# Append more predictions to existing file  
sleap_io.save_nwb(more_labels, "predictions.nwb", append=True)

Impact: Fixes user workflows broken in v0.5.2 and enables incremental prediction saving for large datasets.

🔧 Improvements

Enhanced Documentation Navigation (#229, #235)

Added literate-nav plugin to mkdocs.yml for better API documentation navigation
Fixed version dropdown ordering in documentation to show versions in proper semantic order (dev → newest → oldest)
Improved user experience when browsing different versions of the documentation

💡 Why These Changes Matter

The v0.5.3 release significantly improves the reliability and accuracy of sleap-io:

Data Integrity: Fixed coordinate system and color channel bugs ensure accurate pose data analysis
Format Compatibility: Legacy SLP support maintains seamless workflows with older datasets
Minimal Pose Support: Single-node skeleton fixes enable plant phenotyping and simple tracking applications
API Stability: NWB backwards compatibility fixes prevent user workflow disruptions
Developer Experience: Better documentation navigation improves usability

This release demonstrates sleap-io's commitment to maintaining high data quality standards and supporting diverse research workflows across the pose tracking community.

📋 Changelog

#229: Add literate-nav plugin to mkdocs.yml for better API documentation navigation (@talmo)
#230: Fix OpenCV BGR to RGB conversion and reader initialization (@talmo)
#231: Fix coordinate system for legacy SLP files (FORMAT_ID < 1.1) (@talmo)
#233: Fix single node skeleton decoding (@talmo)
#234: Fix NWB backwards compatibility and add append mode support (@talmo)
#235: Fix docs version dropdown ordering (@talmo)
#236: Bump version to 0.5.3 (@talmo)

Full Changelog: v0.5.2...v0.5.3

v0.5.2 ¶

🎯 Summary

This release significantly expands sleap-io's format compatibility with three major new pose tracking format readers and important bug fixes. The v0.5.2 release adds support for LEAP MATLAB files, AlphaTracker JSON annotations, and introduces a comprehensive NWB training data I/O system with a simplified API. These additions further establish sleap-io as the universal utility for pose tracking data conversion and interoperability.

✨ New Features

LEAP .mat Format Reader (#224)

Added comprehensive support for reading LEAP (LEAP Estimates Animal Pose) MATLAB .mat files, enabling integration with the LEAP deep learning framework.

Key capabilities:

Automatic skeleton detection from nodes/joints fields
Flexible position data parsing (handles multiple array shapes and field names)
Video path inference when missing from metadata
Custom skeleton override support
MATLAB 1-based to Python 0-based index conversion

import sleap_io as sio

# Basic loading
labels = sio.load_leap("path/to/leap_data.mat")

# Auto-detection via load_file
labels = sio.load_file("path/to/leap_data.mat")

# Custom skeleton override
custom_skeleton = sio.Skeleton(
    nodes=["head", "tail"], 
    edges=[("head", "tail")]
)
labels = sio.load_leap("path/to/leap_data.mat", skeleton=custom_skeleton)

Installation: Requires the pymatreader package:

# Install with LEAP support
pip install sleap-io[mat]
# or with all extras
uv sync --all-extras

AlphaTracker Format Reader (#227)

Implemented reading support for AlphaTracker JSON annotation format, a multi-animal pose tracking system that exports annotations with bounding boxes and keypoints.

Key features:

Dynamic skeleton detection by scanning all annotations
Robust annotation grouping (Face markers + sequential keypoints)
Handles variable numbers of keypoints per animal
Graceful handling of extra annotation types
Automatic ImageVideo construction from referenced images

from sleap_io import load_file

# Auto-detect format
labels = load_file("path/to/alphatracker.json")

# Or specify explicitly
labels = load_file("path/to/alphatracker.json", format="alphatracker")

# Access the data
for lf in labels.labeled_frames:
    for instance in lf.instances:
        points = instance.points["xy"]  # Shape: (n_nodes, 2)

Comprehensive NWB Training Data I/O (#228)

Introduced a major enhancement to NWB (Neurodata Without Borders) support with a harmonization layer that unifies reading and writing of both annotations and predictions. See ndx-pose for more information on the specification.

Harmonization Layer:

Unified API: Single load_nwb() and save_nwb() functions with auto-detection
Format flexibility: Supports multiple NWB output formats via NwbFormat enum
Auto-detection: Intelligently routes to appropriate backends based on data type

Training Data I/O:

Full roundtrip conversion with complex skeleton hierarchies
Multi-skeleton support and frame provenance tracking
Video export with embedded frames using optimized MJPEG writer
Custom NWB metadata support

from sleap_io import load_nwb, save_nwb

# Universal loading (auto-detects format)
labels = load_nwb("pose_data.nwb")

# Save with auto-detection
save_nwb(labels, "output.nwb")  # Auto-detects annotation vs prediction

# Force specific format
save_nwb(labels, "annotations.nwb", nwb_format="annotations")
save_nwb(labels, "predictions.nwb", nwb_format="predictions")

# Export with embedded video frames
from sleap_io.io.nwb_annotations import export_labels

export_labels(
    labels,
    output_dir="export/",
    nwb_filename="training_with_video.nwb",
    as_training=True,
    include_videos=True
)

🐛 Bug Fixes

Fixed SLP Skeleton Format Compatibility (#222)

Resolved loading issues with newer SLEAP v1.3.2+ files that use the new "nx_graph" skeleton wrapper format
Maintains backward compatibility with legacy skeleton format
Added comprehensive test coverage with new format fixtures

Impact: Projects created with SLEAP v1.3.2 and later can now be loaded correctly without skeleton parsing errors.

💡 Why These Changes Matter

The v0.5.2 release significantly strengthens sleap-io's position as a universal pose tracking data utility:

Ecosystem Integration: Support for LEAP and AlphaTracker expands compatibility with popular pose estimation frameworks
HITL Workflows: Enhanced NWB training data I/O enables sophisticated human-in-the-loop annotation workflows
Data Provenance: Frame mapping and metadata preservation ensure traceability in complex analysis pipelines
Research Flexibility: Researchers can now seamlessly move data between SLEAP, LEAP, AlphaTracker, and NWB ecosystems
Future-Proofing: Updated format support ensures compatibility with evolving versions of partner tools

📋 Changelog

#222: Fix for SLP skeleton format variant (@talmo)
#223: Bump version to 0.5.2 (@talmo)
#224: Add LEAP .mat format reader (@talmo)
#227: Add AlphaTracker format reader (@talmo)
#228: Add NWB training data I/O with simplified API (@talmo)

Full Changelog: v0.5.1...v0.5.2

v0.5.1 ¶

🎯 Summary

This release introduces powerful merging capabilities for annotation workflows, enhanced image sequence handling, and improved developer tooling. The highlight is the new comprehensive merging system that enables human-in-the-loop (HITL) workflows and smart combination of multiple annotation sources.

✨ New Features

Comprehensive Merging System (#216)

New Labels.merge() method with configurable strategies for combining annotations from multiple sources
Smart merge strategies that preserve user labels over predictions
Instance matching with spatial, identity, and IoU-based methods
Skeleton harmonization for consistent structure across merged data
Video path resolution with automatic path repair
Progress tracking and provenance metadata for merge operations

# Merge annotations with smart conflict resolution
merged = labels1.merge(labels2, strategy="smart")

# Custom merge with specific matchers
merged = labels1.merge(
    labels2,
    video_matcher="shape",  # Match videos by dimensions
    instance_matcher="iou",  # Match instances by overlap
    conflict_strategy="user"  # Prefer user annotations
)

See the new Merging guide for more information.

Enhanced `ImageVideo` support for merging (#219)

Image deduplication with new IMAGE_DEDUP matcher
Shape-based matching for merging videos with same dimensions
CVAT format support with automatic track preservation
New Video methods:
- has_overlapping_images() - Check for duplicate frames
- matches_shape() - Compare video dimensions
- deduplicate_with() - Remove duplicate frames
- merge_with() - Combine image sequences

# Remove duplicate images from a video
video_dedup = video.deduplicate_with(other_video)

# Check if videos have the same shape
if video1.matches_shape(video2):
    merged = video1.merge_with(video2)

🐛 Bug Fixes

Fixed Duplicate Skeleton Symmetries (#217)

Resolved issue where legacy SLEAP files could create duplicate symmetry relationships
Ensures clean YAML exports without redundant symmetry definitions

🔧 Improvements

Developer Tooling (#218)

New coverage analysis script (scripts/cov_summary.py) for PR-aware coverage reporting
GitHub CLI integration for targeted coverage analysis of changed files
Simplified coverage command with line-by-line annotations

# Quick coverage check with annotations
uv run pytest -q --maxfail=1 --cov --cov-branch && uv run coverage annotate

# Get coverage for PR changes only
uv run python scripts/cov_summary.py

📚 Documentation

Improved Documentation Structure (#220)

Restructured merging documentation with detailed algorithm explanations
Enhanced examples with visual improvements using MkDocs Material
Added mermaid flowcharts and behavior matrices for better understanding
Improved organization with progressive disclosure of complex topics

💡 Why These Changes Matter

The v0.5.1 release significantly enhances sleap-io's capabilities for real-world annotation workflows:

HITL Workflows: The new merging system enables seamless integration of manual corrections with model predictions
Data Consolidation: Easily combine annotations from multiple annotators or sessions
Video Management: Better handling of image sequences with automatic deduplication
Developer Experience: Improved tooling for maintaining code quality and test coverage

📋 Changelog

#216: Implement comprehensive merging system for annotation files (@talmo)
#217: Fix duplicate skeleton symmetries from legacy SLEAP files (@talmo)
#218: Add coverage summary script and update coverage command (@talmo)
#219: Deduplicate merging for ImageVideo and add better CVAT support (@talmo)
#220: Documentation improvements for merging capabilities (@talmo)

Full Changelog: v0.5.0...v0.5.1

v0.5.0 ¶

Summary

Overview

The v0.5.0 release of sleap-io represents a major leap forward in format compatibility, development tooling, and data management capabilities. This release introduces support for five new pose tracking formats (COCO, DeepLabCut, TIFF stacks, and enhanced Ultralytics), adds powerful multi-dataset management with LabelsSet, and modernizes the development workflow with UV package manager. The release also includes critical bug fixes, performance improvements, and comprehensive documentation updates.

🚀 New Features

Multi-Dataset Management with LabelsSet (#197)

sleap-io now provides a powerful LabelsSet container for managing multiple Labels objects, enabling seamless handling of train/val/test splits and dataset collections.

This feature includes:

Hybrid dictionary/tuple interface for flexible access patterns
Automatic split creation from existing datasets
Batch I/O operations for entire dataset collections
Backward-compatible API that doesn't break existing code

Usage:

import sleap_io as sio

# Create train/val/test splits
splits = labels.make_training_splits(n_train=0.8, n_test=0.1)

# Access as dictionary
train = splits["train"]
val = splits["val"]
test = splits["test"]

# Or unpack as tuple (backward compatible)
train, val, test = splits

# Save all splits at once
splits.save("splits/", embed=True)  # Creates train.pkg.slp, val.pkg.slp, test.pkg.slp

# Load multi-split datasets
labels_set = sio.load_labels_set("path/to/splits/")

COCO-Style Dataset Support (#199)

Added comprehensive support for reading COCO pose format, enabling integration with the broader computer vision ecosystem.

Features:

Automatic skeleton creation from COCO categories
Multi-species support with different skeletons per category
Flexible directory structure handling (flat, nested, categorized)
Binary and ternary visibility encodings
Memory-efficient shared video objects for images

Usage:

import sleap_io as sio

# Load COCO annotations
labels = sio.load_file("annotations.json", format="coco")

# Load with custom image root
labels = sio.load_file("coco_data.json", dataset_root="/path/to/images")

# Load multi-split COCO dataset
labels_set = sio.load_labels_set("dataset/", format="coco")

DeepLabCut Training Data Support (#201)

Implemented complete support for reading DeepLabCut CSV files, supporting all DLC format variations.

Capabilities:

Single-animal tracking (SADLC)
Multi-animal tracking (MADLC)
Multi-animal with identity tracking (MAUDLC)
Automatic format detection from CSV headers
Proper video grouping from image directories

Usage:

import sleap_io as sio

# Load DLC annotations
labels = sio.load_file("CollectedData_scorer.csv")

# Access the data
print(f"Found {len(labels.labeled_frames)} labeled frames")
for lf in labels.labeled_frames:
    for instance in lf.instances:
        if instance.track:
            print(f"Individual '{instance.track.name}': {instance.numpy()}")

TIFF Stack Support (#195)

Added native support for multi-page TIFF files through a new TiffVideo backend.

Features:

Automatic detection of single vs multi-page TIFFs
Frame-by-frame access for TIFF stacks
Full SLP serialization support
Backward compatibility with older SLEAP versions

Usage:

import sleap_io as sio

# Load multi-page TIFF
video = sio.Video.from_filename("multipage_stack.tif")
print(f"Stack has {video.shape[0]} frames")

# Use in labels
labels = sio.Labels()
labels.videos.append(video)
labels.save("project.slp")  # TiffVideo metadata preserved

Video Plugin Management (#202)

Introduced comprehensive control over video backend selection to handle platform-specific codec issues.

New capabilities:

Global default plugin setting
Flexible plugin name aliases (case-insensitive)
Runtime plugin switching on existing videos
Batch plugin changes for entire projects

Usage:

import sleap_io as sio

# Set global default
sio.set_default_video_plugin("opencv")  # or "cv2", "cv", "ocv"

# Load with specific plugin
video = sio.load_video("video.mp4", plugin="FFMPEG")

# Switch plugin on existing video
video.set_video_plugin("pyav")  # or "av", "PyAV"

# Batch change for all videos in project
labels = sio.load_slp("project.slp")
labels.set_video_plugin("opencv")

🐛 Bug Fixes

Fixed Skeleton Node Order Decoding (#208)

Fixed incorrect node ordering when loading skeletons from training configs with non-sequential py/ids
Resolved edge connection errors that occurred with complex skeletons
Added comprehensive tests for node and edge order preservation

Impact: Skeletons loaded from training configs now correctly preserve their structure and edge connections.

🔧 Improvements

Performance: Labels.replace_filenames Control (#213)

Added open_videos parameter to Labels.replace_filenames() for better performance when working with network storage:

# Replace paths without opening videos (faster on network storage)
labels.replace_filenames(
    prefix_map={"/old/network/path": "/new/network/path"},
    open_videos=False  # Avoid costly file checks
)

Impact: Significantly faster filename replacement operations when dealing with many files on network storage.

Development Workflow: UV Migration (#214)

Migrated CI/CD and development tooling from conda to UV for dramatic performance improvements:

10-100x faster dependency resolution and installation
CI runtime reduced from ~12 minutes to ~2 minutes
Single tool for all Python/package management tasks
PyPI trusted publisher for improved security

Developer experience:

# One-line setup
uv sync --all-extras

# All commands now use uv run prefix
uv run pytest tests/
uv run ruff check sleap_io tests
uv build

Tooling: Unified Linting with Ruff (#194)

Replaced black and pydocstyle with ruff for unified, faster code quality checks:

Single tool for both formatting and linting
Significantly faster CI runs
Fixed 50+ code quality issues during migration
Consistent configuration and error reporting

Documentation Enhancements (#203, #215)

Comprehensive documentation improvements for both AI assistants and human contributors:

Added Claude Code integration with .claude/commands directory
Visual data model diagram with relationships
Modernized CONTRIBUTING.md with Quick Start section
Updated all examples to use UV commands
Streamlined testing and coverage workflows

📦 Dependencies

Added pyyaml for YAML skeleton support (from v0.4.0)
Made OpenCV and PyAV optional dependencies with modular installation
Removed hard conda dependencies in favor of pip/UV

🔄 Migration Notes

Optional Video Backend Dependencies

Video backends are now optional to reduce installation size:

# Basic installation (no video backends)
pip install sleap-io

# With specific backends
pip install sleap-io[opencv]  # OpenCV only
pip install sleap-io[av]      # PyAV only
pip install sleap-io[all]     # All backends

Development Setup

For developers, UV is now the recommended tool:

# Install UV
curl -LsSf https://astral.sh/uv/install.sh | sh

# Setup development environment
uv sync --all-extras

# Run commands with uv run prefix
uv run pytest tests/

Conda environments still work but use pip under the hood for simplicity.

No Breaking API Changes

All existing APIs remain compatible. New features are additive and do not affect existing functionality.

🎯 Why These Changes Matter

Format Ecosystem: Support for COCO and DeepLabCut enables integration with the broader pose tracking community
Dataset Management: LabelsSet provides professional-grade tools for managing train/val/test splits
Performance: UV migration and video control features dramatically improve development and runtime speed
Media Flexibility: TIFF support and plugin management handle diverse video formats and platform requirements
Developer Experience: Modern tooling and documentation reduce friction for contributors
Code Quality: Ruff migration ensures consistent, high-quality code across the project

This release significantly expands sleap-io's capabilities as a universal pose tracking data utility, improving compatibility with external tools, performance for large-scale operations, and the overall development experience.

Changelog

Replace black/pydocstyle with ruff in CI by @talmo in #194
Add TIFF support for multi-page stacks by @talmo in #195
Add LabelsSet for multi-label and split handling by @talmo in #197
Add COCO-style dataset support by @talmo in #199
Add DLC training data support by @talmo in #201
Video plugin conveniences by @talmo in #202
Fix skeleton node order decoding by @talmo in #208
Migrate CI from conda to UV by @talmo in #214
Update to 0.5.0 and refresh docs by @talmo in #203
Add open_videos parameter to Labels.replace_filenames method by @talmo in #213
Documentation improvements for Claude Code and contributors by @talmo in #215

Full Changelog: v0.4.1...v0.5.0

v0.4.1 ¶

Summary

Overview

The v0.4.1 release of sleap-io introduces experimental Ultralytics YOLO pose format support, enhances skeleton loading capabilities, and provides important improvements to video reference handling in saved files. This release focuses on expanding format compatibility and giving users more control over their workflow when working with package files and predictions.

⚠️ Important: This release includes a critical fix (#185) for skeleton decoding that affected v0.4.0. Users working with skeletons should upgrade immediately.

🚀 New Features

Ultralytics YOLO Pose Format Support (#183) [Experimental]

sleap-io now provides experimental support for reading and writing pose annotations in the Ultralytics YOLO pose format, enabling seamless integration with YOLO-based pose estimation workflows.

Note: This is an initial implementation that has not yet been battle-tested in production environments. Please report any issues or edge cases you encounter.

This feature includes:

Full support for YOLO's normalized coordinate system
Multi-instance pose annotations
Automatic skeleton configuration from data.yaml files
Integration with train/val/test splits

Usage:

import sleap_io as sio

# Load YOLO pose annotations
labels = sio.load_ultralytics(
    labels_dir="path/to/labels",
    image_dir="path/to/images", 
    data_yaml="path/to/data.yaml"
)

# Save in YOLO format with train/val/test splits
sio.save_ultralytics(
    labels,
    save_dir="yolo_dataset",
    split_fractions=(0.8, 0.1, 0.1)  # 80% train, 10% val, 10% test
)

# Also works via the generic API
labels = sio.load_file("path/to/labels/*.txt")  # Auto-detects YOLO format
labels.save("output_dir", format="ultralytics")

Video Reference Restoration Control (#192)

Added fine-grained control over video references when saving SLEAP files. This is particularly useful when working with predictions made on .pkg.slp files, allowing you to maintain references to the specific package files used for inference.

Usage:

import sleap_io as sio

# Load a package file used for training/inference
labels = sio.load_file("train.pkg.slp")

# Run inference...

# Save predictions while preserving reference to train.pkg.slp
labels.save("predictions.slp", embed=False, restore_original_videos=False)

# Default behavior still restores original video references
labels.save("predictions_default.slp")  # Links to original video files

Benefits:

Track which exact dataset split was used for predictions
Compare inference results across different models
Maintain data lineage for reproducible workflows

🐛 Bug Fixes

Critical Fix: Skeleton Decoding for Complex Skeletons (#185) 🚨

Fixed a critical bug introduced in v0.4.0 where skeletons with 32+ nodes or non-sequential py/id assignments would fail to decode
Removed hardcoded assumptions about py/id patterns that only worked for simple skeletons
Now correctly handles arbitrary py/id assignments in skeleton data

Impact: This bug affected v0.4.0 and prevented users with complex skeletons (e.g., full fly body models with 32 nodes) from loading their data. If you use complex skeletons and are on v0.4.0, upgrading to v0.4.1 is strongly recommended.

🔧 Improvements

Enhanced Skeleton Loading API (#187)

The load_skeleton() function now intelligently loads skeletons from multiple sources:

.slp files: Extract skeletons directly from SLEAP project files
Training config JSON: Automatically detect and extract embedded skeletons
Existing formats: Continue to support standalone skeleton JSON/YAML files

Usage:

import sleap_io as sio

# Load from various sources
skeleton = sio.load_skeleton("project.slp")           # From SLEAP project
skeleton = sio.load_skeleton("training_config.json")  # From training config
skeleton = sio.load_skeleton("skeleton.yaml")         # Standalone YAML
skeleton = sio.load_skeleton("skeleton.json")         # Standalone JSON

Strengthened Test Coverage (#190)

Enhanced skeleton test assertions with specific expected values
Improved test clarity and regression detection
Ensures skeleton loading behavior is consistent and reliable

📦 Dependencies

No new dependencies added in this release.

🔄 Migration Notes

Video Reference Behavior

The new restore_original_videos parameter defaults to True, maintaining existing behavior:

# These are equivalent (default behavior preserved)
labels.save("output.slp")
labels.save("output.slp", restore_original_videos=True)

# New option to preserve package file references
labels.save("output.slp", restore_original_videos=False)

No Breaking API Changes

All existing APIs remain compatible. New features are additive and do not affect existing functionality.

🎯 Why These Changes Matter

Format Interoperability: YOLO pose format support enables integration with a wider ecosystem of pose estimation tools
Workflow Control: Video reference preservation gives users control over their data lineage and prediction tracking
Reliability: Bug fixes for complex skeletons ensure sleap-io works with diverse anatomical models
Developer Experience: Enhanced skeleton loading API reduces code complexity when working with different file formats
Quality Assurance: Improved test coverage ensures long-term stability and catches regressions early

This release strengthens sleap-io's position as a versatile tool for pose tracking data management, improving both compatibility with external tools and control over complex workflows.

Changelog

Add Ultralytics YOLO pose format support by @talmo in #183
Fix skeleton decoding for non-sequential py/id assignments by @talmo in #185
Skeleton API enhancements by @talmo in #187
Strengthen skeleton test assertions by @talmo in #190
Add video reference restoration control by @talmo in #192
Bump to v0.4.1 by @talmo in #193

Full Changelog: v0.4.0...v0.4.1

v0.4.0 ¶

Summary

Overview

The v0.4.0 release of sleap-io introduces significant improvements to skeleton file handling, enhances the package file (.pkg.slp) saving performance, and fixes several important bugs. This release focuses on making skeleton data more accessible and improving the user experience when working with large datasets.

🚀 New Features

Standalone Skeleton Serialization (#178)

sleap-io now supports reading and writing skeleton files independently from label files. This feature enables:

Loading skeleton definitions from .json files in SLEAP's jsonpickle format
Saving skeletons for reuse across projects
Working with multiple skeletons in a single file

Usage:

import sleap_io as sio

# Load a skeleton
skeleton = sio.load_skeleton("skeleton.json")

# Save a skeleton  
sio.save_skeleton(skeleton, "output.json")

# Also works with lists of skeletons
skeletons = sio.load_skeleton("multiple_skeletons.json")
sio.save_skeleton(skeletons, "output.json")

YAML Skeleton Format Support (#179)

Added support for human-readable YAML format for skeleton files, making it easier to:

Manually create and edit skeleton definitions
Version control skeleton configurations
Share skeleton templates between projects

Usage:

import sleap_io as sio

# Load from YAML
skeleton = sio.load_skeleton("skeleton.yaml")

# Save to YAML  
sio.save_skeleton(skeleton, "output.yml")

Example YAML format:

Skeleton-0:
  nodes:
  - name: head
  - name: thorax
  - name: abdomen
  edges:
  - source:
      name: head
    destination:
      name: thorax
  symmetries:
  - - name: eyeL
    - name: eyeR

Frame Embedding Progress Bar (#174)

Added a progress bar when embedding frames into .pkg.slp files, providing:

Visual feedback during long embedding operations
Better user experience when working with large datasets
Estimate of remaining time for completion

🐛 Bug Fixes

Fixed Package File Saving with Embedded Videos (#177)

Changed default behavior: embed=False is now the default for fast-saving
Prevented data loss: Added detection for self-referential paths
Fixed video referencing: Correctly handles external video references

Impact: Users can now quickly save .pkg.slp files without re-embedding frames, significantly improving save performance for large projects.

Usage:

# Fast save (default behavior)
labels.save("file.slp")  # embed=False by default

# Explicitly re-embed frames
labels.save("file.slp", embed=True)

Fixed ImageVideo Backend Metadata Serialization (#173)

Fixed compatibility issue with core SLEAP when using ImageVideo backend
Resolved cattrs class inference errors
Maintained backward compatibility with existing files

Impact: Users working with image sequences no longer encounter errors when loading files in core SLEAP.

🔧 Improvements

SLP Format v1.3 Support (#176)

Added explicit support for SLEAP label format version 1.3
Enhanced tracking score support on Instance objects
Added comprehensive tests for format compatibility

Impact: Full compatibility with the latest SLEAP format features, including improved tracking metrics.

📦 Dependencies

Added pyyaml dependency for YAML skeleton format support

🔄 Migration Notes

Default Embedding Behavior Change

The default behavior for saving .pkg.slp files has changed:

Before v0.4.0: embed=True (re-embeds all frames)
After v0.4.0: embed=False (references existing files)

To maintain previous behavior, explicitly set embed=True:

labels.save("output.pkg.slp", embed=True)

No Breaking API Changes

All existing APIs remain compatible. New features are additive and do not affect existing functionality.

🎯 Why These Changes Matter

Skeleton Management: Standalone skeleton files enable better project organization and reusability
Performance: Fast-saving package files dramatically reduces save times for large datasets
User Experience: Progress bars and better error messages improve workflow transparency
Compatibility: Bug fixes ensure smooth interoperability with core SLEAP
Flexibility: YAML format provides a human-friendly alternative for skeleton configuration

This release enhances sleap-io's capabilities as a standalone utility for pose tracking data management while maintaining full compatibility with the SLEAP ecosystem.

Changelog

Fix the backend metadata being serialized to SLP for ImageVideo backends by @talmo in #173
Frame embedding progress bar by @talmo in #174
Implement SLP format v1.3 by @talmo in #176
Fix saving package files with embedded videos by @talmo in #177
Standalone Skeleton serialization/deserialization by @talmo in #178
Add YAML support for skeleton serialization by @talmo in #179
Bump to v0.4.0 by @talmo in #180

Full Changelog: v0.3.0...v0.4.0

v0.3.0 ¶

What's Changed

Add skeleton symmetry QOL enhancements by @talmo in #144
Add support for writing to nwb with ndx-pose > 0.2.0 by @h-mayorquin in #143
Check for existence of source video when creating from pkg.slp by @talmo in #148
Add Camera class by @roomrys in #145
Add CameraGroup class by @roomrys in #146
Minimize NWB testing time by @talmo in #155
Implement points array backend by @talmo in #154
Fix saving .pkg.slp with empty videos by @talmo in #156
Add all MV data structures by @roomrys in #151
Integrate recording session with labels by @roomrys in #153
Remove geometric functionality by @roomrys in #158
Refactor data handling and implement setitem for Instances by @talmo in #161
Support user instances in Labels.numpy() by @talmo in #162
Implement update_from_numpy method for instance updating from tracks array by @talmo in #163
Add Labels.from_numpy constructor by @talmo in #166
Add repr for mv classes by @roomrys in #167
Add codespell support (config, workflow to detect/not fix) and make it fix some typos by @yarikoptic in #168
Update ndx-pose dependency to version >=0.2.1 in environment.yml by @lochhh in #169
Remove OpenCV dependency for Rodrigues transformation by @talmo in #170
Bump to v0.3.0 by @talmo in #171

New Contributors

@yarikoptic made their first contribution in #168

Full Changelog: v0.2.0...v0.3.0

v0.2.0 ¶

What's Changed

Update backend filename when backend isn't created on replace by @talmo in #127
Update labels videos list on replace by @talmo in #128
Add video writing by @talmo in #129
- Add sio.VideoWriter: basic imageio-ffmpeg video writer with sensible H264 presets. This can be used as a context manager:
```
with sio.VideoWriter("video.mp4") as vw:
    for frame in video:
        vw(frame)
```
- Add sio.save_video: high-level video writing. This can be used to quickly write a set of frames or even a whole Video for easy (if inefficient) re-encoding:
```
bad_video = sio.load_video("unseekable.avi")
sio.save_video(bad_video, "seekable.mp4")
```
- Added IndexError in VideoBackend to enable sequence protocol for iteration over Videos:
```
for frame in video:
    pass
```
- Refactored sio.io.video to sio.io.video_reading.
Fixes to get JABS export to work with new data by @talmo in #132
Make skeleton nodes mutable by @talmo in #135
Add skeleton manipulation utilities by @talmo in #136
- Skeleton
  - __contains__(node: NodeOrIndex): Returns True if a node exists in the skeleton.
  - rebuild_cache(): Method allowing explicit regeneration of the caching attributes from the nodes.
  - Caching attributes are now named _name_to_node_cache and _node_to_ind_cache, better reflecting the mapping directionality.
  - require_node(node: NodeOrIndex, add_missing: bool = True): Returns a Node given a Node, int or str. If add_missing is True, the node is added or created, otherwise an IndexError is raised. This is helpful for flexibly converting between node representations with convenient existence handling.
  - add_nodes(list[Node | str]): Convenience method to add a list of nodes.
  - add_edges(edges: list[Edge | tuple[NodeOrIndex, NodeOrIndex]]): Convenience method to add a list of edges.
  - rename_nodes(name_map: dict[NodeOrIndex, str] | list[str]): Method to rename nodes either by specifying a potentially partial mapping from node(s) to new name(s), or a list of new names. Handles updating both the Node.name attributes and the cache.
  - rename_node(old_name: NodeOrIndex, new_name: str): Shorter syntax for renaming a single node.
  - remove_nodes(nodes: list[NodeOrIndex]): Method for removing nodes from the skeleton and updating caches. Does NOT update corresponding instances.
  - remove_node(node: NodeOrIndex): Shorter syntax for removing a single node.
  - reorder_nodes(new_order: list[NodeOrIndex]): Method for setting the order of the nodes within the skeleton with cache updating. Does NOT update corresponding instances.
- Instance/PredictedInstance
  - update_skeleton(): Updates the points attribute on the instance to reflect changes in the associated skeleton (removed nodes and reordering). This is called internally after updating the skeleton from the Labels level, but also exposed for more complex data manipulation workflows.
  - replace_skeleton(new_skeleton: Skeleton, node_map: dict[NodeOrIndex, NodeOrIndex] | None = None, rev_node_map: dict[NodeOrIndex, NodeOrIndex] | None = None): Method to replace the skeleton on the instance with optional capability to specify a node mapping so that data stored in the points attribute is retained and associated with the right nodes in the new skeleton. Mapping is specified in node_map from old to new nodes and defaults to mapping between node objects with the same name. rev_node_map maps new nodes to old nodes and is used internally when calling from the Labels level as it bypasses validation.
- Labels
  - instances: Convenience property that returns a generator that loops over all labeled frames and returns all instances. This can be lazily iterated over without having to construct a huge list of all the instances.
  - rename_nodes(name_map: dict[NodeOrIndex, str] | list[str], skeleton: Skeleton | None = None): Method to rename nodes in a specified skeleton within the labels.
  - remove_nodes(nodes: list[NodeOrIndex], skeleton: Skeleton | None = None): Method to remove nodes in a specified skeleton within the labels. This also updates all instances associated with the skeleton, removing point data for the removed nodes.
  - reorder_nodes(new_order: list[NodeOrIndex], skeleton: Skeleton | None = None): Method to reorder nodes in a specified skeleton within the labels. This also updates all instances associated with the skeleton, reordering point data for the nodes.
  - replace_skeleton(new_skeleton: Skeleton, old_skeleton: Skeleton | None = None, node_map: dict[NodeOrIndex, NodeOrIndex] | None = None): Method to replace a skeleton entirely within the labels, updating all instances associated with the old skeleton to use the new skeleton, optionally with node remapping to retain previous point data.
Add more checks for video seeking/reading failure by @talmo in #138
Fix HDF5Video edge cases by @talmo in #137
Docs changelog generation by @talmo in #130
Add Labels.extract, Labels.trim and Video.save by @talmo in #140
- LabeledFrame.frame_idx: Now always converted to int type.
- Video.close(): Now caches backend metadata to Video.backend_metadata to persist metadata on close.
- copy.deepcopy() now works on Video objects even if backend is open.
- Video.save(save_path: str | Path, frame_inds: list[int] | np.ndarray | None = None, video_kwargs: dict[str, Any] | None = None): Method to save a video file to an MP4 using VideoWriter with an optional subset of frames.
- Labels.extract(inds: list[int] | list[tuple[Video, int]] | np.ndarray, copy: bool = True): Add method to extract a subset of frames from the labels, optionally making a copy, and return a new Labels object.
- Labels.trim(save_path: str | Path, frame_inds: list[int] | np.ndarray, video: Video | int | None = None, video_kwargs: dict[str, Any] | None = None): Add method to extract a subset of the labels, write a video clip with the extracted friends, and adjust frame indices to match the clip.
Docs automation by @talmo in #141
Add more examples to docs by @talmo in #142

Full Changelog: v0.1.10...v0.2.0

v0.1.10 ¶

What's Changed

Fix embedded video lookup by @talmo in #122
Add better support for exporting and loading RGB videos from .pkg.slp files by @talmo in #125
Fix video indexing when embedding from labels that already have embedded data by @talmo in #126

Full Changelog: v0.1.9...v0.1.10

v0.1.9 ¶

What's Changed

Dependency management by @talmo in #118
- Drop av as a dependency since it's still a little buggy and doesn't have broad enough platform compatibility.
- Pin ndx-pose < 0.2.0 until #104 is merged in.
- Remove livecov dev tool as it was interfering with VSCode debugging.
Safer video loading from SLP by @talmo in #119
- Added sio.io.utils.is_file_accessible to check for readability by actually reading a byte. This catches permission and other esoteric filesystem errors (addresses #116).
- Explicit control over whether video files should be opened when loading labels with:sio.load_slp(..., open_videos=False)
- Explicit control over whether backend is auto-initialized when creating or using Video objects with Video(..., open_backend=False).
- More sanitization of filenames to posix/forward-slash safe forms when reading and writing SLP files.
Fix split calculation and allow for not embedding by @talmo in #120
- Fix: The function now correctly splits the labels into training, validation, and test sets based on the specified proportions (fixes #117). Previously, the validation fraction was being computed incorrectly in cases where its relative fraction was 1.0 after taking out the train split.
- Enhancement: Labels.make_training_splits(..., embed=False). Previously, the function would always embed the images, which could be slow for large projects. With this change, the embed parameter is introduced, allowing the user to choose whether to embed the images or save the labels with references to the source video files.

Full Changelog: v0.1.8...v0.1.9

v0.1.8 ¶

What's Changed

Fix docs by @talmo in #108
Support NumPy > 2.0 and Python 3.12 by @lochhh in #115

New Contributors

@lochhh made their first contribution in #115

Full Changelog: v0.1.7...v0.1.8

v0.1.7 ¶

What's Changed

Path sanitization hotfix by @talmo in #107

Full Changelog: v0.1.6...v0.1.7

v0.1.6 ¶

What's Changed

Grayscale property passthrough by @talmo in #99

Full Changelog: v0.1.5...v0.1.6

Changelog¶

v0.9.2¶

sleap-io v0.9.2 Release Notes

Summary

Installation / Upgrade

Improvements

Columnar RecordingSession storage: 3D points move out of sessions_json (#546)

Fixes

Read h5wasm flat-2D identity/category links (#548)

Documentation

Changelog

v0.9.1¶

sleap-io v0.9.1 Release Notes

Summary

Installation / Upgrade

Breaking Changes

.category is now a Category, not a str (#542)

SLP embed progress_callback gains a phase argument (#543)

New Features

First-class Category: class membership alongside Track and Identity (#542)

Improvements

Embedding writes are no longer slow or silent (#543)

ImageVideo byte-copy fast path (#543)

Progress on the write phase (#543)

Fixes

⚠️ Spilled source_video metadata made embedded packages unreadable (#543)

Documentation

Changelog

v0.9.0¶

sleap-io v0.9.0 Release Notes

Summary

Installation / Upgrade

Breaking Changes

Identity.color removed — fold color into metadata (#535)

Labels.merge() bounds merge_history to 1000 records by default (#521)

New Features

Re-identification subsystem: Identity + Embedding (#513, #514, #515, #527, #535, #536)

Frame-spanning Event annotations (#540)

Unified interconversion between detection modalities (#531)

Infer left/right symmetries from node names (#534)

sio.download() and sio download CLI (#528)

Machine-readable CLI inspection: --json (#538)

Improvements

Reliable saving for very large projects (#517, #521, #523, #524)

Faster Labels.merge() (#537)

Clearer embed errors (#530)

Fixes

Documentation

Known Issues

Changelog

v0.8.0¶

sleap-io v0.8.0 Release Notes

Summary

Installation / Upgrade

Breaking Changes

⚠️ Merge track matching now defaults to identity, not name (#449, #448)

COCO polygon segmentation now defaults to rasterized masks, not vector ROIs (#479)

Structural skeleton deduplication in update / append / extend (#447)

Re-saving .slp now trusts recorded video metadata by default (#483)

scipy<1.18 upper bound added to [mat] and [all] extras (#485, #484)

New Features

Remote loading: URLs, cloud buckets, and Google Drive (#439, #442, #441, #445)

DeepLabCut project import (#424, #450)

COCO instance-segmentation reader (#479)

Segmentation mask provenance: to_user() and from_predicted (#472, #475, #478)

Virtual on-read video cropping + apply-crops CLI (#460)

Rendering upgrades (#461, #470, #466, #468)

Improvements

Smaller .slp files (#463, #465)

Metadata-preferring .slp serialization (#483)

Path handling and helpful errors

Embedded-subset shape resolution (#476)

Mask provenance persistence and to_user() (#472, #475, #478)

CLI reaches the new readers (#496, #500, #501)

Fixes

Preflight fixes (release-hardening pass)

Post-audit follow-ups (deferred low-severity findings)

Documentation

Known Issues

Changelog

v0.9.2 ¶

Columnar `RecordingSession` storage: 3D points move out of `sessions_json` (#546)

v0.9.1 ¶

`.category` is now a `Category`, not a `str` (#542)

SLP embed `progress_callback` gains a `phase` argument (#543)

First-class `Category`: class membership alongside `Track` and `Identity` (#542)

`ImageVideo` byte-copy fast path (#543)

⚠️ Spilled `source_video` metadata made embedded packages unreadable (#543)

v0.9.0 ¶

`Identity.color` removed — fold color into `metadata` (#535)

`Labels.merge()` bounds `merge_history` to 1000 records by default (#521)

Re-identification subsystem: `Identity` + `Embedding` (#513, #514, #515, #527, #535, #536)

Frame-spanning `Event` annotations (#540)

`sio.download()` and `sio download` CLI (#528)

Machine-readable CLI inspection: `--json` (#538)

Faster `Labels.merge()` (#537)

v0.8.0 ¶

⚠️ Merge track matching now defaults to `identity`, not `name` (#449, #448)

Structural skeleton deduplication in `update` / `append` / `extend` (#447)

Re-saving `.slp` now trusts recorded video metadata by default (#483)

`scipy<1.18` upper bound added to `[mat]` and `[all]` extras (#485, #484)

Segmentation mask provenance: `to_user()` and `from_predicted` (#472, #475, #478)

Virtual on-read video cropping + `apply-crops` CLI (#460)

Smaller `.slp` files (#463, #465)

Metadata-preferring `.slp` serialization (#483)

Mask provenance persistence and `to_user()` (#472, #475, #478)

v0.7.1 ¶

Merge dropped the `is_negative` background-frame marker (#432, fixes #431)

Foreign-`Video` and filename lookups silently returned nothing (#436)

v0.7.0 ¶

Annotation type construction is now via `User` / `Predicted` subclasses (#381)

`BoundingBox` is now `x1/y1/x2/y2` (#381)

Annotations no longer carry `video`/`frame_idx` (#411)

`Labels.centroids/.bboxes/.masks/.label_images/.rois` are now read-only properties (#402)

`Labels.numpy()` and `save_analysis_h5()` shape change (#368)

`ROI.centroid` renamed to `ROI.centroid_xy` (#395)

`ROI.from_bbox()` and `ROI.from_xyxy()` deprecated (#381)

First-class `BoundingBox` (#373, #381)

First-class `LabelImage` for instance segmentation (#377, #389, #390, #392, #393, #395)

`Centroid` data model with uniform `tracking_score` (#397)

`LabeledFrame.append()` dispatcher

v0.6.5 ¶