Merging annotations¶
Merging combines annotations from multiple sources into a single dataset.
Quick start¶
import sleap_io as sio
base = sio.load_file("manual_annotations.slp")
predictions = sio.load_file("predictions.slp")
base.merge(predictions)
base.save("merged.slp")
How merging works¶
Merging proceeds in five steps:
- Match skeletons — Find corresponding skeletons by node structure
- Match videos — Identify same videos across datasets
- Match tracks — Map track identities between datasets
- Merge frames — Combine frames based on the
framestrategy - Match instances — Pair instances within overlapping frames
Preset options¶
These options are controlled via parameters to Labels.merge():
| Parameter | Controls | Options |
|---|---|---|
skeleton |
How skeletons are matched | "structure" (default), "subset", "overlap", "exact" |
video |
How videos are matched | "auto" (default), "path", "basename", "content", "shape", "image_dedup" |
track |
How tracks are matched | "identity" (default), "name" (opt-in) |
frame |
How overlapping frames are combined | "auto" (default), "replace_predictions", "keep_original", "keep_new", "keep_both", "update_tracks" |
instance |
How instances are paired within frames | "spatial" (default), "identity", "iou" |
base.merge(predictions) # All defaults
base.merge(predictions, video="auto", frame="auto") # Explicit defaults
Matching without merging¶
Sometimes you need to inspect matching results without actually merging datasets. This is useful for:
- Evaluation workflows: Aligning predictions with ground truth to compute metrics
- Debugging: Understanding why videos/skeletons aren't matching as expected
- Validation: Checking matches before committing to a merge
Use Labels.match() to build correspondence maps without modifying either dataset:
import sleap_io as sio
gt_labels = sio.load_slp("ground_truth.slp")
pred_labels = sio.load_slp("predictions.slp")
# Match predictions to ground truth (doesn't modify either)
result = gt_labels.match(pred_labels)
# Inspect results
print(result.summary())
# Videos: 2/2 matched
# Skeletons: 1/1 matched
# Tracks: 0/0 matched
# Check if all videos matched
if not result.all_videos_matched:
print("Unmatched videos:")
for video in result.unmatched_videos:
print(f" - {video.filename}")
# Iterate through matched videos
for pred_video, gt_video in result.video_map.items():
if gt_video is not None:
print(f"{pred_video.filename} -> {gt_video.filename}")
MatchResult properties¶
The MatchResult object contains:
| Property | Type | Description |
|---|---|---|
video_map |
dict[Video, Video \| None] |
Maps other's videos to self's videos |
skeleton_map |
dict[Skeleton, Skeleton \| None] |
Maps other's skeletons to self's skeletons |
track_map |
dict[Track, Track \| None] |
Maps other's tracks to self's tracks |
unmatched_videos |
list[Video] |
Videos from other with no match |
unmatched_skeletons |
list[Skeleton] |
Skeletons from other with no match |
unmatched_tracks |
list[Track] |
Tracks from other with no match |
all_videos_matched |
bool |
True if all videos matched |
n_videos_matched |
int |
Count of matched videos |
Customizing matching¶
Labels.match() accepts the same matching parameters as merge():
# Use specific video matching method
result = gt_labels.match(pred_labels, video="basename")
# Use custom matchers
from sleap_io.model.matching import VideoMatcher, VideoMatchMethod
matcher = VideoMatcher(method=VideoMatchMethod.BASENAME)
result = gt_labels.match(pred_labels, video=matcher)
Step 1: Skeleton matching¶
Before merging can proceed, skeletons from both datasets must be matched. Each skeleton in the incoming dataset is compared against skeletons in the base dataset to find correspondence.
Matching methods¶
| Method | Behavior | Use case |
|---|---|---|
"structure" |
Match if same node names, regardless of order | Default. Most common case |
"subset" |
Match if incoming skeleton nodes are a subset of base | Merging partial annotations |
"overlap" |
Match if sufficient overlap between node sets | Flexible matching with threshold |
"exact" |
Match only if nodes and edges are identical | Strict validation |
If no match is found, the skeleton is added as new to the base dataset.
String configuration¶
# Default: match by structure (same nodes, any order)
base.merge(other, skeleton="structure")
# Allow partial matches (incoming can have fewer nodes)
base.merge(other, skeleton="subset")
# Exact match required (nodes and edges must be identical)
base.merge(other, skeleton="exact")
Object configuration¶
For advanced control, use SkeletonMatcher:
from sleap_io.model.matching import SkeletonMatcher
# Overlap matching with custom threshold (70% of nodes must match)
matcher = SkeletonMatcher(method="overlap", threshold=0.7)
base.merge(other, skeleton=matcher)
Step 2: Video matching¶
Videos must match for frames to merge. If video matching fails, the video is added as new—no frames merge because there's no overlap.
Design philosophy¶
The default AUTO algorithm prioritizes avoiding false positives (matching wrong videos) over avoiding false negatives (failing to match correct videos):
| Error type | Consequence | Severity |
|---|---|---|
| False positive | Annotations merged to wrong video | Data corruption (often unrecoverable) |
| False negative | Video added as new | Safe (easily fixed, see below) |
When uncertain, AUTO adds the video as new rather than risk a wrong match.
How AUTO matching works¶
For each incoming video, AUTO runs these checks in order:
| Step | Check | Result |
|---|---|---|
| 1 | Shape incompatible (frames, H, W differ) | Reject |
| 2 | Provenance conflict (different original_video, verifiable) |
Reject |
| 3 | Same physical file (os.path.samefile) |
Match |
| 4 | Exact path string match | Match |
| 5 | Unique basename/parent suffix match | Match |
| 6 | Pose matching (identical annotations on common frames) | Match |
| 7 | Image matching (pixel similarity, if enabled) | Match |
| 8 | No match found | Add as new |
Shape is for rejection only. Same resolution does NOT imply a match—it just means the videos aren't rejected. This prevents matching unrelated videos that happen to have the same dimensions.
Shape resolves through the source chain. A derived video's effective shape is taken from the nearest video in its source_video chain that has a known shape (read from in-memory metadata — no file is opened). An embedded subset of frames therefore reports its source's full shape, so it is shape-compatible with that source and is not rejected on frame count before the definitive same-file check runs. This holds even when the deeper root shape is unknown (e.g. an embedded .pkg.slp whose deeper provenance file is not loaded). Genuine siblings that merely share a file but have no source relationship keep their own shapes and are still rejected when their frame counts differ.
Provenance conflict checking only rejects when files can be verified on disk. If neither file exists (e.g., embedded videos in .pkg.slp files), the check is skipped to allow fall-through to content-based matching.
Examples¶
Cross-platform paths with unique basenames:
Base: C:/Users/alice/data/fly.mp4
Other: /home/bob/data/fly.mp4
Result: MATCH — "fly.mp4" is unique in both sets
Ambiguous basenames, parent disambiguates:
Base: /data/exp1/fly.mp4, /data/exp2/fly.mp4
Other: /remote/exp2/fly.mp4
Result: MATCH to exp2/fly.mp4 — parent directory disambiguates
Same basename, different content:
Base: fly.mp4 (1000 frames)
Other: fly.mp4 (500 frames)
Result: NOT MATCH — shape rejection (different frame counts)
PKG.SLP predictions to external video:
Base: project.slp with /data/fly.mp4
Other: predictions.pkg.slp (embedded, original_video=/data/fly.mp4)
Result: MATCH — provenance chain links to same file
Embedded subset vs restored original (same file, fewer frames):
Base: labels.pkg.slp (embedded subset: 27 of 80 frames, source_video=/data/fly.mp4)
Other: predictions.slp (restored original /data/fly.mp4: 80 frames)
Result: MATCH — the subset resolves to its source's 80-frame shape, so it is
shape-compatible and the same-file check links them
Cross-platform embedded videos (pose matching):
Base: valence.pkg.slp (Linux, original_video=/snlkt/.../CHR/fly.mp4)
Other: stress.pkg.slp (Windows, original_video=X:/.../CHR/fly.mp4)
Result: MATCH — poses on common frames are identical
Content-based matching¶
AUTO mode includes pose-based matching as a default step. When videos have overlapping labeled frames, the matcher compares pose coordinates to identify identical videos even when file paths differ.
How pose matching works¶
- Find common frame indices between videos
- For each common frame, check if ANY instance pair has identical poses
- If enough frames match (default: 3), consider it a match
This is particularly useful for:
- Cross-platform merges (Linux ↔ Windows paths)
- Embedded videos in .pkg.slp files that can't be verified on disk
- Videos that have been moved or renamed
VideoMatcher parameters¶
| Parameter | Default | Description |
|---|---|---|
content_frames |
3 | Minimum matching frames for confident match |
compare_predictions |
"auto" |
Include predictions: "auto", True, or False |
compare_images |
False |
Enable image comparison (expensive) |
image_similarity_threshold |
0.05 | Max pixel difference (0-1 scale) |
compare_predictions modes¶
"auto"(default): Include predictions only if the video has NO user instancesTrue: Always include predictions in comparisonFalse: Only compare user-labeled instances
Image similarity threshold¶
When compare_images=True, frames are compared by mean pixel difference:
0.05(default): ~13/255 pixel difference allowed0.01: Very strict (~3/255 pixels)0.1: Lenient (~26/255 pixels)
# Enable image comparison with custom threshold
from sleap_io.model.matching import VideoMatcher
base.merge(other, video=VideoMatcher(
method="auto",
compare_images=True,
image_similarity_threshold=0.1, # More lenient
))
String configuration¶
# Default: safe AUTO cascade
base.merge(other, video="auto")
# Exact path match only
base.merge(other, video="path")
# Match by filename only (ignores directory)
base.merge(other, video="basename")
Object configuration¶
For advanced control, use VideoMatcher:
from sleap_io.model.matching import VideoMatcher
# Strict path matching (paths must be identical, no normalization)
matcher = VideoMatcher(method="path", strict=True)
base.merge(other, video=matcher)
Other video matching methods¶
| Method | Behavior | Use case |
|---|---|---|
"auto" |
Safe cascade (default) | Most situations |
"path" |
Exact path match only | Strict control |
"basename" |
Filename only, ignores directory | Cross-platform (use with caution) |
"content" |
Shape + backend type | Dangerous — matches any same-resolution video |
"shape" |
Match and merge by shape | Image list merging |
"image_dedup" |
Deduplicate image lists | Remove duplicate images |
Handling false negatives¶
If AUTO doesn't match videos that ARE the same, you have a false negative. This is safe—the video was added as new rather than corrupting data. Here's how to detect and fix it:
Step 1: Detect — Check video count after merge:
base = sio.load_file("base.slp")
other = sio.load_file("other.slp")
print(f"Before: {len(base.videos)} videos")
result = base.merge(other)
print(f"After: {len(base.videos)} videos")
# If count increased unexpectedly, videos weren't matched
for v in base.videos:
print(f" {v.filename}")
Step 2: Verify — Confirm the videos ARE the same:
# Check shapes match
video_a = base.videos[0]
video_b = base.videos[1] # The one that should have matched
print(f"Video A: {video_a.shape}") # e.g., (1000, 480, 640, 3)
print(f"Video B: {video_b.shape}")
# If files exist, check content
if video_a.exists() and video_b.exists():
# Compare first frame visually or by hash
frame_a = video_a[0]
frame_b = video_b[0]
Step 3: Fix — Use replace_filenames before merging:
# Reload and fix paths before merge
base = sio.load_file("base.slp")
other = sio.load_file("other.slp")
# Option A: Map the specific file
other.replace_filenames(filename_map={
"/remote/path/fly.mp4": "/local/path/fly.mp4"
})
# Option B: Map by prefix (for multiple videos)
other.replace_filenames(prefix_map={
"/remote/data": "/local/data"
})
# Now merge
result = base.merge(other)
print(f"Videos after fix: {len(base.videos)}") # Should match original count
Alternative: Force match with explicit matcher:
# Only use this if you're CERTAIN the videos are the same
base.merge(other, video="basename") # Match by filename only
Step 3: Track matching¶
Tracks represent identities (e.g., individual animals) that persist across frames. During merge, tracks from the incoming dataset are matched to tracks in the base dataset.
Breaking change in 0.8.0
The default track matching changed from "name" to "identity" in v0.8.0.
Independently loaded files no longer collapse tracks that merely share a name
(e.g. an arbitrary "track_0"). To restore the pre-0.8.0 behavior, pass
track="name" (Python) or --track name (CLI). This also affects
Labels.match(), TrackMatcher, and sio merge.
Matching methods¶
| Method | Behavior | Use case |
|---|---|---|
"identity" |
Match by track object identity (same Track instance). Default — correctness-first; never collapses distinct tracks by arbitrary tracker-assigned names |
Independently loaded files whose track names (e.g. "track_0") are positional/arbitrary |
"name" |
Match tracks with identical names. Opt-in for semantically meaningful names (user-assigned or identity-classification models) | Named individuals; identity-model outputs |
If no match is found, the track is added as new to the base dataset.
Asymmetry of errors
Name-based over-merge (gluing two different animals together because both
happen to be named "track_0") is harder to detect and undo than
identity-based under-merge (keeping tracks separate that you wanted
combined), which is visible and recoverable. The default therefore favors
under-merge; opt in to "name" only when track names are meaningful. When you
do use "name", merge() emits a
divergence warning if same-named
tracks appear to be distinct animals.
String configuration¶
# Default: match by track object identity (same Track instance).
base.merge(other)
base.merge(other, track="identity") # explicit default
# Opt-in: match by track name (collapses distinct same-named tracks).
base.merge(other, track="name")
Object configuration¶
For advanced control, use TrackMatcher:
from sleap_io.model.matching import TrackMatcher
# Explicit name matching
matcher = TrackMatcher(method="name")
base.merge(other, track=matcher)
Divergence warning for name-based merges¶
When matching by name (track="name"), merge() emits a UserWarning if two
same-named tracks collide on a shared (video, frame) yet their instances fail
to correspond spatially on every overlapping frame — a strong signal that
name-based matching is about to glue distinct tracks together:
UserWarning: Track 'track_0' was merged by name across labels that share video
'...', but instances on that track diverge spatially on all N overlapping
frame(s) ... name-based merging may glue distinct tracks together. Review the
merge or resolve tracks at the instance level.
The warning is diagnostic only — the merge still proceeds. It fires at most once
per colliding track, requires a spatial/IoU instance matcher (it is skipped for
instance="identity"), and never fires for the default track="identity".
Step 4: Frame strategies¶
The frame parameter controls what happens when both datasets have the same frame (same video and frame index).
auto (default)¶
The recommended strategy for human-in-the-loop workflows. Preserves user labels, updates predictions.
| Base instance | Other instance | Result |
|---|---|---|
| User label | Prediction | Keep user label |
| User label | User label | Keep base (conflict) |
| Prediction | User label | Replace with user label |
| Prediction | Prediction | Replace with newer |
Unmatched instances from other are added.
# Typical HITL workflow: merge predictions into labeled project
base.merge(predictions) # Uses auto by default
replace_predictions¶
Replace all predictions in base with predictions from other. User labels are preserved.
# Re-ran inference, want to update predictions
base.merge(new_predictions, frame="replace_predictions")
| Instance type | From base | From other |
|---|---|---|
| User label | Keep | Ignore |
| Prediction | Remove | Add |
Other frame strategies¶
| Strategy | Behavior | Use case |
|---|---|---|
"keep_original" |
Ignore other entirely for overlapping frames | Preserve base annotations |
"keep_new" |
Replace base with other for overlapping frames | Overwrite with new annotations |
"keep_both" |
Concatenate all instances (creates duplicates) | Manual deduplication later |
"update_tracks" |
Copy track assignments only, don't modify poses | Update identity labels |
# Keep only the original annotations
base.merge(other, frame="keep_original")
# Replace with new annotations
base.merge(other, frame="keep_new")
# Keep everything (may create duplicates)
base.merge(other, frame="keep_both")
# Update track assignments without changing poses
base.merge(other, frame="update_tracks")
Negative (background) frames¶
The is_negative marker (a frame explicitly labeled as a background/negative training
example) is preserved across all frame strategies. If either the base or the incoming
frame is marked negative, the merged frame stays negative — the marker is never silently
dropped.
The one exception: if the merge produces a real user pose, the negative marker is
cleared (a frame with a labeled animal is not a background frame) and the merge records
a negative_flag_conflict in MergeResult.conflicts. Predicted instances do not clear
the marker, so the predict → merge-back workflow keeps negative frames negative.
Step 5: Instance matching¶
For frame strategies that need to pair instances (auto, update_tracks), instance matching determines how instances in the base frame correspond to instances in the incoming frame.
Matching methods¶
| Method | Behavior | Use case |
|---|---|---|
"spatial" |
Match by centroid distance | Default. Position-based matching |
"identity" |
Match by track identity | Same track assignment |
"iou" |
Match by bounding box IoU | Overlap-based matching |
String configuration¶
# Default: spatial matching with 5px threshold
base.merge(other, instance="spatial")
# Match by track identity
base.merge(other, instance="identity")
# Match by bounding box overlap
base.merge(other, instance="iou")
Object configuration¶
For advanced control, use InstanceMatcher:
from sleap_io.model.matching import InstanceMatcher
# Tighter spatial matching (2px threshold)
matcher = InstanceMatcher(method="spatial", threshold=2.0)
base.merge(other, instance=matcher)
# IoU matching with 50% overlap threshold
matcher = InstanceMatcher(method="iou", threshold=0.5)
base.merge(other, instance=matcher)
Troubleshooting¶
Videos weren't matched (false negative)¶
See Handling false negatives above.
Videos matched incorrectly (false positive)¶
This shouldn't happen with AUTO matching. If it does:
- Check if videos have identical shapes AND ambiguous paths
- Use
video="path"for strict matching - Report as a bug—AUTO should be conservative
Duplicate instances after merge¶
Use auto instead of keep_both, or tighten the instance match threshold:
from sleap_io.model.matching import InstanceMatcher
base.merge(other, instance=InstanceMatcher(method="spatial", threshold=2.0))
Spatial annotation handling¶
In addition to instances, merge() handles spatial annotations nested in frames:
centroids, bounding boxes, segmentation masks, ROIs, and label images.
When frames are merged (whether creating new frames or merging into existing ones), annotations are copied along with the frame. Their track and video references are remapped using the same mappings built during skeleton, video, and track matching. This ensures annotations reference the correct objects in the merged dataset.
Annotation handling follows the same frame merge strategy used for instances:
| Strategy | Annotation behavior |
|---|---|
keep_original |
Keep self's annotations only |
keep_new |
Replace with other's annotations |
keep_both |
Keep all (deduplicated by identity) |
update_tracks |
Spatial matching, then update track assignments on matched annotations |
replace_predictions |
Keep user annotations from self, add predicted from other |
auto |
Spatial matching + full user-vs-predicted resolution cascade |
For auto and update_tracks, annotations are matched by centroid distance using the
same threshold as instance matching (default 5 pixels). Each modality is resolved
independently — centroids by (x, y), bounding boxes and ROIs by their centroid, and
masks by the centroid of their bounding box.
For segmentation masks, an explicit provenance link takes precedence over spatial
matching. If a UserSegmentationMask records (via from_predicted, set by
PredictedSegmentationMask.to_user())
that it was adopted from a PredictedSegmentationMask present in the merge, the two are
paired directly — the user correction replaces its exact source prediction regardless of
centroid distance — and spatial matching only resolves the remaining, unlinked
annotations. Other modalities do not yet carry a from_predicted link and are matched
spatially only. To list predicted masks that have not been adopted (by link or spatial
overlap), use LabeledFrame.unused_predicted_masks.
New frames (no matching frame in the target) always copy all annotations from the source, regardless of strategy.
Example: merging with annotations¶
>>> import sleap_io as sio
>>> import numpy as np
>>> skeleton = sio.Skeleton(["head", "thorax", "abdomen"])
>>> video = sio.Video("test.mp4", open_backend=False)
>>> inst1 = sio.Instance.from_numpy(
... np.array([[10, 20], [30, 40], [50, 60]]),
... skeleton=skeleton,
... )
>>> lf1 = sio.LabeledFrame(video=video, frame_idx=0, instances=[inst1])
>>> lf1.append(sio.UserBoundingBox(
... x1=5, y1=15, x2=55, y2=65,
... ))
>>> base = sio.Labels(labeled_frames=[lf1])
>>> inst2 = sio.PredictedInstance.from_numpy(
... np.array([[10, 20, 0.9], [30, 40, 0.8], [50, 60, 0.7]]),
... skeleton=skeleton,
... score=0.9,
... )
>>> lf2 = sio.LabeledFrame(video=video, frame_idx=0, instances=[inst2])
>>> lf2.append(sio.PredictedBoundingBox(
... x1=6, y1=16, x2=56, y2=66, score=0.9,
... ))
>>> base.merge(sio.Labels(labeled_frames=[lf2]))
>>> print(len(base[0].bboxes))
1
Per-modality spatial matching¶
For auto and update_tracks strategies, annotations are paired by centroid
distance. Each annotation type extracts its centroid differently:
| Modality | Centroid source |
|---|---|
| Centroids | (x, y) coordinates directly |
| Bounding boxes | centroid_xy property (box center) |
| ROIs | centroid_xy property (geometry centroid) |
| Segmentation masks | Centroid of bbox (bounding box center) |
The matching threshold is controlled by the instance parameter — the same
threshold applies to both instance matching and annotation matching:
from sleap_io.model.matching import InstanceMatcher
# Use a wider threshold (10px) for annotation matching
base.merge(other, instance=InstanceMatcher(method="spatial", threshold=10.0))
Merging label images¶
For segmentation workflows, merge_label_images()
merges label images from multiple SLP files into a single file. This is separate
from Labels.merge(), which merges
keypoint annotations.
import sleap_io as sio
merged = sio.merge_label_images(
["batch_0.slp", "batch_1.slp", "batch_2.slp"],
"all_frames.slp",
)
This is designed for parallelized segmentation pipelines where each batch of frames is processed independently (e.g., with Cellpose or StarDist) and the results need to be combined.
Key behavior:
- Zero-decompression copy: For chunked-format (v2.2) sources, compressed
chunks are copied directly via
read_direct_chunk/write_direct_chunkwith no decompression/recompression overhead. - Legacy format support: Blob-format sources (v1.8-v2.1) are transparently converted to chunked format during the merge.
- Video deduplication: Videos are deduplicated by filename across sources.
Pass an explicit
video=argument to override and assign all frames to a single video. - Track deduplication: Tracks with the same name across sources are merged into a single track.
- Dimension validation: All source files must have the same frame dimensions
(H, W). AValueErroris raised if dimensions differ.
See also
LabelImageWriter for streaming writes of
individual frames — useful for producing the per-batch SLP files that
merge_label_images() can then combine.
Merge history¶
Every merge() appends a record to
labels.provenance["merge_history"] (timestamp, source/target filenames, source
stats, strategy, sleap-io version, and result counts). This is audit metadata and
is not consumed by any logic.
Because iterative correct-and-re-merge workflows can run thousands of merges, the
history is bounded: only the most recent max_merge_history records are kept
(default DEFAULT_MERGE_HISTORY_LIMIT = 1000). Pass max_merge_history=None to
retain the full history, or a smaller integer to keep fewer records:
labels.merge(other) # keep last 1000 records (default)
labels.merge(other, max_merge_history=50) # keep last 50
labels.merge(other, max_merge_history=None) # keep everything
Provenance is stored in the .slp file in a dedicated provenance_json dataset,
so a large merge_history no longer risks exceeding HDF5's metadata limits (see
the SLP format notes).
Reference¶
Labels.merge¶
sleap_io.model.labels.Labels.merge(other, skeleton=None, video=None, track=None, identity=None, category=None, frame='auto', instance=None, validate=True, progress_callback=None, error_mode='continue', max_merge_history=1000)
¶
Merge another Labels object into this one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
other
|
Labels
|
Another Labels object to merge into this one. |
required |
skeleton
|
str | SkeletonMatcher | None
|
Skeleton matching method. Can be a string ("structure", "subset", "overlap", "exact") or a SkeletonMatcher object for advanced configuration. Default is "structure". |
None
|
video
|
str | VideoMatcher | None
|
Video matching method. Can be a string ("auto", "path", "basename", "content", "shape", "image_dedup") or a VideoMatcher object for advanced configuration. Default is "auto". |
None
|
track
|
str | TrackMatcher | None
|
Track matching method. Can be a string ("identity", "name") or a TrackMatcher object. Default is "identity", which matches tracks only by object identity (the same Track instance) and appends all other tracks as new -- a correctness-first default that never collapses distinct tracks by their (often arbitrary, tracker-assigned) names. Pass "name" to match tracks by their name attribute instead, for cases where track names are semantically meaningful (e.g. user-assigned identities or identity-classification model outputs). |
None
|
identity
|
str | IdentityMatcher | None
|
Global |
None
|
category
|
str | CategoryMatcher | None
|
Global |
None
|
frame
|
str
|
Frame merge strategy. One of "auto", "keep_original", "keep_new", "keep_both", "update_tracks", "replace_predictions". Default is "auto". |
'auto'
|
instance
|
str | InstanceMatcher | None
|
Instance matching method for spatial frame strategies. Can be a string ("spatial", "identity", "iou") or an InstanceMatcher object. Default is "spatial" with 5px tolerance. |
None
|
validate
|
bool
|
If True, validate for conflicts before merging. |
True
|
progress_callback
|
Callable | None
|
Optional callback for progress updates. Should accept (current, total, message) arguments. |
None
|
error_mode
|
str
|
How to handle errors: - "continue": Log errors but continue - "strict": Raise exception on first error - "warn": Print warnings but continue |
'continue'
|
max_merge_history
|
int | None
|
Maximum number of records to retain in
|
1000
|
Returns:
| Type | Description |
|---|---|
MergeResult
|
MergeResult object with statistics and any errors/conflicts. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If Labels is lazy-loaded. |
Notes
This method modifies the Labels object in place. The merge is designed to handle common workflows like merging predictions back into a project.
Frame-spanning events (other.events) are carried across too, with each
event's video / subject / target / type rerouted onto this object's merged
catalogs. Events are deduped by identity -- (video, start_frame,
end_frame, type name, subject, target, predicted?) -- so re-merging the
same source is idempotent (confidence scores are not part of the identity).
As a side effect, other's own event catalogs are normalized first (a
no-op unless events were appended to other post-hoc without an
intervening update()).
Provenance tracking: Each merge operation appends a record to
self.provenance["merge_history"] containing:
timestamp: ISO format timestamp of the mergesource_filename: Path from source's provenance (Noneif in-memory)target_filename: Path from target's provenance (Noneif in-memory)source_labels: Statistics about the source Labelsstrategy: The frame strategy usedsleap_io_version: Version of sleap-io that performed the mergeresult: Merge statistics (frames_merged, instances_added, conflicts)
Source code in sleap_io/model/labels.py
def merge(
self,
other: "Labels",
skeleton: "str | SkeletonMatcher | None" = None,
video: "str | VideoMatcher | None" = None,
track: "str | TrackMatcher | None" = None,
identity: "str | IdentityMatcher | None" = None,
category: "str | CategoryMatcher | None" = None,
frame: str = "auto",
instance: "str | InstanceMatcher | None" = None,
validate: bool = True,
progress_callback: Callable | None = None,
error_mode: str = "continue",
max_merge_history: int | None = DEFAULT_MERGE_HISTORY_LIMIT,
) -> "MergeResult":
"""Merge another Labels object into this one.
Args:
other: Another Labels object to merge into this one.
skeleton: Skeleton matching method. Can be a string ("structure",
"subset", "overlap", "exact") or a SkeletonMatcher object for
advanced configuration. Default is "structure".
video: Video matching method. Can be a string ("auto", "path",
"basename", "content", "shape", "image_dedup") or a VideoMatcher
object for advanced configuration. Default is "auto".
track: Track matching method. Can be a string ("identity", "name") or
a TrackMatcher object. Default is "identity", which matches tracks
only by object identity (the same Track instance) and appends all
other tracks as new -- a correctness-first default that never
collapses distinct tracks by their (often arbitrary,
tracker-assigned) names. Pass "name" to match tracks by their name
attribute instead, for cases where track names are semantically
meaningful (e.g. user-assigned identities or identity-classification
model outputs).
identity: Global `Identity` catalog matching method. Can be a string
("name") or an IdentityMatcher object. Default is "name", which
dedupes the identity catalog by `name` so the same animal across
files collapses to one canonical `Identity`. Pass an
`IdentityMatcher` with method "identity" to dedupe by object
identity instead.
category: Global `Category` catalog matching method. Can be a string
("name") or a CategoryMatcher object. Default is "name", which
dedupes the category catalog by `name` so the same class across
files collapses to one canonical `Category`. Pass a
`CategoryMatcher` with method "identity" to dedupe by object
identity instead.
frame: Frame merge strategy. One of "auto", "keep_original",
"keep_new", "keep_both", "update_tracks", "replace_predictions".
Default is "auto".
instance: Instance matching method for spatial frame strategies. Can be
a string ("spatial", "identity", "iou") or an InstanceMatcher object.
Default is "spatial" with 5px tolerance.
validate: If True, validate for conflicts before merging.
progress_callback: Optional callback for progress updates.
Should accept (current, total, message) arguments.
error_mode: How to handle errors:
- "continue": Log errors but continue
- "strict": Raise exception on first error
- "warn": Print warnings but continue
max_merge_history: Maximum number of records to retain in
``provenance["merge_history"]``. After appending this merge's
record, only the most recent ``max_merge_history`` records are
kept so provenance can't grow without bound across many merges.
Defaults to ``DEFAULT_MERGE_HISTORY_LIMIT``; pass ``None`` to keep
the full history.
Returns:
MergeResult object with statistics and any errors/conflicts.
Raises:
RuntimeError: If Labels is lazy-loaded.
Notes:
This method modifies the Labels object in place. The merge is designed to
handle common workflows like merging predictions back into a project.
Frame-spanning events (``other.events``) are carried across too, with each
event's video / subject / target / type rerouted onto this object's merged
catalogs. Events are deduped by identity -- ``(video, start_frame,
end_frame, type name, subject, target, predicted?)`` -- so re-merging the
same source is idempotent (confidence scores are not part of the identity).
As a side effect, ``other``'s own event catalogs are normalized first (a
no-op unless events were appended to ``other`` post-hoc without an
intervening ``update()``).
Provenance tracking: Each merge operation appends a record to
``self.provenance["merge_history"]`` containing:
- ``timestamp``: ISO format timestamp of the merge
- ``source_filename``: Path from source's provenance (``None`` if in-memory)
- ``target_filename``: Path from target's provenance (``None`` if in-memory)
- ``source_labels``: Statistics about the source Labels
- ``strategy``: The frame strategy used
- ``sleap_io_version``: Version of sleap-io that performed the merge
- ``result``: Merge statistics (frames_merged, instances_added, conflicts)
"""
self._check_not_lazy("merge")
# Normalize the source's own event catalogs before building the merge maps.
# ``_collect_events`` registers each event's video / subject / target / type
# into ``other``'s videos / tracks / identities / event_types. It is a no-op
# when ``other`` was built via the constructor, loaded, or saved (all of which
# already collect), and only completes catalogs for a ``Labels`` that had
# events appended post-hoc without an intervening ``update()``. Doing it here
# means event-referenced videos/tracks/identities flow through the same
# matchers as everything else (Steps 2/3/3b), so they dedupe onto ``self``'s
# equivalents instead of landing as orphan duplicate catalog entries bound to
# the wrong object.
other._collect_events()
from datetime import datetime
from pathlib import Path
import sleap_io
from sleap_io.model.matching import (
NAME_CATEGORY_MATCHER,
NAME_IDENTITY_MATCHER,
CategoryMatcher,
ConflictResolution,
ErrorMode,
IdentityMatcher,
InstanceMatcher,
InstanceMatchMethod,
MergeError,
MergeResult,
SkeletonMatcher,
SkeletonMatchMethod,
SkeletonMismatchError,
TrackMatcher,
TrackMatchMethod,
VideoMatcher,
VideoMatchMethod,
)
# Coerce string arguments to Matcher objects
if skeleton is None:
skeleton_matcher = SkeletonMatcher(method=SkeletonMatchMethod.STRUCTURE)
elif isinstance(skeleton, str):
skeleton_matcher = SkeletonMatcher(method=SkeletonMatchMethod(skeleton))
else:
skeleton_matcher = skeleton
if video is None:
video_matcher = VideoMatcher()
elif isinstance(video, str):
video_matcher = VideoMatcher(method=VideoMatchMethod(video))
else:
video_matcher = video
if track is None:
track_matcher = TrackMatcher()
elif isinstance(track, str):
track_matcher = TrackMatcher(method=TrackMatchMethod(track))
else:
track_matcher = track
if instance is None:
instance_matcher = InstanceMatcher()
elif isinstance(instance, str):
instance_matcher = InstanceMatcher(method=InstanceMatchMethod(instance))
else:
instance_matcher = instance
# Parse error mode
error_mode_enum = ErrorMode(error_mode)
# Initialize result
result = MergeResult(successful=True)
# Track merge history in provenance
if "merge_history" not in self.provenance:
self.provenance["merge_history"] = []
merge_record = {
"timestamp": datetime.now().isoformat(),
"source_filename": other.provenance.get("filename"),
"target_filename": self.provenance.get("filename"),
"source_labels": {
"n_frames": len(other.labeled_frames),
"n_videos": len(other.videos),
"n_skeletons": len(other.skeletons),
"n_tracks": len(other.tracks),
},
"strategy": frame,
"sleap_io_version": sleap_io.__version__,
}
try:
# Step 1: Match and merge skeletons
skeleton_map = {}
for other_skel in other.skeletons:
matched = False
for self_skel in self.skeletons:
if skeleton_matcher.match(self_skel, other_skel):
skeleton_map[other_skel] = self_skel
matched = True
break
if not matched:
if validate and error_mode_enum == ErrorMode.STRICT:
raise SkeletonMismatchError(
message=f"No matching skeleton found for {other_skel.name}",
details={"skeleton": other_skel},
)
elif error_mode_enum == ErrorMode.WARN:
print(f"Warning: No matching skeleton for {other_skel.name}")
# Add new skeleton if no match
self.skeletons.append(other_skel)
skeleton_map[other_skel] = other_skel
# Step 2: Match and merge videos
video_map = {}
frame_idx_map = {} # Maps (old_video, old_idx) -> (new_video, new_idx)
for other_video in other.videos:
matched = False
matched_video = None
# IMAGE_DEDUP and SHAPE need special post-match processing
if video_matcher.method in (
VideoMatchMethod.IMAGE_DEDUP,
VideoMatchMethod.SHAPE,
):
for self_video in self.videos:
if video_matcher.match(self_video, other_video):
matched_video = self_video
if video_matcher.method == VideoMatchMethod.IMAGE_DEDUP:
# Deduplicate images from other_video
deduped_video = other_video.deduplicate_with(self_video)
if deduped_video is None:
# All images were duplicates, map to existing video
video_map[other_video] = self_video
# Build frame index mapping for deduplicated frames
if isinstance(
other_video.filename, list
) and isinstance(self_video.filename, list):
other_basenames = [
Path(f).name for f in other_video.filename
]
self_basenames = [
Path(f).name for f in self_video.filename
]
for old_idx, basename in enumerate(
other_basenames
):
if basename in self_basenames:
new_idx = self_basenames.index(basename)
frame_idx_map[
(other_video, old_idx)
] = (
self_video,
new_idx,
)
else:
# Add deduplicated video as new
self.videos.append(deduped_video)
video_map[other_video] = deduped_video
# Build frame index mapping for remaining frames
if isinstance(
other_video.filename, list
) and isinstance(deduped_video.filename, list):
other_basenames = [
Path(f).name for f in other_video.filename
]
deduped_basenames = [
Path(f).name for f in deduped_video.filename
]
self_basenames = [
Path(f).name for f in self_video.filename
]
for old_idx, basename in enumerate(
other_basenames
):
if basename in deduped_basenames:
new_idx = deduped_basenames.index(
basename
)
frame_idx_map[
(other_video, old_idx)
] = (
deduped_video,
new_idx,
)
else:
# Cases where the image was a duplicate,
# present in both self and other labels
# See Issue #239.
assert basename in self_basenames, (
"Unexpected basename mismatch, \
possible file corruption."
)
new_idx = self_basenames.index(basename)
frame_idx_map[
(other_video, old_idx)
] = (
self_video,
new_idx,
)
elif video_matcher.method == VideoMatchMethod.SHAPE:
# Merge videos with same shape
merged_video = self_video.merge_with(other_video)
# Replace self_video with merged version
self_video_idx = self.videos.index(self_video)
self.videos[self_video_idx] = merged_video
video_map[other_video] = merged_video
video_map[self_video] = (
merged_video # Update mapping for self too
)
# Build frame index mapping
if isinstance(
other_video.filename, list
) and isinstance(merged_video.filename, list):
other_basenames = [
Path(f).name for f in other_video.filename
]
merged_basenames = [
Path(f).name for f in merged_video.filename
]
for old_idx, basename in enumerate(other_basenames):
if basename in merged_basenames:
new_idx = merged_basenames.index(basename)
frame_idx_map[(other_video, old_idx)] = (
merged_video,
new_idx,
)
matched = True
break
else:
# All other methods: use find_match() for the full matching cascade
matched_video = video_matcher.find_match(
other_video,
self.videos,
labels_incoming=other,
labels_base=self,
)
if matched_video is not None:
video_map[other_video] = matched_video
matched = True
if not matched:
# Add new video if no match
self.videos.append(other_video)
video_map[other_video] = other_video
# Step 3: Match and merge tracks
track_map = {}
for other_track in other.tracks:
matched = False
for self_track in self.tracks:
if track_matcher.match(self_track, other_track):
track_map[other_track] = self_track
matched = True
break
if not matched:
# Add new track if no match
self.tracks.append(other_track)
track_map[other_track] = other_track
# Warn (diagnostic only) if any name-matched track pair carries
# instances that diverge spatially on every shared frame. This does
# not alter track_map or any merge result.
self._warn_track_name_divergence(
other, video_map, track_map, track_matcher, instance_matcher
)
# Step 3b: Match and merge identities (dedupe by name).
# Mirrors track matching above: the same animal across files maps to a
# single canonical catalog object. ``identity_map`` (keyed by the source
# identity's object id) is threaded into ``_map_instance`` so per-instance
# identities point at the deduped catalog entry instead of a copy.
if isinstance(identity, IdentityMatcher):
identity_matcher = identity
elif isinstance(identity, str):
identity_matcher = IdentityMatcher(method=identity)
else:
identity_matcher = NAME_IDENTITY_MATCHER
identity_map: dict[int, Identity] = {}
for other_identity in other.identities:
matched_identity = None
for self_identity in self.identities:
if identity_matcher.match(self_identity, other_identity):
matched_identity = self_identity
break
if matched_identity is None:
# Add new identity if no match.
self.identities.append(other_identity)
matched_identity = other_identity
identity_map[id(other_identity)] = matched_identity
# Step 3b-cat: Match and merge categories (dedupe by name). Mirrors the
# identity merge: the same class across files maps to a single canonical
# catalog object. ``category_map`` (keyed by the source category's object
# id, since `Category` is ``eq=False``) is threaded into ``_map_instance``
# so per-instance categories point at the deduped catalog entry.
if isinstance(category, CategoryMatcher):
category_matcher = category
elif isinstance(category, str):
category_matcher = CategoryMatcher(method=category)
else:
category_matcher = NAME_CATEGORY_MATCHER
category_map: dict[int, Category] = {}
for other_category in other.categories:
matched_category = None
for self_category in self.categories:
if category_matcher.match(self_category, other_category):
matched_category = self_category
break
if matched_category is None:
# Add new category if no match.
self.categories.append(other_category)
matched_category = other_category
category_map[id(other_category)] = matched_category
# Step 3c: Match and merge event types (dedupe by name). Mirrors the
# identity merge: the same event type across files collapses to one
# canonical catalog entry. ``event_type_map`` (keyed by the source
# type's object id) reroutes each incoming event's ``type`` onto the
# canonical entry in Step 5b.
event_type_map: dict[int, EventType] = {}
for other_event_type in other.event_types:
matched_event_type = None
for self_event_type in self.event_types:
if self_event_type.matches(other_event_type):
matched_event_type = self_event_type
break
if matched_event_type is None:
self.event_types.append(other_event_type)
matched_event_type = other_event_type
event_type_map[id(other_event_type)] = matched_event_type
# Step 4: Merge frames
total_frames = len(other.labeled_frames)
for frame_idx, other_frame in enumerate(other.labeled_frames):
if progress_callback:
progress_callback(
frame_idx,
total_frames,
f"Merging frame {frame_idx + 1}/{total_frames}",
)
# Check if frame index needs remapping (for deduplicated/merged videos)
if (other_frame.video, other_frame.frame_idx) in frame_idx_map:
mapped_video, mapped_frame_idx = frame_idx_map[
(other_frame.video, other_frame.frame_idx)
]
else:
# Map video to self
mapped_video = video_map.get(other_frame.video, other_frame.video)
mapped_frame_idx = other_frame.frame_idx
# Find matching frame in self
matching_frames = self.find(mapped_video, mapped_frame_idx)
if len(matching_frames) == 0:
# No matching frame, create new one. Preserve the negative
# (background) marker from the incoming frame verbatim.
new_frame = LabeledFrame(
video=mapped_video,
frame_idx=mapped_frame_idx,
instances=[],
is_negative=other_frame.is_negative,
)
# Map instances to new skeleton/track
instance_memo: dict[int, Instance | PredictedInstance] = {}
for inst in other_frame.instances:
new_inst = self._map_instance(
inst,
skeleton_map,
track_map,
identity_map=identity_map,
category_map=category_map,
memo=instance_memo,
)
new_frame.instances.append(new_inst)
result.instances_added += 1
# Repair ``from_predicted`` links to the remapped source.
_relink_from_predicted(new_frame.instances, instance_memo)
# Copy annotations from other frame and remap references
new_frame._merge_annotations(other_frame)
self._remap_frame_annotations(new_frame, video_map, track_map)
self._append_indexed(new_frame)
result.frames_merged += 1
else:
# Merge into existing frame
self_frame = matching_frames[0]
# Capture is_negative before merge() resolves it in place.
self_was_negative = self_frame.is_negative
# Merge instances using frame-level merge
merged_instances, conflicts = self_frame.merge(
other_frame,
instance=instance_matcher,
frame=frame,
)
# Remap skeleton and track references for instances from other frame
remapped_instances = []
instance_memo = {}
for inst in merged_instances:
# Check if instance needs remapping (from other_frame)
if inst.skeleton in skeleton_map:
# Instance needs remapping
remapped_inst = self._map_instance(
inst,
skeleton_map,
track_map,
identity_map=identity_map,
category_map=category_map,
memo=instance_memo,
)
remapped_instances.append(remapped_inst)
else:
# Instance already has correct skeleton (from self_frame)
remapped_instances.append(inst)
# Repair ``from_predicted`` links so a remapped user instance
# references the remapped source prediction in this frame.
_relink_from_predicted(remapped_instances, instance_memo)
merged_instances = remapped_instances
# Count changes
n_before = len(self_frame.instances)
n_after = len(merged_instances)
result.instances_added += max(0, n_after - n_before)
# Record conflicts
for orig, new, resolution in conflicts:
result.conflicts.append(
ConflictResolution(
frame=self_frame,
conflict_type="instance_conflict",
original_data=orig,
new_data=new,
resolution=resolution,
)
)
# Record a conflict if a negative (background) marker was
# dropped because the merge produced a user pose.
_, negative_conflict = _resolve_merged_is_negative(
self_was_negative, other_frame.is_negative, merged_instances
)
if negative_conflict:
result.conflicts.append(
ConflictResolution(
frame=self_frame,
conflict_type="negative_flag_conflict",
original_data=self_was_negative,
new_data=other_frame.is_negative,
resolution="dropped_for_user_pose",
)
)
# Update frame instances
self_frame.instances = merged_instances
# Remap annotation references (merge already copied them)
self._remap_frame_annotations(self_frame, video_map, track_map)
result.frames_merged += 1
# Step 5: Merge suggestions
for other_suggestion in other.suggestions:
mapped_video = video_map.get(
other_suggestion.video, other_suggestion.video
)
# Check if suggestion already exists
exists = False
for self_suggestion in self.suggestions:
if (
self_suggestion.video == mapped_video
and self_suggestion.frame_idx == other_suggestion.frame_idx
):
exists = True
break
if not exists:
# Create new suggestion with mapped video
new_suggestion = SuggestionFrame(
video=mapped_video, frame_idx=other_suggestion.frame_idx
)
self.suggestions.append(new_suggestion)
# Step 5b: Merge events. Each incoming event is deep-copied with its
# references rerouted onto this object's merged catalogs via a shared
# ``deepcopy`` memo: video (through ``video_map``), subject/target
# ``Track``s (``track_map``) and ``Identity``s (``identity_map``), and
# ``type`` (``event_type_map``). ``other._collect_events()`` at the top of
# merge guarantees every event reference is in ``other``'s catalogs and so
# in the memo, remapped onto ``self``'s canonical objects.
#
# Events have no per-frame slot to merge into, but they do carry a natural
# identity -- (video, start_frame, end_frame, type name, subject, target,
# predicted?) -- so the merge is idempotent: an incoming event whose
# identity already exists on ``self`` is skipped (mirroring the
# SuggestionFrame dedup in Step 5). Confidence scores are deliberately not
# part of the identity, so an exact re-merge keeps the first copy.
if other.events:
event_memo: dict[int, Any] = {}
for other_video_obj, mapped in video_map.items():
event_memo[id(other_video_obj)] = mapped
for other_track_obj, mapped in track_map.items():
event_memo[id(other_track_obj)] = mapped
event_memo.update(identity_map)
event_memo.update(event_type_map)
def _event_identity(ev: Event) -> tuple:
# Keyed on the remapped (canonical) video/participant objects, so
# object identity is a valid comparison across self + incoming.
return (
id(ev.video),
ev.start_frame,
ev.end_frame,
ev.type.name if ev.type is not None else None,
id(ev.subject),
id(ev.target),
ev.is_predicted,
)
existing_keys = {_event_identity(ev) for ev in self.events}
for other_event in other.events:
new_event = deepcopy(other_event, event_memo)
key = _event_identity(new_event)
if key in existing_keys:
continue
existing_keys.add(key)
self.events.append(new_event)
# Canonicalize any references that fell outside the memo.
self._collect_events()
# Update merge record
merge_record["result"] = {
"frames_merged": result.frames_merged,
"instances_added": result.instances_added,
"conflicts": len(result.conflicts),
}
self.provenance["merge_history"].append(merge_record)
# Bound merge_history so provenance can't grow without limit; keep the
# most recent ``max_merge_history`` records (all of them if None).
if max_merge_history is not None:
history = self.provenance["merge_history"]
if len(history) > max_merge_history:
del history[: len(history) - max_merge_history]
except MergeError as e:
result.successful = False
result.errors.append(e)
if error_mode_enum == ErrorMode.STRICT:
raise
except Exception as e:
result.successful = False
result.errors.append(
MergeError(message=str(e), details={"exception": type(e).__name__})
)
if error_mode_enum == ErrorMode.STRICT:
raise
if progress_callback:
progress_callback(total_frames, total_frames, "Merge complete")
return result
Labels.match¶
sleap_io.model.labels.Labels.match(other, video=None, skeleton=None, track=None)
¶
Match videos, skeletons, and tracks between this Labels and another.
This method builds correspondence maps without modifying either Labels object. Useful for evaluation workflows where you need to align predictions with ground truth without merging them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
other
|
Labels
|
Another Labels object to match against. |
required |
video
|
str | VideoMatcher | None
|
Video matching method. Can be a string ("auto", "path", "basename", "content", "shape", "image_dedup") or a VideoMatcher object for advanced configuration. Default is "auto". |
None
|
skeleton
|
str | SkeletonMatcher | None
|
Skeleton matching method. Can be a string ("structure", "subset", "overlap", "exact") or a SkeletonMatcher object. Default is "structure". |
None
|
track
|
str | TrackMatcher | None
|
Track matching method. Can be a string ("identity", "name") or a TrackMatcher object. Default is "identity", which matches tracks only by object identity (the same Track instance) and appends all other tracks as new -- a correctness-first default that never collapses distinct tracks by their (often arbitrary, tracker-assigned) names. Pass "name" to match tracks by their name attribute instead, for cases where track names are semantically meaningful (e.g. user-assigned identities or identity-classification model outputs). |
None
|
Returns:
| Type | Description |
|---|---|
MatchResult
|
MatchResult object containing correspondence maps. |
Example
Match prediction videos to ground truth for evaluation::
>>> gt_labels = sio.load_slp("ground_truth.slp")
>>> pred_labels = sio.load_slp("predictions.slp")
>>> result = gt_labels.match(pred_labels)
>>> for pred_video, gt_video in result.video_map.items():
... if gt_video is not None:
... print(f"{pred_video.filename} -> {gt_video.filename}")
Check if all videos were matched::
>>> if not result.all_videos_matched:
... print(f"Warning: {len(result.unmatched_videos)} unmatched")
Notes
For video matching with the AUTO method (default), the matching cascade uses multiple strategies in order:
- Shape rejection (filter obviously incompatible candidates)
- original_video conflict rejection
- Definitive file identity (is_same_file)
- Strict path match
- Leaf uniqueness matching at increasing depths
- Pose-based matching (compares annotations between labels)
The match result maps other's items to self's items. For eval
workflows, typically self is ground truth and other is predictions.
Source code in sleap_io/model/labels.py
def match(
self,
other: "Labels",
video: "str | VideoMatcher | None" = None,
skeleton: "str | SkeletonMatcher | None" = None,
track: "str | TrackMatcher | None" = None,
) -> "MatchResult":
"""Match videos, skeletons, and tracks between this Labels and another.
This method builds correspondence maps without modifying either Labels object.
Useful for evaluation workflows where you need to align predictions with
ground truth without merging them.
Args:
other: Another Labels object to match against.
video: Video matching method. Can be a string ("auto", "path",
"basename", "content", "shape", "image_dedup") or a VideoMatcher
object for advanced configuration. Default is "auto".
skeleton: Skeleton matching method. Can be a string ("structure",
"subset", "overlap", "exact") or a SkeletonMatcher object.
Default is "structure".
track: Track matching method. Can be a string ("identity", "name") or
a TrackMatcher object. Default is "identity", which matches tracks
only by object identity (the same Track instance) and appends all
other tracks as new -- a correctness-first default that never
collapses distinct tracks by their (often arbitrary,
tracker-assigned) names. Pass "name" to match tracks by their name
attribute instead, for cases where track names are semantically
meaningful (e.g. user-assigned identities or identity-classification
model outputs).
Returns:
MatchResult object containing correspondence maps.
Example:
Match prediction videos to ground truth for evaluation::
>>> gt_labels = sio.load_slp("ground_truth.slp")
>>> pred_labels = sio.load_slp("predictions.slp")
>>> result = gt_labels.match(pred_labels)
>>> for pred_video, gt_video in result.video_map.items():
... if gt_video is not None:
... print(f"{pred_video.filename} -> {gt_video.filename}")
Check if all videos were matched::
>>> if not result.all_videos_matched:
... print(f"Warning: {len(result.unmatched_videos)} unmatched")
Notes:
For video matching with the AUTO method (default), the matching cascade
uses multiple strategies in order:
1. Shape rejection (filter obviously incompatible candidates)
2. original_video conflict rejection
3. Definitive file identity (is_same_file)
4. Strict path match
5. Leaf uniqueness matching at increasing depths
6. Pose-based matching (compares annotations between labels)
The match result maps `other`'s items to `self`'s items. For eval
workflows, typically `self` is ground truth and `other` is predictions.
"""
from sleap_io.model.matching import (
MatchResult,
SkeletonMatcher,
SkeletonMatchMethod,
TrackMatcher,
TrackMatchMethod,
VideoMatcher,
VideoMatchMethod,
)
# Coerce string arguments to Matcher objects
if skeleton is None:
skeleton_matcher = SkeletonMatcher(method=SkeletonMatchMethod.STRUCTURE)
elif isinstance(skeleton, str):
skeleton_matcher = SkeletonMatcher(method=SkeletonMatchMethod(skeleton))
else:
skeleton_matcher = skeleton
if video is None:
video_matcher = VideoMatcher()
elif isinstance(video, str):
video_matcher = VideoMatcher(method=VideoMatchMethod(video))
else:
video_matcher = video
if track is None:
track_matcher = TrackMatcher()
elif isinstance(track, str):
track_matcher = TrackMatcher(method=TrackMatchMethod(track))
else:
track_matcher = track
# Initialize result
result = MatchResult()
# Match skeletons
for other_skel in other.skeletons:
matched_skel = None
for self_skel in self.skeletons:
if skeleton_matcher.match(self_skel, other_skel):
matched_skel = self_skel
break
result.skeleton_map[other_skel] = matched_skel
# Match videos
# Use find_match for AUTO method to get full matching cascade
for other_video in other.videos:
if video_matcher.method == VideoMatchMethod.AUTO:
matched_video = video_matcher.find_match(
other_video,
self.videos,
labels_incoming=other,
labels_base=self,
)
else:
matched_video = None
for self_video in self.videos:
if video_matcher.match(self_video, other_video):
matched_video = self_video
break
result.video_map[other_video] = matched_video
# Match tracks
for other_track in other.tracks:
matched_track = None
for self_track in self.tracks:
if track_matcher.match(self_track, other_track):
matched_track = self_track
break
result.track_map[other_track] = matched_track
return result
Labels.add_video¶
sleap_io.model.labels.Labels.add_video(video)
¶
Add a video to the labels, preventing duplicates.
This method provides safe video addition by checking if a video with the same file identity already exists. Unlike direct list append, this prevents duplicate videos even when different Video objects point to the same underlying file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
video
|
Video
|
The video to add. |
required |
Returns:
| Type | Description |
|---|---|
Video
|
The video that should be used. If a duplicate was detected, returns the existing video; otherwise returns the input video. |
Notes
This method uses is_same_file() for duplicate detection, which: - Considers source_video for embedded videos (PKG.SLP) - Uses strict path comparison (same basename in different dirs != same) - Handles ImageVideo lists correctly
Use this instead of labels.videos.append(video) to prevent duplicates.
Source code in sleap_io/model/labels.py
def add_video(self, video: Video) -> Video:
"""Add a video to the labels, preventing duplicates.
This method provides safe video addition by checking if a video with
the same file identity already exists. Unlike direct list append, this
prevents duplicate videos even when different Video objects point to
the same underlying file.
Args:
video: The video to add.
Returns:
The video that should be used. If a duplicate was detected, returns
the existing video; otherwise returns the input video.
Notes:
This method uses is_same_file() for duplicate detection, which:
- Considers source_video for embedded videos (PKG.SLP)
- Uses strict path comparison (same basename in different dirs != same)
- Handles ImageVideo lists correctly
Use this instead of `labels.videos.append(video)` to prevent duplicates.
"""
from sleap_io.model.matching import is_same_file
for existing in self.videos:
if is_same_file(existing, video):
return existing
self.videos.append(video)
return video
Labels.replace_filenames¶
sleap_io.model.labels.Labels.replace_filenames(new_filenames=None, filename_map=None, prefix_map=None, open_videos=True)
¶
Replace video filenames.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
new_filenames
|
list[str | Path] | None
|
List of new filenames. Must have the same length as the number of videos in the labels. |
None
|
filename_map
|
dict[str | Path, str | Path] | None
|
Dictionary mapping old filenames (keys) to new filenames (values). |
None
|
prefix_map
|
dict[str | Path, str | Path] | None
|
Dictionary mapping old prefixes (keys) to new prefixes (values). |
None
|
open_videos
|
bool
|
If |
True
|
Notes
Only one of the argument types can be provided.
Source code in sleap_io/model/labels.py
def replace_filenames(
self,
new_filenames: list[str | Path] | None = None,
filename_map: dict[str | Path, str | Path] | None = None,
prefix_map: dict[str | Path, str | Path] | None = None,
open_videos: bool = True,
):
"""Replace video filenames.
Args:
new_filenames: List of new filenames. Must have the same length as the
number of videos in the labels.
filename_map: Dictionary mapping old filenames (keys) to new filenames
(values).
prefix_map: Dictionary mapping old prefixes (keys) to new prefixes (values).
open_videos: If `True` (the default), attempt to open the video backend for
I/O after replacing the filename. If `False`, the backend will not be
opened (useful for operations with costly file existence checks).
Notes:
Only one of the argument types can be provided.
"""
n = 0
if new_filenames is not None:
n += 1
if filename_map is not None:
n += 1
if prefix_map is not None:
n += 1
if n != 1:
raise ValueError(
"Exactly one input method must be provided to replace filenames."
)
if new_filenames is not None:
if len(self.videos) != len(new_filenames):
raise ValueError(
f"Number of new filenames ({len(new_filenames)}) does not match "
f"the number of videos ({len(self.videos)})."
)
for video, new_filename in zip(self.videos, new_filenames):
video.replace_filename(new_filename, open=open_videos)
elif filename_map is not None:
for video in self.videos:
for old_fn, new_fn in filename_map.items():
if type(video.filename) is list:
new_fns = []
for fn in video.filename:
if Path(fn) == Path(old_fn):
new_fns.append(new_fn)
else:
new_fns.append(fn)
video.replace_filename(new_fns, open=open_videos)
else:
if Path(video.filename) == Path(old_fn):
video.replace_filename(new_fn, open=open_videos)
elif prefix_map is not None:
for video in self.videos:
for old_prefix, new_prefix in prefix_map.items():
# Sanitize old_prefix for cross-platform matching
old_prefix_sanitized = sanitize_filename(old_prefix)
# Check if old prefix ends with a separator
old_ends_with_sep = old_prefix_sanitized.endswith("/")
if type(video.filename) is list:
new_fns = []
for fn in video.filename:
# Sanitize filename for matching
fn_sanitized = sanitize_filename(fn)
if fn_sanitized.startswith(old_prefix_sanitized):
# Calculate the remainder after removing the prefix
remainder = fn_sanitized[len(old_prefix_sanitized) :]
# Build the new filename
if remainder.startswith("/"):
# Remainder has separator, remove it to avoid double
# slash
remainder = remainder[1:]
# Always add separator between prefix and remainder
if new_prefix and not new_prefix.endswith(
("/", "\\")
):
new_fn = new_prefix + "/" + remainder
else:
new_fn = new_prefix + remainder
elif old_ends_with_sep:
# Old prefix had separator, preserve it in the new
# one
if new_prefix and not new_prefix.endswith(
("/", "\\")
):
new_fn = new_prefix + "/" + remainder
else:
new_fn = new_prefix + remainder
else:
# No separator in old prefix, don't add one
new_fn = new_prefix + remainder
new_fns.append(new_fn)
else:
new_fns.append(fn)
video.replace_filename(new_fns, open=open_videos)
else:
# Sanitize filename for matching
fn_sanitized = sanitize_filename(video.filename)
if fn_sanitized.startswith(old_prefix_sanitized):
# Calculate the remainder after removing the prefix
remainder = fn_sanitized[len(old_prefix_sanitized) :]
# Build the new filename
if remainder.startswith("/"):
# Remainder has separator, remove it to avoid double
# slash
remainder = remainder[1:]
# Always add separator between prefix and remainder
if new_prefix and not new_prefix.endswith(("/", "\\")):
new_fn = new_prefix + "/" + remainder
else:
new_fn = new_prefix + remainder
elif old_ends_with_sep:
# Old prefix had separator, preserve it in the new one
if new_prefix and not new_prefix.endswith(("/", "\\")):
new_fn = new_prefix + "/" + remainder
else:
new_fn = new_prefix + remainder
else:
# No separator in old prefix, don't add one
new_fn = new_prefix + remainder
video.replace_filename(new_fn, open=open_videos)
FrameStrategy¶
sleap_io.model.matching.FrameStrategy
¶
Bases: builtins.str, enum.Enum
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
Source code in sleap_io/model/matching.py
class FrameStrategy(str, Enum):
"""Strategies for handling frame merging.
Attributes:
AUTO: Automatic merging that preserves user labels over predictions when
they overlap.
KEEP_ORIGINAL: Always keep instances from the original (base) frame.
KEEP_NEW: Always keep instances from the new (incoming) frame.
KEEP_BOTH: Keep all instances from both frames without filtering.
UPDATE_TRACKS: Update track assignments only without modifying poses.
REPLACE_PREDICTIONS: Keep user instances from base, remove base predictions,
add only predictions from incoming frame.
"""
AUTO = "auto"
KEEP_ORIGINAL = "keep_original"
KEEP_NEW = "keep_new"
KEEP_BOTH = "keep_both"
UPDATE_TRACKS = "update_tracks"
REPLACE_PREDICTIONS = "replace_predictions"
AUTO = <FrameStrategy.AUTO: 'auto'>
class-attribute
¶
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
KEEP_BOTH = <FrameStrategy.KEEP_BOTH: 'keep_both'>
class-attribute
¶
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
KEEP_NEW = <FrameStrategy.KEEP_NEW: 'keep_new'>
class-attribute
¶
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
KEEP_ORIGINAL = <FrameStrategy.KEEP_ORIGINAL: 'keep_original'>
class-attribute
¶
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
REPLACE_PREDICTIONS = <FrameStrategy.REPLACE_PREDICTIONS: 'replace_predictions'>
class-attribute
¶
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
UPDATE_TRACKS = <FrameStrategy.UPDATE_TRACKS: 'update_tracks'>
class-attribute
¶
Strategies for handling frame merging.
Attributes:
| Name | Type | Description |
|---|---|---|
AUTO |
Automatic merging that preserves user labels over predictions when they overlap. |
|
KEEP_ORIGINAL |
Always keep instances from the original (base) frame. |
|
KEEP_NEW |
Always keep instances from the new (incoming) frame. |
|
KEEP_BOTH |
Keep all instances from both frames without filtering. |
|
UPDATE_TRACKS |
Update track assignments only without modifying poses. |
|
REPLACE_PREDICTIONS |
Keep user instances from base, remove base predictions, add only predictions from incoming frame. |
__doc__ = 'Strategies for handling frame merging.\n\nAttributes:\n AUTO: Automatic merging that preserves user labels over predictions when\n they overlap.\n KEEP_ORIGINAL: Always keep instances from the original (base) frame.\n KEEP_NEW: Always keep instances from the new (incoming) frame.\n KEEP_BOTH: Keep all instances from both frames without filtering.\n UPDATE_TRACKS: Update track assignments only without modifying poses.\n REPLACE_PREDICTIONS: Keep user instances from base, remove base predictions,\n add only predictions from incoming frame.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
VideoMatcher¶
sleap_io.model.matching.VideoMatcher
¶
Matcher for comparing and matching videos.
Attributes:
| Name | Type | Description |
|---|---|---|
method |
The matching method to use. Can be a VideoMatchMethod enum value or a string that will be converted to the enum. Default is AUTO. |
|
strict |
Whether to use strict path matching for the PATH method. When True, paths must be exactly identical. When False, paths are normalized before comparison. Only used when method is PATH. Default is False. |
|
content_frames |
Minimum number of matching frames required for pose/image matching to confirm a match. If fewer common frames exist, requires all of them to match. Default 3. |
|
compare_predictions |
Whether to include predicted instances in pose matching. "auto" (default): Include only if video has 100% predictions (no user instances). True: Always include predictions. False: Never include predictions (user instances only). |
|
compare_images |
Whether to compare frame images via pixel similarity. Expensive operation requiring frame decoding. Default False. |
|
image_similarity_threshold |
Maximum mean pixel difference (0-1 scale, normalized by 255) for images to be considered matching. Only used when compare_images=True. Default 0.05 (~13/255 pixels). |
Notes
For AUTO method, use find_match() when matching against a list of candidates. The match() method for AUTO uses a simplified pairwise check that doesn't include the full leaf-uniqueness algorithm.
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class VideoMatcher. |
__init__ |
Method generated by attrs for class VideoMatcher. |
__repr__ |
Method generated by attrs for class VideoMatcher. |
__setattr__ |
Method generated by attrs for class VideoMatcher. |
find_match |
Find a matching video from candidates using the configured method. |
match |
Check if two videos match according to the configured method. |
Source code in sleap_io/model/matching.py
@attrs.define
class VideoMatcher:
"""Matcher for comparing and matching videos.
Attributes:
method: The matching method to use. Can be a VideoMatchMethod enum value
or a string that will be converted to the enum. Default is AUTO.
strict: Whether to use strict path matching for the PATH method.
When True, paths must be exactly identical. When False, paths
are normalized before comparison. Only used when method is PATH.
Default is False.
content_frames: Minimum number of matching frames required for pose/image
matching to confirm a match. If fewer common frames exist, requires
all of them to match. Default 3.
compare_predictions: Whether to include predicted instances in pose matching.
"auto" (default): Include only if video has 100% predictions (no user
instances). True: Always include predictions. False: Never include
predictions (user instances only).
compare_images: Whether to compare frame images via pixel similarity.
Expensive operation requiring frame decoding. Default False.
image_similarity_threshold: Maximum mean pixel difference (0-1 scale,
normalized by 255) for images to be considered matching.
Only used when compare_images=True. Default 0.05 (~13/255 pixels).
Notes:
For AUTO method, use find_match() when matching against a list of
candidates. The match() method for AUTO uses a simplified pairwise
check that doesn't include the full leaf-uniqueness algorithm.
"""
method: VideoMatchMethod | str = attrs.field(
default=VideoMatchMethod.AUTO,
converter=lambda x: VideoMatchMethod(x) if isinstance(x, str) else x,
)
strict: bool = False
content_frames: int = 3
compare_predictions: str | bool = "auto"
compare_images: bool = False
image_similarity_threshold: float = 0.05
_frame_cache: dict = attrs.field(factory=dict, init=False, repr=False)
def _get_cached_frame_instances(
self,
labels: "Labels",
video: "Video",
include_predictions: bool,
) -> dict[int, list["Instance"]]:
"""Get frame instances with caching for performance.
Caches the result to avoid recomputing for the same video multiple times
during merge operations.
"""
cache_key = (id(labels), id(video), include_predictions)
if cache_key not in self._frame_cache:
self._frame_cache[cache_key] = _get_frame_instances(
labels, video, include_predictions
)
return self._frame_cache[cache_key]
def match(self, video1: Video, video2: Video) -> bool:
"""Check if two videos match according to the configured method.
For AUTO method, this performs pairwise checks (file identity, path match).
For full AUTO matching with leaf-uniqueness, use find_match() instead.
"""
if self.method == VideoMatchMethod.AUTO:
# Pairwise AUTO: rejection checks + definitive identity + path match
# (Leaf-uniqueness requires full candidate list - use find_match())
# Rejection: incompatible shapes
if shapes_compatible(video1, video2) is False:
return False
# Rejection: conflicting provenance
if original_videos_conflict(video1, video2):
return False
# Definitive: same source file but different crop (mosaic tiles).
# Must run before any path rung, which would otherwise re-match the
# shared root file. For non-crop videos this is always False.
if _same_file_different_crop(video1, video2):
return False
# Definitive: same file identity (crop-aware)
if is_same_file(video1, video2):
return True
# String: strict path match
if video1.matches_path(video2, strict=True):
return True
# String: basename match (for pairwise, this is the fallback)
if video1.matches_path(video2, strict=False):
return True
return False
elif self.method == VideoMatchMethod.PATH:
return video1.matches_path(video2, strict=self.strict)
elif self.method == VideoMatchMethod.BASENAME:
return video1.matches_path(video2, strict=False)
elif self.method == VideoMatchMethod.CONTENT:
return video1.matches_content(video2)
elif self.method == VideoMatchMethod.IMAGE_DEDUP:
# Match ImageVideo instances with overlapping images (ImageVideo only)
return video1.has_overlapping_images(video2)
elif self.method == VideoMatchMethod.SHAPE:
# Match videos by shape only (height, width, channels)
return video1.matches_shape(video2)
else:
raise ValueError(f"Unknown video match method: {self.method}")
def find_match(
self,
incoming: Video,
candidates: list[Video],
labels_incoming: "Labels | None" = None,
labels_base: "Labels | None" = None,
) -> Video | None:
"""Find a matching video from candidates using the configured method.
This is the preferred method for AUTO matching as it implements the
full safe matching cascade including leaf-uniqueness disambiguation.
Args:
incoming: The video to find a match for.
candidates: List of existing videos to search for matches.
labels_incoming: Labels object containing the incoming video's
annotations. Used for pose-based matching in AUTO mode.
labels_base: Labels object containing the candidates' annotations.
Used for pose-based matching in AUTO mode.
Returns:
The matched video, or None if no match found.
Notes:
For AUTO method, implements the safe matching cascade:
1. Shape rejection (filter candidates)
2. original_video conflict rejection (filter candidates)
3. Definitive file identity (is_same_file)
4. Strict path match
5. Leaf uniqueness matching at increasing depths
6. Pose-based matching (if labels provided)
7. Image-based matching (if compare_images=True)
Shape is for REJECTION only - compatible shapes don't imply a match.
"""
from pathlib import Path
from sleap_io.io.utils import sanitize_filename
if self.method == VideoMatchMethod.AUTO:
# Build list of viable candidates (not rejected by shape/provenance)
viable = []
for candidate in candidates:
# REJECTION CHECK 1: Shape compatibility
shape_compat = shapes_compatible(candidate, incoming)
if shape_compat is False:
# Definitely incompatible shapes - skip
continue
# REJECTION CHECK 2: original_video conflict
if original_videos_conflict(candidate, incoming):
# Both have provenance pointing to different files - skip
continue
# REJECTION CHECK 3: same source file, different crop.
# Distinct crops (mosaic tiles) of one physical file share a
# root file, so dropping them here prevents the file-identity,
# strict-path, and leaf-uniqueness rungs from collapsing them.
# For non-crop candidates this is always False.
if _same_file_different_crop(candidate, incoming):
continue
viable.append(candidate)
# DEFINITIVE CHECK: File identity (handles source_video chains)
for candidate in viable:
if is_same_file(candidate, incoming):
return candidate
# STRING CHECK: Full path match
for candidate in viable:
if candidate.matches_path(incoming, strict=True):
return candidate
# STRING CHECK: Leaf path uniqueness
# Match paths by comparing suffixes at increasing depths
if viable:
def get_path_parts(video: Video) -> tuple[str, ...]:
"""Get path parts for comparison, using root video for embedded."""
root = _get_root_video(video)
fn = root.filename
if isinstance(fn, list):
fn = fn[0] # Use first for ImageVideo
return Path(sanitize_filename(fn)).parts
incoming_parts = get_path_parts(incoming)
candidate_parts = [(v, get_path_parts(v)) for v in viable]
# Also need all existing videos for uniqueness check
all_existing_parts = [(v, get_path_parts(v)) for v in candidates]
# Compare at increasing depths until we find a unique match
max_depth = max(
len(incoming_parts),
max((len(p) for _, p in all_existing_parts), default=0),
)
for depth in range(1, max_depth + 1):
if len(incoming_parts) < depth:
continue
incoming_leaf = "/".join(incoming_parts[-depth:])
# Find all viable candidates that match at this depth
matches_at_depth = []
for candidate, parts in candidate_parts:
if len(parts) < depth:
continue
candidate_leaf = "/".join(parts[-depth:])
if candidate_leaf == incoming_leaf:
matches_at_depth.append(candidate)
# If exactly one match at this depth, use it
if len(matches_at_depth) == 1:
return matches_at_depth[0]
# If no matches, try deeper
# If multiple matches, continue deeper to disambiguate
# POSE MATCHING: Compare pose annotations (default in AUTO)
if labels_incoming is not None and labels_base is not None:
match = self._match_by_poses(
incoming, viable, labels_incoming, labels_base
)
if match is not None:
return match
# IMAGE MATCHING: Compare frame images (opt-in)
if self.compare_images:
match = self._match_by_images(incoming, viable)
if match is not None:
return match
# No match found
return None
else:
# Non-AUTO methods: use pairwise match()
for candidate in candidates:
if self.match(candidate, incoming):
return candidate
return None
def _match_by_poses(
self,
incoming: "Video",
candidates: list["Video"],
labels_incoming: "Labels",
labels_base: "Labels",
) -> "Video | None":
"""Try to match video by comparing pose annotations.
Returns matched video if poses match on enough common frames.
"""
# Resolve whether to include predictions for incoming video
include_preds = _resolve_compare_predictions(
self.compare_predictions, labels_incoming, incoming
)
# Get incoming video's frame -> instances map (cached)
incoming_frames = self._get_cached_frame_instances(
labels_incoming, incoming, include_preds
)
if not incoming_frames:
return None # No annotations to compare
for candidate in candidates:
# Get candidate's frame -> instances map (cached for performance)
# Use same prediction setting resolved for candidate
include_preds_cand = _resolve_compare_predictions(
self.compare_predictions, labels_base, candidate
)
candidate_frames = self._get_cached_frame_instances(
labels_base, candidate, include_preds_cand
)
if not candidate_frames:
continue
# Find common frame indices
common_indices = set(incoming_frames.keys()) & set(candidate_frames.keys())
if not common_indices:
continue
# Determine required matches
required_matches = min(self.content_frames, len(common_indices))
# Sample frames if too many (performance)
sample_indices = _sample_frame_indices(
common_indices, max_samples=self.content_frames * 2
)
# Count matching frames
matching_frames = 0
for frame_idx in sample_indices:
if _frame_has_matching_pose(
incoming_frames[frame_idx], candidate_frames[frame_idx]
):
matching_frames += 1
if matching_frames >= required_matches:
return candidate # Found match!
return None
def _match_by_images(
self,
incoming: "Video",
candidates: list["Video"],
) -> "Video | None":
"""Try to match video by comparing image content.
Only used when compare_images=True. Expensive operation.
Returns matched video if images match on enough common frames.
"""
for candidate in candidates:
# Get common embedded frame indices
common_indices = _get_common_embedded_indices(incoming, candidate)
if not common_indices:
continue
required_matches = min(self.content_frames, len(common_indices))
# Sample frames
sample_indices = _sample_frame_indices(
common_indices, max_samples=self.content_frames * 2
)
# Count matching frames
matching_frames = 0
for frame_idx in sample_indices:
if _frames_similar_by_image(
incoming, candidate, frame_idx, self.image_similarity_threshold
):
matching_frames += 1
if matching_frames >= required_matches:
return candidate
return None
__annotations__ = {'method': 'VideoMatchMethod | str', 'strict': 'bool', 'content_frames': 'int', 'compare_predictions': 'str | bool', 'compare_images': 'bool', 'image_similarity_threshold': 'float', '_frame_cache': 'dict'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = True
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Matcher for comparing and matching videos.\n\nAttributes:\n method: The matching method to use. Can be a VideoMatchMethod enum value\n or a string that will be converted to the enum. Default is AUTO.\n strict: Whether to use strict path matching for the PATH method.\n When True, paths must be exactly identical. When False, paths\n are normalized before comparison. Only used when method is PATH.\n Default is False.\n content_frames: Minimum number of matching frames required for pose/image\n matching to confirm a match. If fewer common frames exist, requires\n all of them to match. Default 3.\n compare_predictions: Whether to include predicted instances in pose matching.\n "auto" (default): Include only if video has 100% predictions (no user\n instances). True: Always include predictions. False: Never include\n predictions (user instances only).\n compare_images: Whether to compare frame images via pixel similarity.\n Expensive operation requiring frame decoding. Default False.\n image_similarity_threshold: Maximum mean pixel difference (0-1 scale,\n normalized by 255) for images to be considered matching.\n Only used when compare_images=True. Default 0.05 (~13/255 pixels).\n\nNotes:\n For AUTO method, use find_match() when matching against a list of\n candidates. The match() method for AUTO uses a simplified pairwise\n check that doesn\'t include the full leaf-uniqueness algorithm.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 946
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('method', 'strict', 'content_frames', 'compare_predictions', 'compare_images', 'image_similarity_threshold')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('method', 'strict', 'content_frames', 'compare_predictions', 'compare_images', 'image_similarity_threshold', '_frame_cache', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
__init__(method=<VideoMatchMethod.AUTO: 'auto'>, strict=False, content_frames=3, compare_predictions='auto', compare_images=False, image_similarity_threshold=0.05)
¶
Method generated by attrs for class VideoMatcher.
Source code in sleap_io/model/matching.py
__repr__()
¶
Method generated by attrs for class VideoMatcher.
Source code in sleap_io/model/matching.py
"""Unified matcher system for comparing and matching data structures during merging.
This module provides configurable matchers for comparing skeletons, instances, tracks,
and videos during merge operations. The matchers use various strategies to determine
when data structures should be considered equivalent during merging.
Key features:
- Skeleton matching: exact, structure-based, overlap, and subset matching
- Instance matching: spatial proximity, track identity, and bounding box IoU
- Track matching: by name or object identity
- Identity matching: by name or object identity
- Category matching: by name or object identity
- Video matching: path, basename, content, and auto matching
Video matching supports path-based, filename-based, content-based, and
__setattr__(name, val)
¶
find_match(incoming, candidates, labels_incoming=None, labels_base=None)
¶
Find a matching video from candidates using the configured method.
This is the preferred method for AUTO matching as it implements the full safe matching cascade including leaf-uniqueness disambiguation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
incoming
|
Video
|
The video to find a match for. |
required |
candidates
|
list[Video]
|
List of existing videos to search for matches. |
required |
labels_incoming
|
Labels | None
|
Labels object containing the incoming video's annotations. Used for pose-based matching in AUTO mode. |
None
|
labels_base
|
Labels | None
|
Labels object containing the candidates' annotations. Used for pose-based matching in AUTO mode. |
None
|
Returns:
| Type | Description |
|---|---|
Video | None
|
The matched video, or None if no match found. |
Notes
For AUTO method, implements the safe matching cascade: 1. Shape rejection (filter candidates) 2. original_video conflict rejection (filter candidates) 3. Definitive file identity (is_same_file) 4. Strict path match 5. Leaf uniqueness matching at increasing depths 6. Pose-based matching (if labels provided) 7. Image-based matching (if compare_images=True)
Shape is for REJECTION only - compatible shapes don't imply a match.
Source code in sleap_io/model/matching.py
def find_match(
self,
incoming: Video,
candidates: list[Video],
labels_incoming: "Labels | None" = None,
labels_base: "Labels | None" = None,
) -> Video | None:
"""Find a matching video from candidates using the configured method.
This is the preferred method for AUTO matching as it implements the
full safe matching cascade including leaf-uniqueness disambiguation.
Args:
incoming: The video to find a match for.
candidates: List of existing videos to search for matches.
labels_incoming: Labels object containing the incoming video's
annotations. Used for pose-based matching in AUTO mode.
labels_base: Labels object containing the candidates' annotations.
Used for pose-based matching in AUTO mode.
Returns:
The matched video, or None if no match found.
Notes:
For AUTO method, implements the safe matching cascade:
1. Shape rejection (filter candidates)
2. original_video conflict rejection (filter candidates)
3. Definitive file identity (is_same_file)
4. Strict path match
5. Leaf uniqueness matching at increasing depths
6. Pose-based matching (if labels provided)
7. Image-based matching (if compare_images=True)
Shape is for REJECTION only - compatible shapes don't imply a match.
"""
from pathlib import Path
from sleap_io.io.utils import sanitize_filename
if self.method == VideoMatchMethod.AUTO:
# Build list of viable candidates (not rejected by shape/provenance)
viable = []
for candidate in candidates:
# REJECTION CHECK 1: Shape compatibility
shape_compat = shapes_compatible(candidate, incoming)
if shape_compat is False:
# Definitely incompatible shapes - skip
continue
# REJECTION CHECK 2: original_video conflict
if original_videos_conflict(candidate, incoming):
# Both have provenance pointing to different files - skip
continue
# REJECTION CHECK 3: same source file, different crop.
# Distinct crops (mosaic tiles) of one physical file share a
# root file, so dropping them here prevents the file-identity,
# strict-path, and leaf-uniqueness rungs from collapsing them.
# For non-crop candidates this is always False.
if _same_file_different_crop(candidate, incoming):
continue
viable.append(candidate)
# DEFINITIVE CHECK: File identity (handles source_video chains)
for candidate in viable:
if is_same_file(candidate, incoming):
return candidate
# STRING CHECK: Full path match
for candidate in viable:
if candidate.matches_path(incoming, strict=True):
return candidate
# STRING CHECK: Leaf path uniqueness
# Match paths by comparing suffixes at increasing depths
if viable:
def get_path_parts(video: Video) -> tuple[str, ...]:
"""Get path parts for comparison, using root video for embedded."""
root = _get_root_video(video)
fn = root.filename
if isinstance(fn, list):
fn = fn[0] # Use first for ImageVideo
return Path(sanitize_filename(fn)).parts
incoming_parts = get_path_parts(incoming)
candidate_parts = [(v, get_path_parts(v)) for v in viable]
# Also need all existing videos for uniqueness check
all_existing_parts = [(v, get_path_parts(v)) for v in candidates]
# Compare at increasing depths until we find a unique match
max_depth = max(
len(incoming_parts),
max((len(p) for _, p in all_existing_parts), default=0),
)
for depth in range(1, max_depth + 1):
if len(incoming_parts) < depth:
continue
incoming_leaf = "/".join(incoming_parts[-depth:])
# Find all viable candidates that match at this depth
matches_at_depth = []
for candidate, parts in candidate_parts:
if len(parts) < depth:
continue
candidate_leaf = "/".join(parts[-depth:])
if candidate_leaf == incoming_leaf:
matches_at_depth.append(candidate)
# If exactly one match at this depth, use it
if len(matches_at_depth) == 1:
return matches_at_depth[0]
# If no matches, try deeper
# If multiple matches, continue deeper to disambiguate
# POSE MATCHING: Compare pose annotations (default in AUTO)
if labels_incoming is not None and labels_base is not None:
match = self._match_by_poses(
incoming, viable, labels_incoming, labels_base
)
if match is not None:
return match
# IMAGE MATCHING: Compare frame images (opt-in)
if self.compare_images:
match = self._match_by_images(incoming, viable)
if match is not None:
return match
# No match found
return None
else:
# Non-AUTO methods: use pairwise match()
for candidate in candidates:
if self.match(candidate, incoming):
return candidate
return None
match(video1, video2)
¶
Check if two videos match according to the configured method.
For AUTO method, this performs pairwise checks (file identity, path match). For full AUTO matching with leaf-uniqueness, use find_match() instead.
Source code in sleap_io/model/matching.py
def match(self, video1: Video, video2: Video) -> bool:
"""Check if two videos match according to the configured method.
For AUTO method, this performs pairwise checks (file identity, path match).
For full AUTO matching with leaf-uniqueness, use find_match() instead.
"""
if self.method == VideoMatchMethod.AUTO:
# Pairwise AUTO: rejection checks + definitive identity + path match
# (Leaf-uniqueness requires full candidate list - use find_match())
# Rejection: incompatible shapes
if shapes_compatible(video1, video2) is False:
return False
# Rejection: conflicting provenance
if original_videos_conflict(video1, video2):
return False
# Definitive: same source file but different crop (mosaic tiles).
# Must run before any path rung, which would otherwise re-match the
# shared root file. For non-crop videos this is always False.
if _same_file_different_crop(video1, video2):
return False
# Definitive: same file identity (crop-aware)
if is_same_file(video1, video2):
return True
# String: strict path match
if video1.matches_path(video2, strict=True):
return True
# String: basename match (for pairwise, this is the fallback)
if video1.matches_path(video2, strict=False):
return True
return False
elif self.method == VideoMatchMethod.PATH:
return video1.matches_path(video2, strict=self.strict)
elif self.method == VideoMatchMethod.BASENAME:
return video1.matches_path(video2, strict=False)
elif self.method == VideoMatchMethod.CONTENT:
return video1.matches_content(video2)
elif self.method == VideoMatchMethod.IMAGE_DEDUP:
# Match ImageVideo instances with overlapping images (ImageVideo only)
return video1.has_overlapping_images(video2)
elif self.method == VideoMatchMethod.SHAPE:
# Match videos by shape only (height, width, channels)
return video1.matches_shape(video2)
else:
raise ValueError(f"Unknown video match method: {self.method}")
SkeletonMatcher¶
sleap_io.model.matching.SkeletonMatcher
¶
Matcher for comparing and matching skeletons.
Attributes:
| Name | Type | Description |
|---|---|---|
method |
The matching method to use. Can be a SkeletonMatchMethod enum value or a string that will be converted to the enum. Default is STRUCTURE. |
|
require_same_order |
Whether to require nodes in the same order for STRUCTURE matching. Only used when method is STRUCTURE. Default is False. |
|
min_overlap |
Minimum Jaccard similarity required for OVERLAP matching. Only used when method is OVERLAP. Default is 0.5. |
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class SkeletonMatcher. |
__init__ |
Method generated by attrs for class SkeletonMatcher. |
__repr__ |
Method generated by attrs for class SkeletonMatcher. |
__setattr__ |
Method generated by attrs for class SkeletonMatcher. |
match |
Check if two skeletons match according to the configured method. |
Source code in sleap_io/model/matching.py
@attrs.define
class SkeletonMatcher:
"""Matcher for comparing and matching skeletons.
Attributes:
method: The matching method to use. Can be a SkeletonMatchMethod enum value
or a string that will be converted to the enum. Default is STRUCTURE.
require_same_order: Whether to require nodes in the same order for STRUCTURE
matching. Only used when method is STRUCTURE. Default is False.
min_overlap: Minimum Jaccard similarity required for OVERLAP matching.
Only used when method is OVERLAP. Default is 0.5.
"""
method: SkeletonMatchMethod | str = attrs.field(
default=SkeletonMatchMethod.STRUCTURE,
converter=lambda x: SkeletonMatchMethod(x) if isinstance(x, str) else x,
)
require_same_order: bool = False
min_overlap: float = 0.5
def match(self, skeleton1: Skeleton, skeleton2: Skeleton) -> bool:
"""Check if two skeletons match according to the configured method."""
if self.method == SkeletonMatchMethod.EXACT:
return skeleton1.matches(skeleton2, require_same_order=True)
elif self.method == SkeletonMatchMethod.STRUCTURE:
return skeleton1.matches(
skeleton2, require_same_order=self.require_same_order
)
elif self.method == SkeletonMatchMethod.OVERLAP:
metrics = skeleton1.node_similarities(skeleton2)
return metrics["jaccard"] >= self.min_overlap
elif self.method == SkeletonMatchMethod.SUBSET:
# Check if skeleton1 nodes are subset of skeleton2
nodes1 = set(skeleton1.node_names)
nodes2 = set(skeleton2.node_names)
return nodes1.issubset(nodes2)
else:
raise ValueError(f"Unknown skeleton match method: {self.method}")
__annotations__ = {'method': 'SkeletonMatchMethod | str', 'require_same_order': 'bool', 'min_overlap': 'float'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = True
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Matcher for comparing and matching skeletons.\n\nAttributes:\n method: The matching method to use. Can be a SkeletonMatchMethod enum value\n or a string that will be converted to the enum. Default is STRUCTURE.\n require_same_order: Whether to require nodes in the same order for STRUCTURE\n matching. Only used when method is STRUCTURE. Default is False.\n min_overlap: Minimum Jaccard similarity required for OVERLAP matching.\n Only used when method is OVERLAP. Default is 0.5.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 756
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('method', 'require_same_order', 'min_overlap')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('method', 'require_same_order', 'min_overlap', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
__init__(method=<SkeletonMatchMethod.STRUCTURE: 'structure'>, require_same_order=False, min_overlap=0.5)
¶
__repr__()
¶
Method generated by attrs for class SkeletonMatcher.
Source code in sleap_io/model/matching.py
"""Unified matcher system for comparing and matching data structures during merging.
This module provides configurable matchers for comparing skeletons, instances, tracks,
and videos during merge operations. The matchers use various strategies to determine
when data structures should be considered equivalent during merging.
Key features:
- Skeleton matching: exact, structure-based, overlap, and subset matching
- Instance matching: spatial proximity, track identity, and bounding box IoU
- Track matching: by name or object identity
- Identity matching: by name or object identity
- Category matching: by name or object identity
- Video matching: path, basename, content, and auto matching
Video matching supports path-based, filename-based, content-based, and
__setattr__(name, val)
¶
match(skeleton1, skeleton2)
¶
Check if two skeletons match according to the configured method.
Source code in sleap_io/model/matching.py
def match(self, skeleton1: Skeleton, skeleton2: Skeleton) -> bool:
"""Check if two skeletons match according to the configured method."""
if self.method == SkeletonMatchMethod.EXACT:
return skeleton1.matches(skeleton2, require_same_order=True)
elif self.method == SkeletonMatchMethod.STRUCTURE:
return skeleton1.matches(
skeleton2, require_same_order=self.require_same_order
)
elif self.method == SkeletonMatchMethod.OVERLAP:
metrics = skeleton1.node_similarities(skeleton2)
return metrics["jaccard"] >= self.min_overlap
elif self.method == SkeletonMatchMethod.SUBSET:
# Check if skeleton1 nodes are subset of skeleton2
nodes1 = set(skeleton1.node_names)
nodes2 = set(skeleton2.node_names)
return nodes1.issubset(nodes2)
else:
raise ValueError(f"Unknown skeleton match method: {self.method}")
TrackMatcher¶
sleap_io.model.matching.TrackMatcher
¶
Matcher for comparing and matching tracks.
Attributes:
| Name | Type | Description |
|---|---|---|
method |
The matching method to use. Can be a TrackMatchMethod enum value or a string that will be converted to the enum. Default is IDENTITY (matches only the same Track object; correctness-first). Use NAME to match by track name. |
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class TrackMatcher. |
__init__ |
Method generated by attrs for class TrackMatcher. |
__repr__ |
Method generated by attrs for class TrackMatcher. |
__setattr__ |
Method generated by attrs for class TrackMatcher. |
match |
Check if two tracks match according to the configured method. |
Source code in sleap_io/model/matching.py
@attrs.define
class TrackMatcher:
"""Matcher for comparing and matching tracks.
Attributes:
method: The matching method to use. Can be a TrackMatchMethod enum value
or a string that will be converted to the enum. Default is IDENTITY
(matches only the same Track object; correctness-first). Use NAME to
match by track name.
"""
method: TrackMatchMethod | str = attrs.field(
default=TrackMatchMethod.IDENTITY,
converter=lambda x: TrackMatchMethod(x) if isinstance(x, str) else x,
)
def match(self, track1: Track, track2: Track) -> bool:
"""Check if two tracks match according to the configured method."""
return track1.matches(track2, method=self.method.value)
__annotations__ = {'method': 'TrackMatchMethod | str'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = True
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Matcher for comparing and matching tracks.\n\nAttributes:\n method: The matching method to use. Can be a TrackMatchMethod enum value\n or a string that will be converted to the enum. Default is IDENTITY\n (matches only the same Track object; correctness-first). Use NAME to\n match by track name.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 883
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('method',)
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('method', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
__init__(method=<TrackMatchMethod.IDENTITY: 'identity'>)
¶
__repr__()
¶
Method generated by attrs for class TrackMatcher.
Source code in sleap_io/model/matching.py
"""Unified matcher system for comparing and matching data structures during merging.
This module provides configurable matchers for comparing skeletons, instances, tracks,
and videos during merge operations. The matchers use various strategies to determine
when data structures should be considered equivalent during merging.
Key features:
- Skeleton matching: exact, structure-based, overlap, and subset matching
- Instance matching: spatial proximity, track identity, and bounding box IoU
- Track matching: by name or object identity
- Identity matching: by name or object identity
- Category matching: by name or object identity
- Video matching: path, basename, content, and auto matching
Video matching supports path-based, filename-based, content-based, and
__setattr__(name, val)
¶
match(track1, track2)
¶
InstanceMatcher¶
sleap_io.model.matching.InstanceMatcher
¶
Matcher for comparing and matching instances.
Attributes:
| Name | Type | Description |
|---|---|---|
method |
The matching method to use. Can be an InstanceMatchMethod enum value or a string that will be converted to the enum. Default is SPATIAL. |
|
threshold |
The threshold value used for matching. For SPATIAL method, this is the maximum pixel distance. For IOU method, this is the minimum IoU value. Not used for IDENTITY method. Default is 5.0. |
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class InstanceMatcher. |
__init__ |
Method generated by attrs for class InstanceMatcher. |
__repr__ |
Method generated by attrs for class InstanceMatcher. |
__setattr__ |
Method generated by attrs for class InstanceMatcher. |
find_matches |
Find all matching instances between two lists. |
match |
Check if two instances match according to the configured method. |
Source code in sleap_io/model/matching.py
@attrs.define
class InstanceMatcher:
"""Matcher for comparing and matching instances.
Attributes:
method: The matching method to use. Can be an InstanceMatchMethod enum value
or a string that will be converted to the enum. Default is SPATIAL.
threshold: The threshold value used for matching. For SPATIAL method, this is
the maximum pixel distance. For IOU method, this is the minimum IoU value.
Not used for IDENTITY method. Default is 5.0.
"""
method: InstanceMatchMethod | str = attrs.field(
default=InstanceMatchMethod.SPATIAL,
converter=lambda x: InstanceMatchMethod(x) if isinstance(x, str) else x,
)
threshold: float = 5.0
def match(self, instance1: Instance, instance2: Instance) -> bool:
"""Check if two instances match according to the configured method."""
if self.method == InstanceMatchMethod.SPATIAL:
return instance1.same_pose_as(instance2, tolerance=self.threshold)
elif self.method == InstanceMatchMethod.IDENTITY:
return instance1.same_identity_as(instance2)
elif self.method == InstanceMatchMethod.IOU:
return instance1.overlaps_with(instance2, iou_threshold=self.threshold)
else:
raise ValueError(f"Unknown instance match method: {self.method}")
def find_matches(
self, instances1: list[Instance], instances2: list[Instance]
) -> list[tuple[int, int, float]]:
"""Find all matching instances between two lists.
Returns:
List of (idx1, idx2, score) tuples for matching instances.
"""
matches = []
for i, inst1 in enumerate(instances1):
for j, inst2 in enumerate(instances2):
if self.match(inst1, inst2):
# Calculate match score based on method
if self.method == InstanceMatchMethod.SPATIAL:
# Use inverse distance as score
pts1 = inst1.numpy()
pts2 = inst2.numpy()
valid = ~(np.isnan(pts1[:, 0]) | np.isnan(pts2[:, 0]))
if valid.any():
distances = np.linalg.norm(
pts1[valid] - pts2[valid], axis=1
)
score = 1.0 / (1.0 + np.mean(distances))
else:
score = 0.0
elif self.method == InstanceMatchMethod.IOU:
# Calculate actual IoU as score
bbox1 = inst1.bounding_box()
bbox2 = inst2.bounding_box()
if bbox1 is not None and bbox2 is not None:
# Calculate IoU
intersection_min = np.maximum(bbox1[0], bbox2[0])
intersection_max = np.minimum(bbox1[1], bbox2[1])
if np.all(intersection_min < intersection_max):
intersection_area = np.prod(
intersection_max - intersection_min
)
area1 = np.prod(bbox1[1] - bbox1[0])
area2 = np.prod(bbox2[1] - bbox2[0])
union_area = area1 + area2 - intersection_area
score = (
intersection_area / union_area
if union_area > 0
else 0
)
else:
score = 0.0
else:
score = 0.0
else:
score = 1.0 # Binary match for identity
matches.append((i, j, score))
return matches
__annotations__ = {'method': 'InstanceMatchMethod | str', 'threshold': 'float'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = True
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Matcher for comparing and matching instances.\n\nAttributes:\n method: The matching method to use. Can be an InstanceMatchMethod enum value\n or a string that will be converted to the enum. Default is SPATIAL.\n threshold: The threshold value used for matching. For SPATIAL method, this is\n the maximum pixel distance. For IOU method, this is the minimum IoU value.\n Not used for IDENTITY method. Default is 5.0.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 796
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('method', 'threshold')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('method', 'threshold', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
__init__(method=<InstanceMatchMethod.SPATIAL: 'spatial'>, threshold=5.0)
¶
__repr__()
¶
Method generated by attrs for class InstanceMatcher.
Source code in sleap_io/model/matching.py
"""Unified matcher system for comparing and matching data structures during merging.
This module provides configurable matchers for comparing skeletons, instances, tracks,
and videos during merge operations. The matchers use various strategies to determine
when data structures should be considered equivalent during merging.
Key features:
- Skeleton matching: exact, structure-based, overlap, and subset matching
- Instance matching: spatial proximity, track identity, and bounding box IoU
- Track matching: by name or object identity
- Identity matching: by name or object identity
- Category matching: by name or object identity
- Video matching: path, basename, content, and auto matching
Video matching supports path-based, filename-based, content-based, and
__setattr__(name, val)
¶
find_matches(instances1, instances2)
¶
Find all matching instances between two lists.
Returns:
| Type | Description |
|---|---|
list[tuple[int, int, float]]
|
List of (idx1, idx2, score) tuples for matching instances. |
Source code in sleap_io/model/matching.py
def find_matches(
self, instances1: list[Instance], instances2: list[Instance]
) -> list[tuple[int, int, float]]:
"""Find all matching instances between two lists.
Returns:
List of (idx1, idx2, score) tuples for matching instances.
"""
matches = []
for i, inst1 in enumerate(instances1):
for j, inst2 in enumerate(instances2):
if self.match(inst1, inst2):
# Calculate match score based on method
if self.method == InstanceMatchMethod.SPATIAL:
# Use inverse distance as score
pts1 = inst1.numpy()
pts2 = inst2.numpy()
valid = ~(np.isnan(pts1[:, 0]) | np.isnan(pts2[:, 0]))
if valid.any():
distances = np.linalg.norm(
pts1[valid] - pts2[valid], axis=1
)
score = 1.0 / (1.0 + np.mean(distances))
else:
score = 0.0
elif self.method == InstanceMatchMethod.IOU:
# Calculate actual IoU as score
bbox1 = inst1.bounding_box()
bbox2 = inst2.bounding_box()
if bbox1 is not None and bbox2 is not None:
# Calculate IoU
intersection_min = np.maximum(bbox1[0], bbox2[0])
intersection_max = np.minimum(bbox1[1], bbox2[1])
if np.all(intersection_min < intersection_max):
intersection_area = np.prod(
intersection_max - intersection_min
)
area1 = np.prod(bbox1[1] - bbox1[0])
area2 = np.prod(bbox2[1] - bbox2[0])
union_area = area1 + area2 - intersection_area
score = (
intersection_area / union_area
if union_area > 0
else 0
)
else:
score = 0.0
else:
score = 0.0
else:
score = 1.0 # Binary match for identity
matches.append((i, j, score))
return matches
match(instance1, instance2)
¶
Check if two instances match according to the configured method.
Source code in sleap_io/model/matching.py
def match(self, instance1: Instance, instance2: Instance) -> bool:
"""Check if two instances match according to the configured method."""
if self.method == InstanceMatchMethod.SPATIAL:
return instance1.same_pose_as(instance2, tolerance=self.threshold)
elif self.method == InstanceMatchMethod.IDENTITY:
return instance1.same_identity_as(instance2)
elif self.method == InstanceMatchMethod.IOU:
return instance1.overlaps_with(instance2, iou_threshold=self.threshold)
else:
raise ValueError(f"Unknown instance match method: {self.method}")
MergeResult¶
sleap_io.model.matching.MergeResult
¶
Result of a merge operation.
Attributes:
| Name | Type | Description |
|---|---|---|
successful |
Whether the merge completed successfully. |
|
frames_merged |
Number of frames that were merged. |
|
instances_added |
Number of new instances added. |
|
instances_updated |
Number of existing instances that were updated. |
|
instances_skipped |
Number of instances that were skipped. |
|
conflicts |
List of conflicts that were resolved during merging. |
|
errors |
List of errors encountered during merging. |
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class MergeResult. |
__init__ |
Method generated by attrs for class MergeResult. |
__repr__ |
Method generated by attrs for class MergeResult. |
summary |
Generate a human-readable summary of the merge result. |
Source code in sleap_io/model/matching.py
@attrs.define
class MergeResult:
"""Result of a merge operation.
Attributes:
successful: Whether the merge completed successfully.
frames_merged: Number of frames that were merged.
instances_added: Number of new instances added.
instances_updated: Number of existing instances that were updated.
instances_skipped: Number of instances that were skipped.
conflicts: List of conflicts that were resolved during merging.
errors: List of errors encountered during merging.
"""
successful: bool
frames_merged: int = 0
instances_added: int = 0
instances_updated: int = 0
instances_skipped: int = 0
conflicts: list[ConflictResolution] = attrs.field(factory=list)
errors: list[MergeError] = attrs.field(factory=list)
def summary(self) -> str:
"""Generate a human-readable summary of the merge result."""
lines = []
if self.successful:
lines.append("✓ Merge completed successfully")
else:
lines.append("✗ Merge completed with errors")
lines.append(f" Frames merged: {self.frames_merged}")
lines.append(f" Instances added: {self.instances_added}")
if self.instances_updated:
lines.append(f" Instances updated: {self.instances_updated}")
if self.instances_skipped:
lines.append(f" Instances skipped: {self.instances_skipped}")
if self.conflicts:
lines.append(f" Conflicts resolved: {len(self.conflicts)}")
if self.errors:
lines.append(f" Errors encountered: {len(self.errors)}")
for error in self.errors[:5]: # Show first 5 errors
lines.append(f" - {error.message}")
if len(self.errors) > 5:
lines.append(f" ... and {len(self.errors) - 5} more")
return "\n".join(lines)
__annotations__ = {'successful': 'bool', 'frames_merged': 'int', 'instances_added': 'int', 'instances_updated': 'int', 'instances_skipped': 'int', 'conflicts': 'list[ConflictResolution]', 'errors': 'list[MergeError]'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = False
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Result of a merge operation.\n\nAttributes:\n successful: Whether the merge completed successfully.\n frames_merged: Number of frames that were merged.\n instances_added: Number of new instances added.\n instances_updated: Number of existing instances that were updated.\n instances_skipped: Number of instances that were skipped.\n conflicts: List of conflicts that were resolved during merging.\n errors: List of errors encountered during merging.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 1363
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('successful', 'frames_merged', 'instances_added', 'instances_updated', 'instances_skipped', 'conflicts', 'errors')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('successful', 'frames_merged', 'instances_added', 'instances_updated', 'instances_skipped', 'conflicts', 'errors', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
__init__(successful, frames_merged=0, instances_added=0, instances_updated=0, instances_skipped=0, conflicts=NOTHING, errors=NOTHING)
¶
Method generated by attrs for class MergeResult.
Source code in sleap_io/model/matching.py
from sleap_io.model.identity import Identity
from sleap_io.model.instance import Instance, Track
from sleap_io.model.labeled_frame import LabeledFrame
from sleap_io.model.skeleton import Skeleton
from sleap_io.model.video import Video
if TYPE_CHECKING:
from sleap_io.model.labels import Labels
class SkeletonMatchMethod(str, Enum):
"""Methods for matching skeletons.
Attributes:
__repr__()
¶
Method generated by attrs for class MergeResult.
Source code in sleap_io/model/matching.py
"""Unified matcher system for comparing and matching data structures during merging.
This module provides configurable matchers for comparing skeletons, instances, tracks,
and videos during merge operations. The matchers use various strategies to determine
when data structures should be considered equivalent during merging.
Key features:
- Skeleton matching: exact, structure-based, overlap, and subset matching
- Instance matching: spatial proximity, track identity, and bounding box IoU
- Track matching: by name or object identity
- Identity matching: by name or object identity
- Category matching: by name or object identity
- Video matching: path, basename, content, and auto matching
Video matching supports path-based, filename-based, content-based, and
summary()
¶
Generate a human-readable summary of the merge result.
Source code in sleap_io/model/matching.py
def summary(self) -> str:
"""Generate a human-readable summary of the merge result."""
lines = []
if self.successful:
lines.append("✓ Merge completed successfully")
else:
lines.append("✗ Merge completed with errors")
lines.append(f" Frames merged: {self.frames_merged}")
lines.append(f" Instances added: {self.instances_added}")
if self.instances_updated:
lines.append(f" Instances updated: {self.instances_updated}")
if self.instances_skipped:
lines.append(f" Instances skipped: {self.instances_skipped}")
if self.conflicts:
lines.append(f" Conflicts resolved: {len(self.conflicts)}")
if self.errors:
lines.append(f" Errors encountered: {len(self.errors)}")
for error in self.errors[:5]: # Show first 5 errors
lines.append(f" - {error.message}")
if len(self.errors) > 5:
lines.append(f" ... and {len(self.errors) - 5} more")
return "\n".join(lines)
MatchResult¶
sleap_io.model.matching.MatchResult
¶
Result of matching two Labels objects.
This class holds correspondence maps between items in two Labels objects, without modifying either. Useful for evaluation workflows where you need to align predictions with ground truth without merging them.
Attributes:
| Name | Type | Description |
|---|---|---|
video_map |
Dictionary mapping videos from the |
|
skeleton_map |
Dictionary mapping skeletons from |
|
track_map |
Dictionary mapping tracks from |
Example
Match prediction videos to ground truth for evaluation::
>>> gt_labels = sio.load_slp("ground_truth.slp")
>>> pred_labels = sio.load_slp("predictions.slp")
>>> result = gt_labels.match(pred_labels)
>>> for pred_video, gt_video in result.video_map.items():
... if gt_video is not None:
... print(f"{pred_video.filename} -> {gt_video.filename}")
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class MatchResult. |
__init__ |
Method generated by attrs for class MatchResult. |
__repr__ |
Method generated by attrs for class MatchResult. |
summary |
Generate a human-readable summary of the match result. |
Source code in sleap_io/model/matching.py
@attrs.define
class MatchResult:
"""Result of matching two Labels objects.
This class holds correspondence maps between items in two Labels objects,
without modifying either. Useful for evaluation workflows where you need
to align predictions with ground truth without merging them.
Attributes:
video_map: Dictionary mapping videos from the `other` Labels to videos in
`self`. Values are None if no match was found.
skeleton_map: Dictionary mapping skeletons from `other` to `self`.
track_map: Dictionary mapping tracks from `other` to `self`.
Example:
Match prediction videos to ground truth for evaluation::
>>> gt_labels = sio.load_slp("ground_truth.slp")
>>> pred_labels = sio.load_slp("predictions.slp")
>>> result = gt_labels.match(pred_labels)
>>> for pred_video, gt_video in result.video_map.items():
... if gt_video is not None:
... print(f"{pred_video.filename} -> {gt_video.filename}")
"""
video_map: dict[Video, Video | None] = attrs.field(factory=dict)
skeleton_map: dict[Skeleton, Skeleton | None] = attrs.field(factory=dict)
track_map: dict[Track, Track | None] = attrs.field(factory=dict)
@property
def unmatched_videos(self) -> list[Video]:
"""Videos from other Labels that had no match in self."""
return [v for v, match in self.video_map.items() if match is None]
@property
def unmatched_skeletons(self) -> list[Skeleton]:
"""Skeletons from other Labels that had no match in self."""
return [s for s, match in self.skeleton_map.items() if match is None]
@property
def unmatched_tracks(self) -> list[Track]:
"""Tracks from other Labels that had no match in self."""
return [t for t, match in self.track_map.items() if match is None]
@property
def all_videos_matched(self) -> bool:
"""True if all videos from other were matched."""
return len(self.unmatched_videos) == 0
@property
def all_skeletons_matched(self) -> bool:
"""True if all skeletons from other were matched."""
return len(self.unmatched_skeletons) == 0
@property
def all_tracks_matched(self) -> bool:
"""True if all tracks from other were matched."""
return len(self.unmatched_tracks) == 0
@property
def n_videos_matched(self) -> int:
"""Number of videos that were successfully matched."""
return sum(1 for v in self.video_map.values() if v is not None)
@property
def n_skeletons_matched(self) -> int:
"""Number of skeletons that were successfully matched."""
return sum(1 for s in self.skeleton_map.values() if s is not None)
@property
def n_tracks_matched(self) -> int:
"""Number of tracks that were successfully matched."""
return sum(1 for t in self.track_map.values() if t is not None)
def summary(self) -> str:
"""Generate a human-readable summary of the match result."""
lines = []
lines.append(f"Videos: {self.n_videos_matched}/{len(self.video_map)} matched")
lines.append(
f"Skeletons: {self.n_skeletons_matched}/{len(self.skeleton_map)} matched"
)
lines.append(f"Tracks: {self.n_tracks_matched}/{len(self.track_map)} matched")
if self.unmatched_videos:
lines.append("Unmatched videos:")
for v in self.unmatched_videos[:5]:
fn = v.filename if isinstance(v.filename, str) else v.filename[0]
lines.append(f" - {fn}")
if len(self.unmatched_videos) > 5:
lines.append(f" ... and {len(self.unmatched_videos) - 5} more")
return "\n".join(lines)
__annotations__ = {'video_map': 'dict[Video, Video | None]', 'skeleton_map': 'dict[Skeleton, Skeleton | None]', 'track_map': 'dict[Track, Track | None]'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = False
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Result of matching two Labels objects.\n\nThis class holds correspondence maps between items in two Labels objects,\nwithout modifying either. Useful for evaluation workflows where you need\nto align predictions with ground truth without merging them.\n\nAttributes:\n video_map: Dictionary mapping videos from the `other` Labels to videos in\n `self`. Values are None if no match was found.\n skeleton_map: Dictionary mapping skeletons from `other` to `self`.\n track_map: Dictionary mapping tracks from `other` to `self`.\n\nExample:\n Match prediction videos to ground truth for evaluation::\n\n >>> gt_labels = sio.load_slp("ground_truth.slp")\n >>> pred_labels = sio.load_slp("predictions.slp")\n >>> result = gt_labels.match(pred_labels)\n >>> for pred_video, gt_video in result.video_map.items():\n ... if gt_video is not None:\n ... print(f"{pred_video.filename} -> {gt_video.filename}")\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 1416
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('video_map', 'skeleton_map', 'track_map')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.model.matching'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('video_map', 'skeleton_map', 'track_map', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
all_skeletons_matched
property
¶
True if all skeletons from other were matched.
all_tracks_matched
property
¶
True if all tracks from other were matched.
all_videos_matched
property
¶
True if all videos from other were matched.
n_skeletons_matched
property
¶
Number of skeletons that were successfully matched.
n_tracks_matched
property
¶
Number of tracks that were successfully matched.
n_videos_matched
property
¶
Number of videos that were successfully matched.
unmatched_skeletons
property
¶
Skeletons from other Labels that had no match in self.
unmatched_tracks
property
¶
Tracks from other Labels that had no match in self.
unmatched_videos
property
¶
Videos from other Labels that had no match in self.
__eq__(other)
¶
__init__(video_map=NOTHING, skeleton_map=NOTHING, track_map=NOTHING)
¶
Method generated by attrs for class MatchResult.
Source code in sleap_io/model/matching.py
import attrs
import numpy as np
from sleap_io.model.category import Category
from sleap_io.model.identity import Identity
from sleap_io.model.instance import Instance, Track
from sleap_io.model.labeled_frame import LabeledFrame
from sleap_io.model.skeleton import Skeleton
from sleap_io.model.video import Video
if TYPE_CHECKING:
from sleap_io.model.labels import Labels
__repr__()
¶
Method generated by attrs for class MatchResult.
Source code in sleap_io/model/matching.py
"""Unified matcher system for comparing and matching data structures during merging.
This module provides configurable matchers for comparing skeletons, instances, tracks,
and videos during merge operations. The matchers use various strategies to determine
when data structures should be considered equivalent during merging.
Key features:
- Skeleton matching: exact, structure-based, overlap, and subset matching
- Instance matching: spatial proximity, track identity, and bounding box IoU
- Track matching: by name or object identity
- Identity matching: by name or object identity
- Category matching: by name or object identity
- Video matching: path, basename, content, and auto matching
Video matching supports path-based, filename-based, content-based, and
summary()
¶
Generate a human-readable summary of the match result.
Source code in sleap_io/model/matching.py
def summary(self) -> str:
"""Generate a human-readable summary of the match result."""
lines = []
lines.append(f"Videos: {self.n_videos_matched}/{len(self.video_map)} matched")
lines.append(
f"Skeletons: {self.n_skeletons_matched}/{len(self.skeleton_map)} matched"
)
lines.append(f"Tracks: {self.n_tracks_matched}/{len(self.track_map)} matched")
if self.unmatched_videos:
lines.append("Unmatched videos:")
for v in self.unmatched_videos[:5]:
fn = v.filename if isinstance(v.filename, str) else v.filename[0]
lines.append(f" - {fn}")
if len(self.unmatched_videos) > 5:
lines.append(f" ... and {len(self.unmatched_videos) - 5} more")
return "\n".join(lines)