SLP File Format¶

The .slp file format is SLEAP's native format for storing pose tracking data. It is built on top of HDF5, a hierarchical data format designed for storing and organizing large amounts of scientific data.

SLP files can contain:

Video metadata and references to source video files (Video)
Embedded images with configurable encoding (PNG, JPEG, or raw arrays)
Skeleton definitions with nodes, edges, and symmetries (Skeleton)
Labeled frames with user annotations and model predictions (LabeledFrame)
Tracks for identity tracking across frames (Track)
Suggestions for frames to label (SuggestionFrame)
Recording sessions for multi-camera setups (RecordingSession)

HDF5 Layout¶

SLP files have the following hierarchical structure:

file.slp
├── /metadata                    # Group: Format and skeleton metadata
│   ├── @format_id               # Attribute: Format version (float, e.g., 1.4)
│   └── @json                    # Attribute: JSON metadata string
│
├── /videos_json                 # Dataset: Video metadata (variable-length bytes)
├── /tracks_json                 # Dataset: Track metadata (variable-length bytes)
├── /suggestions_json            # Dataset: Suggestions (variable-length bytes, optional)
├── /sessions_json               # Dataset: Recording sessions (variable-length bytes, optional)
│
├── /frames                      # Dataset: Labeled frame metadata (structured array)
├── /instances                   # Dataset: Instance metadata (structured array)
├── /points                      # Dataset: User-labeled points (structured array)
├── /pred_points                 # Dataset: Predicted points (structured array)
│
└── /video{N}/                   # Group: Per-video embedded data (one per video)
    ├── /video                   # Dataset: Embedded image data
    │   ├── @format              # Attribute: "png", "jpg", or "hdf5"
    │   ├── @channel_order       # Attribute: "RGB" or "BGR"
    │   ├── @frames              # Attribute: Total frames in source video
    │   ├── @height              # Attribute: Frame height
    │   ├── @width               # Attribute: Frame width
    │   ├── @channels            # Attribute: Number of channels
    │   └── @fps                 # Attribute: Frames per second (optional)
    ├── /frame_numbers           # Dataset: Embedded frame indices (int array)
    └── /source_video/           # Group: Source video metadata
        └── @json                # Attribute: JSON with source video info

Core Datasets¶

Dataset	Shape	Dtype	Description
`points`	`(N,)`	structured	User-labeled point coordinates
`pred_points`	`(N,)`	structured	Predicted point coordinates with scores
`instances`	`(N,)`	structured	Instance metadata linking to points
`frames`	`(N,)`	structured	Frame metadata linking to instances

Metadata Datasets¶

Dataset	Type	Description
`videos_json`	`bytes[]`	JSON array of video metadata
`tracks_json`	`bytes[]`	JSON array of track definitions
`suggestions_json`	`bytes[]`	JSON array of suggested frames (optional)
`sessions_json`	`bytes[]`	JSON array of recording sessions (optional)

Videos¶

Video metadata is stored in the /videos_json dataset as an array of JSON strings. Each video entry contains:

{
    "filename": "path/to/video.mp4",
    "backend": {
        "type": "MediaVideo",
        "shape": [1000, 480, 640, 3],
        "filename": "path/to/video.mp4",
        "grayscale": false,
        "bgr": true,
        "fps": 30.0
    },
    "source_video": null
}

Backend Types¶

Type	Description	Key Fields
`MediaVideo`	Standard video files (mp4, avi, mov, etc.)	`filename`, `bgr`, `fps`
`HDF5Video`	Embedded frames in HDF5	`dataset`, `input_format`, `has_embedded_images`
`ImageVideo`	Image sequences	`filename`, `filenames` (list)
`TiffVideo`	TIFF stacks	`filename`, `keep_open`

Source Video Lineage¶

Videos can have a source_video field that tracks the original video when frames are embedded:

{
    "filename": ".",
    "backend": {
        "type": "HDF5Video",
        "dataset": "video0/video"
    },
    "source_video": {
        "filename": "original.mp4",
        "backend": { ... }
    }
}

This creates a chain of provenance, allowing the original video to be restored when extracting embedded data.

Embedded Images¶

Frames can be embedded directly in SLP files for portability. Embedded frames are stored in /video{N}/ groups.

Encoding Formats¶

Format	Storage	Compression	Notes
`png`	`int8[]`	Lossless PNG	Default, best quality
`jpg`	`int8[]`	Lossy JPEG	Smaller files, some quality loss
`hdf5`	Raw array	Optional gzip	No encoding overhead, large files

Frame Selection Modes¶

When saving with embedded frames, the embed parameter controls which frames to include:

Mode	Description
`None`	Re-embed existing embedded frames
`True` / `"all"`	All labeled frames and suggestions
`"user"`	Only user-labeled frames
`"suggestions"`	Only suggested frames
`"user+suggestions"`	Both user and suggested frames
`"source"`	No embedding, restore source video
`list[(Video, int)]`	Custom list of (video, frame_idx) pairs

Channel Order¶

The channel_order attribute (introduced in format 1.4) tracks the color channel ordering:

OpenCV encoding: BGR ("BGR")
imageio encoding: RGB ("RGB")
Raw HDF5 arrays: RGB ("RGB")

This ensures correct color reproduction when reading embedded frames.

Skeletons¶

Skeleton definitions are stored in the /metadata group's json attribute. The metadata JSON contains both a global node list and skeleton definitions that reference Nodes by index.

JSON Format (SLP Files)¶

{
    "version": "2.0.0",
    "skeletons": [
        {
            "directed": true,
            "graph": {"name": "Skeleton-0", "num_edges_inserted": 5},
            "links": [
                {"source": 0, "target": 1, "type": {"py/reduce": [{"py/type": "sleap.skeleton.EdgeType"}, {"py/tuple": [1]}]}},
                {"source": 1, "target": 2, "type": {"py/id": 1}}
            ],
            "nodes": [{"id": 0}, {"id": 1}, {"id": 2}]
        }
    ],
    "nodes": [
        {"name": "head", "weight": 1.0},
        {"name": "thorax", "weight": 1.0},
        {"name": "abdomen", "weight": 1.0}
    ]
}

Edge Types¶

Edges are encoded with a type field using Python's pickle-style encoding:

Type ID	Meaning	Description
`1`	Regular edge	Skeletal connection between nodes
`2`	Symmetry edge	Bilateral symmetry relationship

The first occurrence of each type uses the full py/reduce encoding; subsequent occurrences use py/id references.

Symmetries¶

Symmetry relationships define bilateral pairings (e.g., left/right body parts). They are stored as edges with type 2:

{
    "source": 3,
    "target": 4,
    "type": {"py/reduce": [{"py/type": "sleap.skeleton.EdgeType"}, {"py/tuple": [2]}]}
}

Symmetry Deduplication

Legacy SLEAP files may store symmetries bidirectionally. The decoder automatically deduplicates them.

YAML Format¶

In addition to the JSON format stored in SLP files, sleap-io supports a simplified YAML format for Skeleton definitions. This format is more human-readable and easier to edit manually.

Preferred Format

The YAML format is the preferred format for skeleton definitions in sleap-nn and new tooling. The JSON format will be maintained for backwards compatibility with existing SLP files.

Structure¶

skeleton_name:
  nodes:
    - name: head
    - name: thorax
    - name: abdomen
  edges:
    - source:
        name: head
      destination:
        name: thorax
    - source:
        name: thorax
      destination:
        name: abdomen
  symmetries:
    - - name: left_wing
      - name: right_wing

Fields¶

Field	Type	Description
`nodes`	`list[dict]`	List of `Node` definitions with `name` key
`edges`	`list[dict]`	List of `Edge` definitions with `source` and `destination`
`symmetries`	`list[list]`	List of `Symmetry` pairs, each as a list of two node references

Multiple Skeletons¶

The YAML format supports multiple skeletons in a single file, with skeleton names as top-level keys:

fly:
  nodes:
    - name: head
    - name: thorax
  edges:
    - source: { name: head }
      destination: { name: thorax }
  symmetries: []

mouse:
  nodes:
    - name: nose
    - name: spine
  edges:
    - source: { name: nose }
      destination: { name: spine }
  symmetries: []

API Functions¶

Use the following functions to work with YAML skeletons:

encode_yaml_skeleton - Encode skeleton(s) to YAML string
decode_yaml_skeleton - Decode skeleton(s) from YAML string or file

Metadata¶

The /metadata group stores format information and serialized metadata about the labels.

Attributes¶

Attribute	Type	Description
`format_id`	`float`	Format version (e.g., `1.4`)
`json`	`bytes`	JSON-encoded metadata string

JSON Structure¶

The json attribute contains a JSON object with the following fields:

{
    "version": "2.0.0",
    "skeletons": [...],
    "nodes": [...],
    "videos": [],
    "tracks": [],
    "suggestions": [],
    "negative_anchors": {},
    "provenance": {
        "sleap_version": "1.3.4",
        "filename": "labels.slp"
    }
}

Field	Type	Description
`version`	`string`	SLEAP software version
`skeletons`	`array`	Skeleton graph definitions (see Skeletons)
`nodes`	`array`	Node definitions with names and weights
`videos`	`array`	Empty (stored in `/videos_json` dataset)
`tracks`	`array`	Empty (stored in `/tracks_json` dataset)
`suggestions`	`array`	Empty (stored in `/suggestions_json` dataset)
`negative_anchors`	`object`	Negative sample anchors for training
`provenance`	`object`	File origin and creation metadata

Provenance¶

The provenance field tracks the origin and history of the labels file:

Key	Type	Description
`sleap_version`	`string`	SLEAP version that created the file
`filename`	`string`	Original filename
custom	`any`	Additional user-defined provenance data

Custom Provenance

The provenance dictionary can contain arbitrary key-value pairs for tracking custom metadata like model training runs, data sources, or processing history.

Tracks¶

Tracks enable identity tracking of individual animals across frames. Track metadata is stored in the /tracks_json dataset as an array of JSON strings.

JSON Structure¶

Each track is stored as a two-element JSON array:

[0, "track_name"]

Index	Type	Description
`0`	`int`	Spawned frame index (reserved, currently always `0`)
`1`	`string`	Track name for identification

Example¶

[0, "mouse_1"]
[0, "mouse_2"]
[0, "female"]

Instance Linking¶

Instances reference tracks by index in the /instances dataset:

track = 0 → First track in /tracks_json
track = 1 → Second track in /tracks_json
track = -1 → Untracked instance

Track Identity

Tracks are compared by object identity, not name. Two tracks with the same name are considered different unless they are the same object. This allows multiple tracks to share a name if needed.

Suggestions¶

Suggestions indicate frames that should be labeled, typically generated by active learning algorithms or manual selection. Suggestion metadata is stored in the /suggestions_json dataset.

JSON Structure¶

Each suggestion is stored as a JSON object:

{
    "video": "0",
    "frame_idx": 42,
    "group": 0
}

Field	Type	Description
`video`	`string`	Video index (as string)
`frame_idx`	`int`	Frame index within the video
`group`	`int`	Suggestion group ID (default: `0`)

Groups¶

The group field enables organizing suggestions into batches, useful for:

Separating suggestions by generation method
Tracking labeling progress across multiple sessions
Grouping frames by difficulty or priority

Optional Dataset¶

The /suggestions_json dataset is optional. Files without suggestions will not contain this dataset, and the reader returns an empty list when it's missing.

Sessions¶

Recording sessions store multi-camera calibration data and synchronized frame groups. Session metadata is stored in the /sessions_json dataset.

JSON Structure¶

Each session is stored as a JSON object:

{
    "calibration": {
        "cam_0": {
            "name": "Camera 1",
            "size": [1080, 1920],
            "matrix": [[...], [...], [...]],
            "distortions": [...],
            "rotation": [...],
            "translation": [...]
        },
        "cam_1": {...},
        "metadata": {}
    },
    "camcorder_to_video_idx_map": {
        "0": 0,
        "1": 1
    },
    "frame_group_dicts": [...]
}

Calibration¶

Camera calibration data is stored per camera with these fields:

Field	Type	Shape	Description
`name`	`string`	-	Camera identifier
`size`	`int[]`	`(2,)`	Image dimensions `[height, width]`
`matrix`	`float[][]`	`(3, 3)`	Intrinsic camera matrix
`distortions`	`float[]`	`(5,)`	Radial-tangential distortion coefficients `[k1, k2, p1, p2, k3]`
`rotation`	`float[]`	`(3,)`	Rotation vector (axis-angle representation)
`translation`	`float[]`	`(3,)`	Translation vector

Camera-Video Mapping¶

The camcorder_to_video_idx_map object maps camera indices to video indices in /videos_json:

{
    "0": 0,
    "1": 1
}

This links each camera in the calibration to its corresponding video.

Frame Groups¶

Frame groups synchronize labeled frames across multiple cameras at the same time point. Each frame group contains:

A frame index identifying the synchronized time point
Instance groups linking instances across camera views
References to LabeledFrame objects by index

Optional Dataset¶

The /sessions_json dataset is optional. Files without multi-camera sessions will not contain this dataset.

Instances¶

Instances represent individual animals or objects in a frame. They are stored in the /instances dataset as a structured array.

Instance Dtype¶

instance_dtype = np.dtype([
    ("instance_id", "i8"),        # Unique instance identifier
    ("instance_type", "u1"),      # 0=USER, 1=PREDICTED
    ("frame_id", "u8"),           # Index into frames dataset
    ("skeleton", "u4"),           # Index into skeletons list
    ("track", "i4"),              # Index into tracks list (-1 if untracked)
    ("from_predicted", "i8"),     # Parent prediction ID (-1 if none)
    ("score", "f4"),              # Prediction score (0.0 for user)
    ("point_id_start", "u8"),     # Start index in points array
    ("point_id_end", "u8"),       # End index (exclusive)
    ("tracking_score", "f4"),     # Tracking confidence (format >= 1.2)
])

Instance Types¶

Type	Value	Data Model Class	Description
`USER`	`0`	`Instance`	User-labeled annotation
`PREDICTED`	`1`	`PredictedInstance`	Model prediction

Instance Linking¶

The from_predicted field links user Instances to their source PredictedInstances, enabling tracking of corrections made to model outputs.

Points¶

Point coordinates are stored in separate datasets for user-labeled and predicted instances.

User Points (`/points`)¶

point_dtype = np.dtype([
    ("x", "f8"),        # X coordinate
    ("y", "f8"),        # Y coordinate
    ("visible", "?"),   # Is point visible
    ("complete", "?"),  # Is point marked complete
])

Predicted Points (`/pred_points`)¶

predicted_point_dtype = np.dtype([
    ("x", "f8"),        # X coordinate
    ("y", "f8"),        # Y coordinate
    ("visible", "?"),   # Is point visible
    ("complete", "?"),  # Is point marked complete
    ("score", "f8"),    # Prediction confidence
])

Coordinate System¶

Coordinate System Change

Format 1.1 changed the coordinate system from pixel corner to pixel center.

Format	Origin	Notes
< 1.1	Top-left corner of pixel at (0, 0)	Legacy
>= 1.1	Center of pixel at (0, 0)	Current

When reading format < 1.1 files, the reader applies a -0.5 offset to convert coordinates.

Labeled Frames¶

LabeledFrames are stored in the /frames dataset, linking video frames to their instances.

Frame Dtype¶

frame_dtype = np.dtype([
    ("frame_id", "u8"),           # Unique frame identifier
    ("video", "u4"),              # Video index or sparse video ID
    ("frame_idx", "u8"),          # Frame index within video
    ("instance_id_start", "u8"),  # Start index in instances array
    ("instance_id_end", "u8"),    # End index (exclusive)
])

Video ID Mapping¶

Modern SLP files use sequential video indices (0, 1, 2, ...), but legacy files may contain sparse video IDs derived from the embedded video group names (e.g., 0, 15, 29). The reader handles both cases transparently.

Lazy Loading¶

For large SLP files with hundreds of thousands of frames, sleap-io provides a lazy loading mode that defers Labels object creation until needed.

Architecture¶

Labels (lazy mode)
├── LazyDataStore
│   ├── frames_data (numpy array)
│   ├── instances_data (numpy array)
│   ├── points_data (numpy array)
│   └── pred_points_data (numpy array)
└── LazyFrameList
    └── Materializes frames on-demand

Performance Benefits¶

Operation	Eager	Lazy	Speedup
Load file	~0.5s	~0.005s	~100x
Load + numpy()	~0.9s	~0.4s	~2x
Full iteration	~0.0002s	~0.4s	Eager faster

Benchmarks on 18,000 frames with ~40,000 instances.

When to Use Lazy Loading¶

Recommended for:

Converting to NumPy arrays (labels.numpy())
Saving to another file without modifications
Accessing a small subset of frames
Quick metadata inspection

Not recommended for:

Iterating over all frames (eager is faster)
Modifying data (must materialize first)
Multiple passes over the data

Fast Paths¶

Lazy labels support optimized code paths:

numpy() conversion: Builds arrays directly from raw HDF5 data without creating Python objects
Saving: Copies raw arrays directly without materialization (when embed is None, False, or "source")
Metadata queries: Properties like n_user_instances, n_pred_instances use O(1) array operations

Usage¶

import sleap_io as sio

# Load lazily
labels = sio.load_slp("predictions.slp", lazy=True)

# Check lazy state
print(labels.is_lazy)  # True

# Fast numpy conversion
poses = labels.numpy()

# Materialize when modifications needed
labels = labels.materialize()
labels.append(new_frame)  # Now works

Version History¶

The SLP format has evolved through several versions, tracked by the format_id attribute in /metadata.

Format 1.0¶

Initial release format.

Format 1.1¶

Coordinate system change: Changed from top-left pixel corner at (0, 0) to pixel center at (0, 0).

Reading: Applies -0.5 offset to coordinates from older files
Writing: Always uses new coordinate system

Format 1.2¶

Added tracking_score field to instance dtype.

Instance dtype expanded from 9 to 10 fields
tracking_score stores tracking confidence for multi-animal workflows
Reading: Defaults to 0.0 for older files

Format 1.3¶

Minor handling improvements for tracking_score (no schema change from 1.2).

Format 1.4 (Current)¶

Added channel_order attribute to embedded video datasets.

Tracks RGB vs BGR channel ordering for embedded images
Ensures correct color reproduction across different encoding backends
Reading: Defaults to RGB if attribute missing

API¶

High-Level Functions¶

`sleap_io.io.main.load_slp(filename, open_videos=True, lazy=False)` ¶

Load a SLEAP dataset.

Parameters:

Name	Type	Description	Default
`filename`	`str`	Path to a SLEAP labels file (`.slp`).	required
`open_videos`	`bool`	If `True` (the default), attempt to open the video backend for I/O. If `False`, the backend will not be opened (useful for reading metadata when the video files are not available).	`True`
`lazy`	`bool`	If `True`, defer instance materialization for faster loading. Lazy-loaded Labels support read operations and fast numpy/save. To modify, call `labels.materialize()` first. Default is `False`.	`False`

Returns:

Type	Description
`Labels`	The dataset as a `Labels` object.

`sleap_io.io.main.save_slp(labels, filename, embed=False, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, progress_callback=None)` ¶

Save a SLEAP dataset to a .slp file.

Parameters:

Name	Type	Description	Default
`labels`	`Labels`	A SLEAP `Labels` object (see `load_slp`).	required
`filename`	`str`	Path to save labels to ending with `.slp`.	required
`embed`	`bool \| str \| list[tuple[Video, int]] \| None`	Frames to embed in the saved labels file. One of `None`, `True`, `"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list of tuples of `(video, frame_idx)`. If `False` is specified (the default), the source video will be restored if available, otherwise the embedded frames will be re-saved. If `True` or `"all"`, all labeled frames and suggested frames will be embedded. If `"source"` is specified, no images will be embedded and the source video will be restored if available. This argument is only valid for the SLP backend.	`False`
`restore_original_videos`	`bool`	If `True` (default) and `embed=False`, use original video files. If `False` and `embed=False`, keep references to source `.pkg.slp` files. Only applies when `embed=False`.	`True`
`embed_inplace`	`bool`	If `False` (default), a copy of the labels is made before embedding to avoid modifying the in-memory labels. If `True`, the labels will be modified in-place to point to the embedded videos, which is faster but mutates the input. Only applies when embedding.	`False`
`verbose`	`bool`	If `True` (the default), display a progress bar when embedding frames.	`True`
`plugin`	`str \| None`	Image plugin to use for encoding embedded frames. One of "opencv" or "imageio". If None, uses the global default from `get_default_image_plugin()`. If no global default is set, auto-detects based on available packages (opencv preferred, then imageio).	`None`
`progress_callback`	`Callable[[int, int], bool] \| None`	Optional callback function called during frame embedding with `(current, total)` arguments. If it returns `False`, the operation is cancelled and `ExportCancelled` is raised. When provided, tqdm progress bar is disabled in favor of the callback.	`None`

Source code in sleap_io/io/main.py

def save_slp(
    labels: Labels,
    filename: str,
    embed: bool | str | list[tuple[Video, int]] | None = False,
    restore_original_videos: bool = True,
    embed_inplace: bool = False,
    verbose: bool = True,
    plugin: str | None = None,
    progress_callback: Callable[[int, int], bool] | None = None,
):
    """Save a SLEAP dataset to a `.slp` file.

    Args:
        labels: A SLEAP `Labels` object (see `load_slp`).
        filename: Path to save labels to ending with `.slp`.
        embed: Frames to embed in the saved labels file. One of `None`, `True`,
            `"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list
            of tuples of `(video, frame_idx)`.

            If `False` is specified (the default), the source video will be restored
            if available, otherwise the embedded frames will be re-saved.

            If `True` or `"all"`, all labeled frames and suggested frames will be
            embedded.

            If `"source"` is specified, no images will be embedded and the source video
            will be restored if available.

            This argument is only valid for the SLP backend.
        restore_original_videos: If `True` (default) and `embed=False`, use original
            video files. If `False` and `embed=False`, keep references to source
            `.pkg.slp` files. Only applies when `embed=False`.
        embed_inplace: If `False` (default), a copy of the labels is made before
            embedding to avoid modifying the in-memory labels. If `True`, the
            labels will be modified in-place to point to the embedded videos,
            which is faster but mutates the input. Only applies when embedding.
        verbose: If `True` (the default), display a progress bar when embedding frames.
        plugin: Image plugin to use for encoding embedded frames. One of "opencv"
            or "imageio". If None, uses the global default from
            `get_default_image_plugin()`. If no global default is set, auto-detects
            based on available packages (opencv preferred, then imageio).
        progress_callback: Optional callback function called during frame embedding
            with `(current, total)` arguments. If it returns `False`, the operation
            is cancelled and `ExportCancelled` is raised. When provided, tqdm
            progress bar is disabled in favor of the callback.
    """
    from sleap_io.io import slp

    return slp.write_labels(
        filename,
        labels,
        embed=embed,
        restore_original_videos=restore_original_videos,
        embed_inplace=embed_inplace,
        verbose=verbose,
        plugin=plugin,
        progress_callback=progress_callback,
    )

Core Module¶

`sleap_io.io.slp.read_labels(labels_path, open_videos=True)` ¶

Read a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file.	required
`open_videos`	`bool`	If `True` (the default), attempt to open the video backend for I/O. If `False`, the backend will not be opened (useful for reading metadata when the video files are not available).	`True`

Returns:

Type	Description
`Labels`	The processed `Labels` object.

Source code in sleap_io/io/slp.py

def read_labels(labels_path: str, open_videos: bool = True) -> Labels:
    """Read a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file.
        open_videos: If `True` (the default), attempt to open the video backend for
            I/O. If `False`, the backend will not be opened (useful for reading metadata
            when the video files are not available).

    Returns:
        The processed `Labels` object.
    """
    tracks = read_tracks(labels_path)
    videos = read_videos(labels_path, open_backend=open_videos)
    skeletons = read_skeletons(labels_path)
    points = read_points(labels_path)
    pred_points = read_pred_points(labels_path)
    format_id = read_hdf5_attrs(labels_path, "metadata", "format_id")
    instances = read_instances(
        labels_path, skeletons, tracks, points, pred_points, format_id
    )
    suggestions = read_suggestions(labels_path, videos)
    metadata = read_metadata(labels_path)
    provenance = metadata.get("provenance", dict())

    frames = read_hdf5_dataset(labels_path, "frames")

    # Check if video IDs in frames are sequential list indices (0, 1, 2, ..., n-1)
    # or sparse embedded IDs (e.g., 0, 15, 29, 47, ...) that need remapping
    frame_video_ids = set(int(frame[1]) for frame in frames)
    max_frame_video_id = max(frame_video_ids) if frame_video_ids else 0

    # If max video ID == len(videos) - 1 and IDs are contiguous, they're list indices
    # In this case, use identity mapping (backwards compatible behavior)
    frames_use_list_indices = (
        len(frame_video_ids) == len(videos) and max_frame_video_id == len(videos) - 1
    )

    if frames_use_list_indices:
        # Video IDs are sequential list indices - use identity mapping
        video_id_to_index = {i: i for i in range(len(videos))}
    else:
        # Build mapping from sparse video IDs to list indices
        # This handles files from old SLEAP where video IDs can be sparse
        # (e.g., 0, 15, 29, 47, ...) rather than sequential (0, 1, 2, 3, ...)
        video_id_to_index = {}
        for i, video in enumerate(videos):
            # For embedded videos, extract the video ID from backend.dataset
            if (
                hasattr(video, "backend")
                and video.backend is not None
                and hasattr(video.backend, "dataset")
                and video.backend.dataset is not None
            ):
                dataset = video.backend.dataset
                # Extract video ID from dataset name (e.g., "video15/video" → 15)
                if "/" in dataset:
                    video_group = dataset.split("/")[0]
                    if video_group.startswith("video"):
                        video_id_str = video_group[5:]  # Remove "video" prefix
                        if video_id_str.isdigit():
                            video_id = int(video_id_str)
                            video_id_to_index[video_id] = i
                            continue

            # For non-embedded videos or videos without extractable IDs,
            # assume sequential indexing (backwards compatible behavior)
            video_id_to_index[i] = i

    labeled_frames = []
    for _, video_id, frame_idx, instance_id_start, instance_id_end in frames:
        # Map sparse video_id to sequential list index
        video_index = video_id_to_index.get(video_id, video_id)

        labeled_frames.append(
            LabeledFrame(
                video=videos[video_index],
                frame_idx=int(frame_idx),
                instances=instances[instance_id_start:instance_id_end],
            )
        )

    sessions = read_sessions(labels_path, videos, labeled_frames)

    labels = Labels(
        labeled_frames=labeled_frames,
        videos=videos,
        skeletons=skeletons,
        tracks=tracks,
        suggestions=suggestions,
        sessions=sessions,
        provenance=provenance,
    )
    labels.provenance["filename"] = labels_path

    return labels

`sleap_io.io.slp.write_labels(labels_path, labels, embed=None, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, embed_all_videos=True, progress_callback=None)` ¶

Write a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file to save.	required
`labels`	`Labels`	A `Labels` object to save.	required
`embed`	`bool \| str \| list[tuple[Video, int]] \| None`	Frames to embed in the saved labels file. One of `None`, `True`, `"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list of tuples of `(video, frame_idx)`. If `None` is specified (the default) and the labels contains embedded frames, those embedded frames will be re-saved to the new file. If `True` or `"all"`, all labeled frames and suggested frames will be embedded. If `"source"` is specified, no images will be embedded and the source video will be restored if available. This argument is only valid for the SLP backend.	`None`
`restore_original_videos`	`bool`	If `True` (default) and `embed=False`, use original video files. If `False` and `embed=False`, keep references to source `.pkg.slp` files. Only applies when `embed=False`.	`True`
`embed_inplace`	`bool`	If `False` (default), a copy of the labels is made before embedding to avoid modifying the in-memory labels. If `True`, the labels will be modified in-place to point to the embedded videos, which is faster but mutates the input. Only applies when embedding.	`False`
`verbose`	`bool`	If `True` (the default), display a progress bar when embedding frames.	`True`
`plugin`	`str \| None`	Image plugin to use for encoding embedded frames. One of "opencv" or "imageio". If None, uses the global default from `get_default_image_plugin()`. If no global default is set, auto-detects based on available packages.	`None`
`embed_all_videos`	`bool`	If `True` (the default), all videos in the labels will be converted to embedded references, even if they have no frames to embed. This ensures package files are portable. If `False`, only videos with frames to embed are converted.	`True`
`progress_callback`	`Callable[[int, int], bool] \| None`	Optional callback function called during frame embedding with `(current, total)` arguments. If it returns `False`, the operation is cancelled and `ExportCancelled` is raised.	`None`

Source code in sleap_io/io/slp.py

def write_labels(
    labels_path: str,
    labels: Labels,
    embed: bool | str | list[tuple[Video, int]] | None = None,
    restore_original_videos: bool = True,
    embed_inplace: bool = False,
    verbose: bool = True,
    plugin: str | None = None,
    embed_all_videos: bool = True,
    progress_callback: Callable[[int, int], bool] | None = None,
):
    """Write a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file to save.
        labels: A `Labels` object to save.
        embed: Frames to embed in the saved labels file. One of `None`, `True`,
            `"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list
            of tuples of `(video, frame_idx)`.

            If `None` is specified (the default) and the labels contains embedded
            frames, those embedded frames will be re-saved to the new file.

            If `True` or `"all"`, all labeled frames and suggested frames will be
            embedded.

            If `"source"` is specified, no images will be embedded and the source video
            will be restored if available.

            This argument is only valid for the SLP backend.
        restore_original_videos: If `True` (default) and `embed=False`, use original
            video files. If `False` and `embed=False`, keep references to source
            `.pkg.slp` files. Only applies when `embed=False`.
        embed_inplace: If `False` (default), a copy of the labels is made before
            embedding to avoid modifying the in-memory labels. If `True`, the
            labels will be modified in-place to point to the embedded videos,
            which is faster but mutates the input. Only applies when embedding.
        verbose: If `True` (the default), display a progress bar when embedding frames.
        plugin: Image plugin to use for encoding embedded frames. One of "opencv"
            or "imageio". If None, uses the global default from
            `get_default_image_plugin()`. If no global default is set, auto-detects
            based on available packages.
        embed_all_videos: If `True` (the default), all videos in the labels will be
            converted to embedded references, even if they have no frames to embed.
            This ensures package files are portable. If `False`, only videos with
            frames to embed are converted.
        progress_callback: Optional callback function called during frame embedding
            with `(current, total)` arguments. If it returns `False`, the operation
            is cancelled and `ExportCancelled` is raised.
    """
    # Fast path for lazy labels (avoids materializing frames/instances)
    # Supported for simple embed modes: None, False, "source"
    if labels.is_lazy:
        # Check if embed mode requires materialization
        needs_materialization = (
            embed is True
            or embed
            in (
                "all",
                "user",
                "suggestions",
                "user+suggestions",
            )
            or isinstance(embed, list)
        )

        if needs_materialization:
            # Materialize to support embedding
            labels = labels.materialize()
        else:
            # Use fast path - copy raw arrays directly
            _write_labels_lazy(
                labels_path,
                labels,
                embed=embed,
                restore_original_videos=restore_original_videos,
                verbose=verbose,
            )
            return

    if Path(labels_path).exists():
        Path(labels_path).unlink()

    # Make a copy to avoid mutating the input labels when embedding
    if embed and not embed_inplace:
        original_labels = labels
        labels = labels.copy(open_videos=True)

        # If embed is a list of (video, frame_idx) tuples, remap videos to the copy
        if isinstance(embed, list):
            # Create mapping from original videos to copied videos
            video_map = {
                orig: copied
                for orig, copied in zip(original_labels.videos, labels.videos)
            }
            # Remap the embed list to use copied video objects
            embed = [
                (video_map.get(video, video), frame_idx) for video, frame_idx in embed
            ]

    # Store original videos before embedding modifies them
    # We need to make a copy of the actual video objects, not just the list
    original_videos = [v for v in labels.videos] if embed else None

    if embed:
        embed_videos(
            labels_path,
            labels,
            embed,
            verbose=verbose,
            plugin=plugin,
            embed_all_videos=embed_all_videos,
            progress_callback=progress_callback,
        )

    # Determine reference mode based on parameters
    if embed == "source" or (embed is False and restore_original_videos):
        reference_mode = VideoReferenceMode.RESTORE_ORIGINAL
    elif embed is False and not restore_original_videos:
        reference_mode = VideoReferenceMode.PRESERVE_SOURCE
    else:
        reference_mode = VideoReferenceMode.EMBED

    write_videos(
        labels_path,
        labels.videos,
        reference_mode=reference_mode,
        original_videos=original_videos,
        verbose=verbose,
    )
    write_tracks(labels_path, labels.tracks)
    write_suggestions(labels_path, labels.suggestions, labels.videos)
    write_sessions(labels_path, labels.sessions, labels.videos, labels.labeled_frames)
    write_metadata(labels_path, labels)
    write_lfs(labels_path, labels)

Video I/O¶

`sleap_io.io.slp.read_videos(labels_path, open_backend=True)` ¶

Read Video dataset in a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file.	required
`open_backend`	`bool`	If `True` (the default), attempt to open the video backend for I/O. If `False`, the backend will not be opened (useful for reading metadata when the video files are not available).	`True`

Returns:

Type	Description
`list[Video]`	A list of `Video` objects.

Source code in sleap_io/io/slp.py

def read_videos(labels_path: str, open_backend: bool = True) -> list[Video]:
    """Read `Video` dataset in a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file.
        open_backend: If `True` (the default), attempt to open the video backend for
            I/O. If `False`, the backend will not be opened (useful for reading metadata
            when the video files are not available).

    Returns:
        A list of `Video` objects.
    """
    videos = []
    # Open file once and pass handle to make_video to avoid repeated opens
    # for embedded videos (which would otherwise open the file per video).
    with h5py.File(labels_path, "r") as f:
        videos_metadata = f["videos_json"][:]
        for video_data in videos_metadata:
            video_json = json.loads(video_data)
            video = make_video(
                labels_path, video_json, open_backend=open_backend, _hdf5_file=f
            )
            videos.append(video)
    return videos

`sleap_io.io.slp.write_videos(labels_path, videos, restore_source=False, reference_mode=None, original_videos=None, verbose=True)` ¶

Write video metadata to a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file.	required
`videos`	`list[Video]`	A list of `Video` objects to store the metadata for.	required
`restore_source`	`bool`	Deprecated. Use reference_mode instead. If `True`, restore source videos if available and will not re-embed the embedded images. If `False` (the default), will re-embed images that were previously embedded.	`False`
`reference_mode`	`VideoReferenceMode \| None`	How to handle video references: - EMBED: Re-embed frames that were previously embedded - RESTORE_ORIGINAL: Use original video if available - PRESERVE_SOURCE: Keep reference to source file (e.g., .pkg.slp)	`None`
`original_videos`	`list[Video] \| None`	Optional list of original video objects before embedding. Used when reference_mode is EMBED to preserve metadata.	`None`
`verbose`	`bool`	If `True` (the default), display a progress bar when embedding frames.	`True`

Source code in sleap_io/io/slp.py

def write_videos(
    labels_path: str,
    videos: list[Video],
    restore_source: bool = False,
    reference_mode: VideoReferenceMode | None = None,
    original_videos: list[Video] | None = None,
    verbose: bool = True,
):
    """Write video metadata to a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file.
        videos: A list of `Video` objects to store the metadata for.
        restore_source: Deprecated. Use reference_mode instead. If `True`, restore
            source videos if available and will not re-embed the embedded images.
            If `False` (the default), will re-embed images that were previously
            embedded.
        reference_mode: How to handle video references:
            - EMBED: Re-embed frames that were previously embedded
            - RESTORE_ORIGINAL: Use original video if available
            - PRESERVE_SOURCE: Keep reference to source file (e.g., .pkg.slp)
        original_videos: Optional list of original video objects before embedding.
            Used when reference_mode is EMBED to preserve metadata.
        verbose: If `True` (the default), display a progress bar when embedding frames.
    """
    # Handle backwards compatibility
    if reference_mode is None:
        if restore_source:
            reference_mode = VideoReferenceMode.RESTORE_ORIGINAL
        else:
            reference_mode = VideoReferenceMode.EMBED

    videos_to_embed = []
    videos_to_write = []
    videos_to_copy = []  # For embedded videos without backend (raw HDF5 copy)

    # First determine which videos need embedding
    for video_ind, video in enumerate(videos):
        # Check if video has an open backend with embedded images
        has_backend_with_embedded = (
            type(video.backend) is HDF5Video and video.backend.has_embedded_images
        )
        # Also detect embedded videos via metadata (when backend is None)
        has_embedded_via_metadata = (
            video.backend is None and _is_embedded_video_metadata(video)
        )

        if has_backend_with_embedded:
            if reference_mode == VideoReferenceMode.RESTORE_ORIGINAL:
                if video.source_video is None:
                    # No source video available, reference the current embedded video
                    # file
                    videos_to_write.append((video_ind, video))
                else:
                    # Use the source video
                    videos_to_write.append((video_ind, video.source_video))
            elif reference_mode == VideoReferenceMode.PRESERVE_SOURCE:
                # Keep the reference to the source .pkg.slp file
                videos_to_write.append((video_ind, video))
            else:  # EMBED mode
                # If the video has embedded images, check if we need to re-embed them
                already_embedded = False
                if Path(labels_path).exists():
                    with h5py.File(labels_path, "r") as f:
                        already_embedded = f"video{video_ind}/video" in f

                if already_embedded:
                    videos_to_write.append((video_ind, video))
                else:
                    # Collect information for embedding
                    frames_to_embed = [
                        (video, frame_idx) for frame_idx in video.backend.source_inds
                    ]
                    videos_to_embed.append((video_ind, video, frames_to_embed))
        elif has_embedded_via_metadata:
            # Video has embedded frames but backend is not open (open_videos=False)
            if reference_mode == VideoReferenceMode.RESTORE_ORIGINAL:
                if video.source_video is None:
                    videos_to_write.append((video_ind, video))
                else:
                    videos_to_write.append((video_ind, video.source_video))
            elif reference_mode == VideoReferenceMode.PRESERVE_SOURCE:
                videos_to_write.append((video_ind, video))
            else:  # EMBED mode
                # Check if already embedded in destination
                already_embedded = False
                if Path(labels_path).exists():
                    with h5py.File(labels_path, "r") as f:
                        already_embedded = f"video{video_ind}/video" in f

                if already_embedded:
                    videos_to_write.append((video_ind, video))
                else:
                    # Need to copy raw HDF5 data from source file
                    videos_to_copy.append((video_ind, video))
        else:
            videos_to_write.append((video_ind, video))

    # Process videos that need embedding
    if videos_to_embed:
        # Prepare all frames to embed
        all_frames_to_embed = []
        for video_ind, video, frames in videos_to_embed:
            for frame in frames:
                all_frames_to_embed.append(frame)

        # Create a temporary Labels object for embedding
        temp_labels = Labels(
            videos=[v for _, v, _ in videos_to_embed], labeled_frames=[]
        )

        # Prepare and embed all frames in a single process
        frames_metadata = prepare_frames_to_embed(
            labels_path, temp_labels, all_frames_to_embed
        )
        replaced_videos = process_and_embed_frames(
            labels_path,
            frames_metadata,
            image_format=[
                v.backend.image_format if hasattr(v.backend, "image_format") else "png"
                for _, v, _ in videos_to_embed
            ][0],  # Use the first video's format
            verbose=verbose,
        )

        # Add the embedded videos to the list
        for video_ind, video, _ in videos_to_embed:
            if video in replaced_videos:
                videos_to_write.append((video_ind, replaced_videos[video]))

    # Copy raw HDF5 data for embedded videos without backends
    if videos_to_copy:
        for video_ind, video in videos_to_copy:
            # Get the source file path (video.filename points to the source pkg.slp)
            source_path = video.filename
            if not Path(source_path).exists():
                # Can't copy if source doesn't exist, just write metadata
                videos_to_write.append((video_ind, video))
                continue

            # Get the source dataset name from backend_metadata
            meta = video.backend_metadata
            source_dataset = meta.get("dataset", "") if meta else ""
            if not source_dataset:
                videos_to_write.append((video_ind, video))
                continue

            # Extract the video group name (e.g., "video0" from "video0/video")
            source_group = source_dataset.split("/")[0] if "/" in source_dataset else ""
            if not source_group:
                videos_to_write.append((video_ind, video))
                continue

            # Destination group name uses the current video index
            dest_group = f"video{video_ind}"

            # Copy the entire video group from source to destination
            with h5py.File(source_path, "r") as src_f:
                if source_group not in src_f:
                    videos_to_write.append((video_ind, video))
                    continue

                with h5py.File(labels_path, "a") as dst_f:
                    # Copy the video group with all its datasets and attributes
                    src_f.copy(source_group, dst_f, name=dest_group)

            # Add to videos_to_write - the metadata will reference the copied data
            videos_to_write.append((video_ind, video))

    # Write video metadata
    video_jsons = []
    for video_ind, video in sorted(videos_to_write, key=lambda x: x[0]):
        video_json = video_to_dict(video, labels_path)
        video_jsons.append(np.bytes_(json.dumps(video_json, separators=(",", ":"))))

    with h5py.File(labels_path, "a") as f:
        if "videos_json" not in f:
            f.create_dataset("videos_json", data=video_jsons, maxshape=(None,))

    # Save source_video lineage metadata in a separate pass to ensure video groups exist
    # Note: original_video is now a computed property derived from source_video chain,
    # so we only need to store source_video (immediate parent).
    with h5py.File(labels_path, "a") as f:
        for video_ind, video in enumerate(videos):
            dataset = f"video{video_ind}"

            # If original_videos is provided (e.g., during embedding), use those
            pre_embed_video = original_videos[video_ind] if original_videos else video

            # Determine source_video to save based on reference mode
            source_to_save = None
            if reference_mode != VideoReferenceMode.PRESERVE_SOURCE:
                if reference_mode == VideoReferenceMode.EMBED and original_videos:
                    # For embed mode, save the pre-embedding video as source
                    source_to_save = pre_embed_video
                elif pre_embed_video.source_video is not None:
                    source_to_save = pre_embed_video.source_video

            # Write source_video metadata to the video group
            if dataset in f and source_to_save is not None:
                video_group = f[dataset]

                # For EMBED mode with original_videos, we need to overwrite
                # source_video because embed_videos saves the wrong metadata
                if (
                    reference_mode == VideoReferenceMode.EMBED
                    and original_videos
                    and "source_video" in video_group
                ):
                    del video_group["source_video"]

                if "source_video" not in video_group:
                    source_grp = video_group.require_group("source_video")
                    source_json = video_to_dict(source_to_save, labels_path)
                    source_grp.attrs["json"] = json.dumps(
                        source_json, separators=(",", ":")
                    )

`sleap_io.io.slp.embed_videos(labels_path, labels, embed, verbose=True, plugin=None, embed_all_videos=True, progress_callback=None)` ¶

Embed videos in a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file to save.	required
`labels`	`Labels`	A `Labels` object to save.	required
`embed`	`bool \| str \| list[tuple[Video, int]]`	Frames to embed in the saved labels file. One of `None`, `True`, `"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list of tuples of `(video, frame_idx)`. If `None` is specified (the default) and the labels contains embedded frames, those embedded frames will be re-saved to the new file. If `True` or `"all"`, all labeled frames and suggested frames will be embedded.	required
`verbose`	`bool`	If `True` (the default), display a progress bar for the embedding process.	`True`
`plugin`	`str \| None`	Image plugin to use for encoding. One of "opencv" or "imageio". If None, uses the global default from `get_default_image_plugin()`. If `"source"` is specified, no images will be embedded and the source video will be restored if available. This argument is only valid for the SLP backend.	`None`
`embed_all_videos`	`bool`	If `True` (the default), all videos in the labels will be converted to embedded references, even if they have no frames to embed. This ensures package files are portable. If `False`, only videos with frames to embed are converted.	`True`
`progress_callback`	`Callable[[int, int], bool] \| None`	Optional callback function called during frame embedding with `(current, total)` arguments. If it returns `False`, the operation is cancelled and `ExportCancelled` is raised.	`None`

Source code in sleap_io/io/slp.py

def embed_videos(
    labels_path: str,
    labels: Labels,
    embed: bool | str | list[tuple[Video, int]],
    verbose: bool = True,
    plugin: str | None = None,
    embed_all_videos: bool = True,
    progress_callback: Callable[[int, int], bool] | None = None,
):
    """Embed videos in a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file to save.
        labels: A `Labels` object to save.
        embed: Frames to embed in the saved labels file. One of `None`, `True`,
            `"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list
            of tuples of `(video, frame_idx)`.

            If `None` is specified (the default) and the labels contains embedded
            frames, those embedded frames will be re-saved to the new file.

            If `True` or `"all"`, all labeled frames and suggested frames will be
            embedded.
        verbose: If `True` (the default), display a progress bar for the embedding
            process.
        plugin: Image plugin to use for encoding. One of "opencv" or "imageio".
            If None, uses the global default from `get_default_image_plugin()`.

            If `"source"` is specified, no images will be embedded and the source video
            will be restored if available.

            This argument is only valid for the SLP backend.
        embed_all_videos: If `True` (the default), all videos in the labels will be
            converted to embedded references, even if they have no frames to embed.
            This ensures package files are portable. If `False`, only videos with
            frames to embed are converted.
        progress_callback: Optional callback function called during frame embedding
            with `(current, total)` arguments. If it returns `False`, the operation
            is cancelled and `ExportCancelled` is raised.
    """
    if embed is True:
        embed = "all"
    if embed == "user":
        embed = [(lf.video, lf.frame_idx) for lf in labels.user_labeled_frames]
    elif embed == "suggestions":
        embed = [(sf.video, sf.frame_idx) for sf in labels.suggestions]
    elif embed == "user+suggestions":
        embed = [(lf.video, lf.frame_idx) for lf in labels.user_labeled_frames]
        embed += [(sf.video, sf.frame_idx) for sf in labels.suggestions]
    elif embed == "all":
        embed = [(lf.video, lf.frame_idx) for lf in labels]
        embed += [(sf.video, sf.frame_idx) for sf in labels.suggestions]
    elif embed == "source":
        embed = []
    elif isinstance(embed, list):
        embed = embed
    else:
        raise ValueError(f"Invalid value for embed: {embed}")

    embed_frames(
        labels_path,
        labels,
        embed,
        verbose=verbose,
        plugin=plugin,
        embed_all_videos=embed_all_videos,
        progress_callback=progress_callback,
    )

Skeleton I/O¶

`sleap_io.io.slp.read_skeletons(labels_path)` ¶

Read Skeleton dataset from a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string that contains the path to the labels file.	required

Returns:

Type	Description
`list[Skeleton]`	A list of `Skeleton` objects.

Source code in sleap_io/io/slp.py

def read_skeletons(labels_path: str) -> list[Skeleton]:
    """Read `Skeleton` dataset from a SLEAP labels file.

    Args:
        labels_path: A string that contains the path to the labels file.

    Returns:
        A list of `Skeleton` objects.
    """
    metadata = read_metadata(labels_path)

    # Get node names. This is a superset of all nodes across all skeletons. Note that
    # node ordering is specific to each skeleton, so we'll need to fix this afterwards.
    node_names = [x["name"] for x in metadata["nodes"]]

    # Use the SLP skeleton decoder
    decoder = SkeletonSLPDecoder()
    return decoder.decode(metadata, node_names)

`sleap_io.io.slp.serialize_skeletons(skeletons)` ¶

Serialize a list of Skeleton objects to JSON-compatible dicts.

Parameters:

Name	Type	Description	Default
`skeletons`	`list[Skeleton]`	A list of `Skeleton` objects.	required

Returns:

Type	Description
`tuple[list[dict], list[dict]]`	A tuple of `skeletons_dicts, nodes_dicts`. `nodes_dicts` is a list of dicts containing the nodes in all the skeletons. `skeletons_dicts` is a list of dicts containing the skeletons.

Notes

This function attempts to replicate the serialization of skeletons in legacy SLEAP which relies on a combination of networkx's graph serialization and our own metadata used to store nodes and edges independent of the graph structure.

However, because sleap-io does not currently load in the legacy metadata, this function will not produce byte-level compatible serialization with legacy formats, even though the ordering and all attributes of nodes and edges should match up.

Source code in sleap_io/io/slp.py

def serialize_skeletons(skeletons: list[Skeleton]) -> tuple[list[dict], list[dict]]:
    """Serialize a list of `Skeleton` objects to JSON-compatible dicts.

    Args:
        skeletons: A list of `Skeleton` objects.

    Returns:
        A tuple of `skeletons_dicts, nodes_dicts`.

        `nodes_dicts` is a list of dicts containing the nodes in all the skeletons.

        `skeletons_dicts` is a list of dicts containing the skeletons.

    Notes:
        This function attempts to replicate the serialization of skeletons in legacy
        SLEAP which relies on a combination of networkx's graph serialization and our
        own metadata used to store nodes and edges independent of the graph structure.

        However, because sleap-io does not currently load in the legacy metadata, this
        function will not produce byte-level compatible serialization with legacy
        formats, even though the ordering and all attributes of nodes and edges should
        match up.
    """
    # Use the SLP skeleton encoder
    encoder = SkeletonSLPEncoder()
    return encoder.encode_skeletons(skeletons)

Instance I/O¶

`sleap_io.io.slp.read_instances(labels_path, skeletons, tracks, points, pred_points, format_id)` ¶

Read Instance dataset in a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file.	required
`skeletons`	`list[Skeleton]`	A list of `Skeleton` objects (see `read_skeletons`).	required
`tracks`	`list[Track]`	A list of `Track` objects (see `read_tracks`).	required
`points`	`ndarray`	A structured array of point data (see `read_points`).	required
`pred_points`	`ndarray`	A structured array of predicted point data (see `read_pred_points`).	required
`format_id`	`float`	The format version identifier used to specify the format of the input file.	required

Returns:

Type	Description
`list[Instance \| PredictedInstance]`	A list of `Instance` and/or `PredictedInstance` objects.

Source code in sleap_io/io/slp.py

def read_instances(
    labels_path: str,
    skeletons: list[Skeleton],
    tracks: list[Track],
    points: np.ndarray,
    pred_points: np.ndarray,
    format_id: float,
) -> list[Instance | PredictedInstance]:
    """Read `Instance` dataset in a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file.
        skeletons: A list of `Skeleton` objects (see `read_skeletons`).
        tracks: A list of `Track` objects (see `read_tracks`).
        points: A structured array of point data (see `read_points`).
        pred_points: A structured array of predicted point data (see
            `read_pred_points`).
        format_id: The format version identifier used to specify the format of the input
            file.

    Returns:
        A list of `Instance` and/or `PredictedInstance` objects.
    """
    instances_data = read_hdf5_dataset(labels_path, "instances")

    instances = {}
    from_predicted_pairs = []
    for instance_data in instances_data:
        if format_id < 1.2:
            (
                instance_id,
                instance_type,
                frame_id,
                skeleton_id,
                track_id,
                from_predicted,
                instance_score,
                point_id_start,
                point_id_end,
            ) = instance_data
            tracking_score = 0.0
        elif format_id >= 1.2:
            (
                instance_id,
                instance_type,
                frame_id,
                skeleton_id,
                track_id,
                from_predicted,
                instance_score,
                point_id_start,
                point_id_end,
                tracking_score,
            ) = instance_data

        skeleton = skeletons[skeleton_id]
        track = tracks[track_id] if track_id >= 0 else None

        if instance_type == InstanceType.USER:
            pts_data = points[point_id_start:point_id_end]
            # Fast path: Build PointsArray directly from HDF5 data
            points_array = _points_from_hdf5_data(
                pts_data, skeleton, is_predicted=False
            )
            if format_id < 1.1:
                # Legacy coordinate system: top-left of pixel is (0, 0)
                # Adjust to new system: center of pixel is (0, 0)
                points_array["xy"] -= 0.5
            inst = Instance(
                points_array,
                skeleton=skeleton,
                track=track,
                tracking_score=tracking_score,
            )
            instances[instance_id] = inst

        elif instance_type == InstanceType.PREDICTED:
            pts_data = pred_points[point_id_start:point_id_end]
            # Fast path: Build PredictedPointsArray directly from HDF5 data
            points_array = _points_from_hdf5_data(pts_data, skeleton, is_predicted=True)
            if format_id < 1.1:
                # Legacy coordinate system: top-left of pixel is (0, 0)
                # Adjust to new system: center of pixel is (0, 0)
                points_array["xy"] -= 0.5
            inst = PredictedInstance(
                points_array,
                skeleton=skeleton,
                track=track,
                score=instance_score,
                tracking_score=tracking_score,
            )
            instances[instance_id] = inst

        if from_predicted >= 0:
            from_predicted_pairs.append((instance_id, from_predicted))

    # Link instances based on from_predicted field.
    for instance_id, from_predicted in from_predicted_pairs:
        instances[instance_id].from_predicted = instances[from_predicted]

    # Convert instances back to list.
    instances = list(instances.values())

    return instances

`sleap_io.io.slp.read_points(labels_path)` ¶

Read points dataset from a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file.	required

Returns:

Type	Description
`ndarray`	A structured array of point data.

Source code in sleap_io/io/slp.py

def read_points(labels_path: str) -> np.ndarray:
    """Read points dataset from a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file.

    Returns:
        A structured array of point data.
    """
    pts = read_hdf5_dataset(labels_path, "points")
    return pts

`sleap_io.io.slp.read_pred_points(labels_path)` ¶

Read predicted points dataset from a SLEAP labels file.

Parameters:

Name	Type	Description	Default
`labels_path`	`str`	A string path to the SLEAP labels file.	required

Returns:

Type	Description
`ndarray`	A structured array of predicted point data.

Source code in sleap_io/io/slp.py

def read_pred_points(labels_path: str) -> np.ndarray:
    """Read predicted points dataset from a SLEAP labels file.

    Args:
        labels_path: A string path to the SLEAP labels file.

    Returns:
        A structured array of predicted point data.
    """
    pred_pts = read_hdf5_dataset(labels_path, "pred_points")
    return pred_pts

Lazy Loading¶

`sleap_io.io.slp_lazy.LazyDataStore` ¶

Holds raw HDF5 data and provides lazy access methods.

Attributes:

Name	Type	Description
`frames_data`		Structured array from /frames HDF5 dataset. Fields: frame_id, video_id, frame_idx, instance_id_start, instance_id_end.
`instances_data`		Structured array from /instances HDF5 dataset. Fields vary by format_id but include: instance_id, instance_type, frame_id, skeleton_id, track_id, from_predicted, instance_score, point_id_start, point_id_end, and optionally tracking_score.
`pred_points_data`		Structured array from /pred_points HDF5 dataset. Fields: x, y, visible, complete, score.
`points_data`		Structured array from /points HDF5 dataset. Fields: x, y, visible, complete.
`videos`		List of eagerly loaded Video objects.
`skeletons`		List of eagerly loaded Skeleton objects.
`tracks`		List of eagerly loaded Track objects.
`format_id`		SLP format version.
`_source_path`		Path to source SLP file (for debugging).

Methods:

Name	Description
`materialize_frame`	Create a fully materialized LabeledFrame.
`materialize_all`	Materialize all frames.
`to_numpy`	Build numpy array directly from raw data (fast path).
`get_user_frame_indices`	Find indices of frames containing user (non-predicted) instances.

`materialize_frame(idx)` ¶

Create a fully materialized LabeledFrame.

Parameters:

Name	Type	Description	Default
`idx`	`int`	Index into frames_data array.	required

Returns:

Type	Description
`LabeledFrame`	A real LabeledFrame with real Instance objects.

`materialize_all()` ¶

Materialize all frames.

Returns:

Type	Description
`list[LabeledFrame]`	List of all LabeledFrame objects.

`to_numpy(video=None, untracked=False, return_confidence=False, user_instances=True)` ¶

Build numpy array directly from raw data (fast path).

This method builds the output array directly from raw HDF5 data without creating any Instance or LabeledFrame objects, providing significant performance improvement for workflows that only need numpy output.

Parameters:

Name	Type	Description	Default
`video`	`Video \| None`	Video to filter by. If None, uses the first video.	`None`
`untracked`	`bool`	If True, index by instance order instead of tracks. If False (default), organize instances by their track assignment.	`False`
`return_confidence`	`bool`	If True, include confidence as third coordinate. For user instances, confidence is set to 1.0.	`False`
`user_instances`	`bool`	If True (default), prefer user instances over predicted instances. If False, only include predicted instances.	`True`

Returns:

Type	Description
`ndarray`	Array of shape (n_frames, n_tracks, n_nodes, 2) or (n_frames, n_tracks, n_nodes, 3) if return_confidence is True. Missing data is filled with np.nan.

`get_user_frame_indices()` ¶

Find indices of frames containing user (non-predicted) instances.

Returns:

Type	Description
`list[int]`	List of frame indices (into frames_data) that have at least one user instance.

`sleap_io.io.slp_lazy.LazyFrameList` ¶

List-like proxy that materializes LabeledFrame objects on access.

This provides backward compatibility for code that accesses labels.labeled_frames directly. Frames are created on-demand when accessed via indexing or iteration.

Mutations are blocked with helpful error messages suggesting to call labels.materialize() first.

Methods:

Name	Description
`__delitem__`	Block item deletion with helpful error.
`__getitem__`	Get frame(s) by index or slice.
`__init__`	Initialize with a LazyDataStore.
`__iter__`	Iterate over frames, materializing each.
`__len__`	Return number of frames.
`__repr__`	Return informative representation.
`__setitem__`	Block item assignment with helpful error.
`append`	Block append with helpful error.
`extend`	Block extend with helpful error.
`insert`	Block insert with helpful error.

Attributes:

Name	Type	Description
`__dict__`		Read-only proxy of a mapping.
`__doc__`		str(object='') -> str
`__firstlineno__`		int([x]) -> integer
`__module__`		str(object='') -> str
`__static_attributes__`		Built-in immutable sequence.
`__weakref__`		list of weak references to the object

dict = mappingproxy({'module': 'sleap_io.io.slp_lazy', 'firstlineno': 654, 'doc': 'List-like proxy that materializes LabeledFrame objects on access.\n\nThis provides backward compatibility for code that accesses\nlabels.labeled_frames directly. Frames are created on-demand when\naccessed via indexing or iteration.\n\nMutations are blocked with helpful error messages suggesting to call\nlabels.materialize() first.\n', 'init': <function LazyFrameList.init at 0x7f41a8f73e20>, 'len': <function LazyFrameList.len at 0x7f41a8f73ec0>, 'getitem': <function LazyFrameList.getitem at 0x7f41a8fdc400>, 'iter': <function LazyFrameList.iter at 0x7f41a8fdcc20>, 'repr': <function LazyFrameList.repr at 0x7f41a8fdccc0>, '_mutation_error': <function LazyFrameList._mutation_error at 0x7f41a8fdcd60>, 'append': <function LazyFrameList.append at 0x7f41a8fdce00>, 'extend': <function LazyFrameList.extend at 0x7f41a8fdcea0>, 'insert': <function LazyFrameList.insert at 0x7f41a8fdcf40>, 'setitem': <function LazyFrameList.setitem at 0x7f41a8fdcfe0>, 'delitem': <function LazyFrameList.delitem at 0x7f41a8fdd080>, '__static_attributes__': ('_store',), 'dict': <attribute 'dict' of 'LazyFrameList' objects>, 'weakref': <attribute 'weakref' of 'LazyFrameList' objects>}) `class-attribute` ¶

Read-only proxy of a mapping.

`doc = 'List-like proxy that materializes LabeledFrame objects on access.\n\nThis provides backward compatibility for code that accesses\nlabels.labeled_frames directly. Frames are created on-demand when\naccessed via indexing or iteration.\n\nMutations are blocked with helpful error messages suggesting to call\nlabels.materialize() first.\n'` `class-attribute` ¶

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.

`firstlineno = 654` `class-attribute` ¶

int([x]) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.

int('0b100', base=0) 4

`module = 'sleap_io.io.slp_lazy'` `class-attribute` ¶

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.

`__static_attributes__ = ('_store',)` `class-attribute` ¶

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

`weakref` `property` ¶

list of weak references to the object

`delitem(idx)` ¶

Block item deletion with helpful error.

Raises:

Type	Description
`RuntimeError`	Always, with guidance to materialize first.

`getitem(idx)` ¶

Get frame(s) by index or slice.

Parameters:

Name	Type	Description	Default
`idx`	`int \| slice`	Integer index or slice object.	required

Returns:

Type	Description
`LabeledFrame \| list[LabeledFrame]`	A single LabeledFrame for integer indexing, or a list of LabeledFrames for slicing.

Raises:

Type	Description
`IndexError`	If index is out of range.

`init(store)` ¶

Initialize with a LazyDataStore.

Parameters:

Name	Type	Description	Default
`store`	`LazyDataStore`	The LazyDataStore containing raw frame data.	required

`iter()` ¶

Iterate over frames, materializing each.

`len()` ¶

Return number of frames.

`repr()` ¶

Return informative representation.

`setitem(idx, value)` ¶

Block item assignment with helpful error.

Raises:

Type	Description
`RuntimeError`	Always, with guidance to materialize first.

`append(item)` ¶

Block append with helpful error.

Raises:

Type	Description
`RuntimeError`	Always, with guidance to materialize first.

`extend(items)` ¶

Block extend with helpful error.

Raises:

Type	Description
`RuntimeError`	Always, with guidance to materialize first.

`insert(idx, item)` ¶

Block insert with helpful error.

Raises:

Type	Description
`RuntimeError`	Always, with guidance to materialize first.

SLP File Format¶

HDF5 Layout¶

Core Datasets¶

Metadata Datasets¶

Videos¶

Backend Types¶

Source Video Lineage¶

Embedded Images¶

Encoding Formats¶

Frame Selection Modes¶

Channel Order¶

Skeletons¶

JSON Format (SLP Files)¶

Edge Types¶

Symmetries¶

YAML Format¶

Structure¶

Fields¶

Multiple Skeletons¶

API Functions¶

Metadata¶

Attributes¶

JSON Structure¶

Provenance¶

Tracks¶

JSON Structure¶

Example¶

Instance Linking¶

Suggestions¶

JSON Structure¶

Groups¶

Optional Dataset¶

Sessions¶

JSON Structure¶

Calibration¶

Camera-Video Mapping¶

Frame Groups¶

Optional Dataset¶

Instances¶

Instance Dtype¶

Instance Types¶

Instance Linking¶

Points¶

User Points (/points)¶

Predicted Points (/pred_points)¶

Coordinate System¶

Labeled Frames¶

Frame Dtype¶

Video ID Mapping¶

Lazy Loading¶

Architecture¶

Performance Benefits¶

When to Use Lazy Loading¶

Fast Paths¶

Usage¶

Version History¶

Format 1.0¶

Format 1.1¶

Format 1.2¶

Format 1.3¶

Format 1.4 (Current)¶

API¶

High-Level Functions¶

sleap_io.io.main.load_slp(filename, open_videos=True, lazy=False) ¶

sleap_io.io.main.save_slp(labels, filename, embed=False, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, progress_callback=None) ¶

Core Module¶

sleap_io.io.slp.read_labels(labels_path, open_videos=True) ¶

sleap_io.io.slp.write_labels(labels_path, labels, embed=None, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, embed_all_videos=True, progress_callback=None) ¶

Video I/O¶

sleap_io.io.slp.read_videos(labels_path, open_backend=True) ¶

sleap_io.io.slp.write_videos(labels_path, videos, restore_source=False, reference_mode=None, original_videos=None, verbose=True) ¶

sleap_io.io.slp.embed_videos(labels_path, labels, embed, verbose=True, plugin=None, embed_all_videos=True, progress_callback=None) ¶

Skeleton I/O¶

sleap_io.io.slp.read_skeletons(labels_path) ¶

sleap_io.io.slp.serialize_skeletons(skeletons) ¶

Instance I/O¶

sleap_io.io.slp.read_instances(labels_path, skeletons, tracks, points, pred_points, format_id) ¶

sleap_io.io.slp.read_points(labels_path) ¶

sleap_io.io.slp.read_pred_points(labels_path) ¶

Lazy Loading¶

User Points (`/points`)¶

Predicted Points (`/pred_points`)¶

`sleap_io.io.main.load_slp(filename, open_videos=True, lazy=False)` ¶

`sleap_io.io.main.save_slp(labels, filename, embed=False, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, progress_callback=None)` ¶

`sleap_io.io.slp.read_labels(labels_path, open_videos=True)` ¶

`sleap_io.io.slp.write_labels(labels_path, labels, embed=None, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, embed_all_videos=True, progress_callback=None)` ¶

`sleap_io.io.slp.read_videos(labels_path, open_backend=True)` ¶

`sleap_io.io.slp.write_videos(labels_path, videos, restore_source=False, reference_mode=None, original_videos=None, verbose=True)` ¶

`sleap_io.io.slp.embed_videos(labels_path, labels, embed, verbose=True, plugin=None, embed_all_videos=True, progress_callback=None)` ¶

`sleap_io.io.slp.read_skeletons(labels_path)` ¶

`sleap_io.io.slp.serialize_skeletons(skeletons)` ¶

`sleap_io.io.slp.read_instances(labels_path, skeletons, tracks, points, pred_points, format_id)` ¶

`sleap_io.io.slp.read_points(labels_path)` ¶

`sleap_io.io.slp.read_pred_points(labels_path)` ¶

`sleap_io.io.slp_lazy.LazyDataStore` ¶

`materialize_frame(idx)` ¶

`materialize_all()` ¶

`to_numpy(video=None, untracked=False, return_confidence=False, user_instances=True)` ¶

`get_user_frame_indices()` ¶

`sleap_io.io.slp_lazy.LazyFrameList` ¶

`firstlineno = 654` `class-attribute` ¶

`module = 'sleap_io.io.slp_lazy'` `class-attribute` ¶

`__static_attributes__ = ('_store',)` `class-attribute` ¶

`weakref` `property` ¶

`delitem(idx)` ¶

`getitem(idx)` ¶

`init(store)` ¶

`iter()` ¶

`len()` ¶

`repr()` ¶

`setitem(idx, value)` ¶

`append(item)` ¶

`extend(items)` ¶

`insert(idx, item)` ¶