Data formats¶
sleap-io provides a unified interface for reading and writing pose tracking data across multiple formats. The library automatically detects file formats and provides harmonized I/O operations.
Universal I/O Functions¶
sleap_io.io.main.load_file(filename, format=None, **kwargs)
¶
Load a file and return the appropriate object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str | Path
|
Path to a file. |
required |
format
|
str | None
|
Optional format to load as. If not provided, will be inferred from the file extension. Available formats are: "slp", "nwb", "alphatracker", "labelstudio", "coco", "jabs", "analysis_h5", "dlc", "ultralytics", "leap", and "video". |
None
|
**kwargs
|
Additional arguments passed to the format-specific loading function:
- For "slp" format: No additional arguments.
- For "nwb" format: No additional arguments.
- For "alphatracker" format: No additional arguments.
- For "leap" format: skeleton (Optional[Skeleton]): Skeleton to use if not
defined in the file.
- For "labelstudio" format: skeleton (Optional[Skeleton]): Skeleton to
use for
the labels.
- For "coco" format: dataset_root (Optional[str]): Root directory of the
dataset. grayscale (bool): If True, load images as grayscale (1 channel).
If False, load as RGB (3 channels). Default is False.
- For "jabs" format: skeleton (Optional[Skeleton]): Skeleton to use for
the labels.
- For "analysis_h5" format: video (Optional[Video | str]): Video to
associate with data. If None, uses video_path stored in the file.
- For "dlc" format: video_search_paths (Optional[List[str]]): Paths to
search for video files.
- For "ultralytics" format: See |
required |
Returns:
| Type | Description |
|---|---|
Labels | Video
|
A |
Source code in sleap_io/io/main.py
def load_file(
filename: str | Path, format: str | None = None, **kwargs
) -> Labels | Video:
"""Load a file and return the appropriate object.
Args:
filename: Path to a file.
format: Optional format to load as. If not provided, will be inferred from the
file extension. Available formats are: "slp", "nwb", "alphatracker",
"labelstudio", "coco", "jabs", "analysis_h5", "dlc", "ultralytics", "leap",
and "video".
**kwargs: Additional arguments passed to the format-specific loading function:
- For "slp" format: No additional arguments.
- For "nwb" format: No additional arguments.
- For "alphatracker" format: No additional arguments.
- For "leap" format: skeleton (Optional[Skeleton]): Skeleton to use if not
defined in the file.
- For "labelstudio" format: skeleton (Optional[Skeleton]): Skeleton to
use for
the labels.
- For "coco" format: dataset_root (Optional[str]): Root directory of the
dataset. grayscale (bool): If True, load images as grayscale (1 channel).
If False, load as RGB (3 channels). Default is False.
- For "jabs" format: skeleton (Optional[Skeleton]): Skeleton to use for
the labels.
- For "analysis_h5" format: video (Optional[Video | str]): Video to
associate with data. If None, uses video_path stored in the file.
- For "dlc" format: video_search_paths (Optional[List[str]]): Paths to
search for video files.
- For "ultralytics" format: See `load_ultralytics` for supported arguments.
- For "video" format: See `load_video` for supported arguments.
Returns:
A `Labels` or `Video` object.
"""
if isinstance(filename, Path):
filename = filename.as_posix()
if format is None:
if filename.lower().endswith(".slp"):
format = "slp"
elif filename.lower().endswith(".nwb"):
format = "nwb"
elif filename.lower().endswith(".mat"):
format = "leap"
elif filename.lower().endswith(".json"):
# Detect JSON format: AlphaTracker, COCO, or Label Studio
if _detect_alphatracker_format(filename):
format = "alphatracker"
elif _detect_coco_format(filename):
format = "coco"
else:
format = "json"
elif filename.lower().endswith(".h5"):
# Check if this is Analysis HDF5 or JABS
from sleap_io.io import analysis_h5
if analysis_h5.is_analysis_h5_file(filename):
format = "analysis_h5"
else:
format = "jabs"
elif filename.endswith("data.yaml") or (
Path(filename).is_dir() and (Path(filename) / "data.yaml").exists()
):
format = "ultralytics"
elif filename.lower().endswith(".csv"):
from sleap_io.io import dlc
if dlc.is_dlc_file(filename):
format = "dlc"
else:
format = "csv"
else:
for vid_ext in Video.EXTS:
if filename.lower().endswith(vid_ext.lower()):
format = "video"
break
if format is None:
raise ValueError(f"Could not infer format from filename: '{filename}'.")
if filename.lower().endswith(".slp"):
return load_slp(filename, **kwargs)
elif filename.lower().endswith(".nwb"):
return load_nwb(filename, **kwargs)
elif filename.lower().endswith(".mat"):
return load_leap(filename, **kwargs)
elif filename.lower().endswith(".json"):
if format == "alphatracker":
return load_alphatracker(filename, **kwargs)
elif format == "coco":
return load_coco(filename, **kwargs)
else:
return load_labelstudio(filename, **kwargs)
elif filename.lower().endswith(".h5"):
if format == "analysis_h5":
return load_analysis_h5(filename, **kwargs)
else:
return load_jabs(filename, **kwargs)
elif format == "dlc":
return load_dlc(filename, **kwargs)
elif format == "csv":
return load_csv(filename, **kwargs)
elif format == "ultralytics":
return load_ultralytics(filename, **kwargs)
elif format == "video":
return load_video(filename, **kwargs)
sleap_io.io.main.save_file(labels, filename, format=None, verbose=True, progress_callback=None, **kwargs)
¶
Save a file based on the extension.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
A SLEAP |
required |
filename
|
str | Path
|
Path to save labels to. |
required |
format
|
str | None
|
Optional format to save as. If not provided, will be inferred from the file extension. Available formats are: "slp", "nwb", "labelstudio", "coco", "jabs", "analysis_h5", and "ultralytics". |
None
|
verbose
|
bool
|
If |
True
|
progress_callback
|
Callable[[int, int], bool] | None
|
Optional callback function called during frame embedding
(SLP format only) with |
None
|
**kwargs
|
Additional arguments passed to the format-specific saving function:
- For "slp" format: embed (bool | str | list[tuple[Video, int]] |
None): Frames
to embed in the saved labels file. One of None, True, "all", "user",
"suggestions", "user+suggestions", "source" or list of tuples of
(video, frame_idx). If False (the default), no frames are embedded.
embed_inplace (bool): If False (default), copy labels before embedding
to avoid mutating the input. If True, modify labels in-place.
- For "nwb" format: pose_estimation_metadata (dict): Metadata to store
in the
NWB file. append (bool): If True, append to existing NWB file.
- For "labelstudio" format: No additional arguments.
- For "coco" format: image_filenames (Optional[Union[str, List[str]]]):
Image filenames to use. visibility_encoding (str): Either "binary" or
"ternary" (default).
- For "jabs" format: pose_version (int): JABS pose format version (1-6).
root_folder (Optional[str]): Root folder for JABS project structure.
- For "analysis_h5" format: See |
required |
Source code in sleap_io/io/main.py
def save_file(
labels: Labels,
filename: str | Path,
format: str | None = None,
verbose: bool = True,
progress_callback: Callable[[int, int], bool] | None = None,
**kwargs,
):
"""Save a file based on the extension.
Args:
labels: A SLEAP `Labels` object (see `load_slp`).
filename: Path to save labels to.
format: Optional format to save as. If not provided, will be inferred from the
file extension. Available formats are: "slp", "nwb", "labelstudio", "coco",
"jabs", "analysis_h5", and "ultralytics".
verbose: If `True` (the default), display a progress bar when embedding frames
(only applies to the SLP format).
progress_callback: Optional callback function called during frame embedding
(SLP format only) with `(current, total)` arguments. If it returns `False`,
the operation is cancelled and `ExportCancelled` is raised.
**kwargs: Additional arguments passed to the format-specific saving function:
- For "slp" format: embed (bool | str | list[tuple[Video, int]] |
None): Frames
to embed in the saved labels file. One of None, True, "all", "user",
"suggestions", "user+suggestions", "source" or list of tuples of
(video, frame_idx). If False (the default), no frames are embedded.
embed_inplace (bool): If False (default), copy labels before embedding
to avoid mutating the input. If True, modify labels in-place.
- For "nwb" format: pose_estimation_metadata (dict): Metadata to store
in the
NWB file. append (bool): If True, append to existing NWB file.
- For "labelstudio" format: No additional arguments.
- For "coco" format: image_filenames (Optional[Union[str, List[str]]]):
Image filenames to use. visibility_encoding (str): Either "binary" or
"ternary" (default).
- For "jabs" format: pose_version (int): JABS pose format version (1-6).
root_folder (Optional[str]): Root folder for JABS project structure.
- For "analysis_h5" format: See `save_analysis_h5` for supported arguments.
- For "ultralytics" format: See `save_ultralytics` for supported arguments.
"""
if isinstance(filename, Path):
filename = str(filename)
if format is None:
if filename.lower().endswith(".slp"):
format = "slp"
elif filename.lower().endswith(".nwb"):
format = "nwb"
elif filename.lower().endswith(".json"):
# Check if this should be COCO format based on kwargs
if "visibility_encoding" in kwargs or "image_filenames" in kwargs:
format = "coco"
else:
format = "labelstudio"
elif filename.lower().endswith(".h5") or filename.lower().endswith(
".analysis.h5"
):
# Analysis HDF5 can be detected by extension pattern or kwargs
if "min_occupancy" in kwargs or filename.lower().endswith(".analysis.h5"):
format = "analysis_h5"
elif "pose_version" in kwargs:
format = "jabs"
else:
# Default to analysis_h5 for .h5 extension without specific jabs kwargs
format = "analysis_h5"
elif "pose_version" in kwargs:
format = "jabs"
elif "split_ratios" in kwargs or Path(filename).is_dir():
format = "ultralytics"
if format == "slp":
save_slp(
labels,
filename,
verbose=verbose,
progress_callback=progress_callback,
**kwargs,
)
elif format == "nwb":
save_nwb(labels, filename, **kwargs)
elif format == "labelstudio":
save_labelstudio(labels, filename, **kwargs)
elif format == "coco":
save_coco(labels, filename, **kwargs)
elif format == "jabs":
pose_version = kwargs.pop("pose_version", 5)
root_folder = kwargs.pop("root_folder", filename)
save_jabs(labels, pose_version=pose_version, root_folder=root_folder)
elif format == "analysis_h5":
# Filter kwargs to those accepted by save_analysis_h5
analysis_kwargs = {
k: v
for k, v in kwargs.items()
if k
in (
"video",
"labels_path",
"all_frames",
"min_occupancy",
"preset",
"frame_dim",
"track_dim",
"node_dim",
"xy_dim",
"save_metadata",
)
}
save_analysis_h5(labels, filename, **analysis_kwargs)
elif format == "ultralytics":
save_ultralytics(labels, filename, **kwargs)
elif format == "csv" or filename.lower().endswith(".csv"):
csv_format = kwargs.pop("csv_format", "sleap")
# Filter kwargs to only those accepted by save_csv
csv_kwargs = {
k: v
for k, v in kwargs.items()
if k in ("video", "include_score", "scorer", "save_metadata")
}
save_csv(labels, filename, format=csv_format, **csv_kwargs)
else:
raise ValueError(f"Unknown format '{format}' for filename: '{filename}'.")
Video I/O¶
sleap_io.io.main.load_video(filename, **kwargs)
¶
Load a video file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
The filename(s) of the video. Supported extensions: "mp4", "avi", "mov", "mj2", "mkv", "h5", "hdf5", "slp", "png", "jpg", "jpeg", "tif", "tiff", "bmp". If the filename is a list, a list of image filenames are expected. If filename is a folder, it will be searched for images. |
required |
**kwargs
|
Additional arguments passed to If not specified, uses the following priority:
1. Global default set via To set a global default:
|
required |
Returns:
| Type | Description |
|---|---|
Video
|
A |
See Also
set_default_video_plugin: Set the default video plugin globally. get_default_video_plugin: Get the current default video plugin.
Source code in sleap_io/io/main.py
def load_video(filename: str, **kwargs) -> Video:
"""Load a video file.
Args:
filename: The filename(s) of the video. Supported extensions: "mp4", "avi",
"mov", "mj2", "mkv", "h5", "hdf5", "slp", "png", "jpg", "jpeg", "tif",
"tiff", "bmp". If the filename is a list, a list of image filenames are
expected. If filename is a folder, it will be searched for images.
**kwargs: Additional arguments passed to `Video.from_filename`.
Currently supports:
- dataset: Name of dataset in HDF5 file.
- grayscale: Whether to force grayscale. If None, autodetect on first
frame load.
- keep_open: Whether to keep the video reader open between calls to read
frames.
If False, will close the reader after each call. If True (the
default), it will
keep the reader open and cache it for subsequent calls which may
enhance the
performance of reading multiple frames.
- source_video: Source video object if this is a proxy video. This is
metadata
and does not affect reading.
- backend_metadata: Metadata to store on the video backend. This is
useful for
storing metadata that requires an open backend (e.g., shape
information) without
having to open the backend.
- plugin: Video plugin to use for MediaVideo backend. One of "opencv",
"FFMPEG",
or "pyav". Also accepts aliases (case-insensitive):
* opencv: "opencv", "cv", "cv2", "ocv"
* FFMPEG: "FFMPEG", "ffmpeg", "imageio-ffmpeg", "imageio_ffmpeg"
* pyav: "pyav", "av"
If not specified, uses the following priority:
1. Global default set via `sio.set_default_video_plugin()`
2. Auto-detection based on available packages
To set a global default:
>>> import sleap_io as sio
>>> sio.set_default_video_plugin("opencv")
>>> video = sio.load_video("video.mp4") # Uses opencv
- input_format: Format of the data in HDF5 datasets. One of
"channels_last" (the
default) in (frames, height, width, channels) order or "channels_first" in
(frames, channels, width, height) order.
- frame_map: Mapping from frame indices to indices in the HDF5 dataset.
This is
used to translate between frame indices of images within their source
video
and indices of images in the dataset.
- source_filename: Path to the source video file for HDF5 embedded videos.
- source_inds: Indices of frames in the source video file for HDF5
embedded videos.
- image_format: Format of images in HDF5 embedded dataset.
Returns:
A `Video` object.
See Also:
set_default_video_plugin: Set the default video plugin globally.
get_default_video_plugin: Get the current default video plugin.
"""
return Video.from_filename(filename, **kwargs)
sleap_io.io.main.save_video(frames, filename, fps=30, pixelformat='yuv420p', codec='libx264', crf=25, preset='superfast', output_params=None)
¶
Write a list of frames to a video file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
frames
|
ndarray | Video
|
Sequence of frames to write to video. Each frame should be a 2D or 3D numpy array with dimensions (height, width) or (height, width, channels). |
required |
filename
|
str | Path
|
Path to output video file. |
required |
fps
|
float
|
Frames per second. Defaults to 30. |
30
|
pixelformat
|
str
|
Pixel format for video. Defaults to "yuv420p". |
'yuv420p'
|
codec
|
str
|
Codec to use for encoding. Defaults to "libx264". |
'libx264'
|
crf
|
int
|
Constant rate factor to control lossiness of video. Values go from 2 to 32, with numbers in the 18 to 30 range being most common. Lower values mean less compressed/higher quality. Defaults to 25. No effect if codec is not "libx264". |
25
|
preset
|
str
|
H264 encoding preset. Defaults to "superfast". No effect if codec is not "libx264". |
'superfast'
|
output_params
|
list | None
|
Additional output parameters for FFMPEG. This should be a list of
strings corresponding to command line arguments for FFMPEG and libx264. Use
|
None
|
See also: sio.VideoWriter
Source code in sleap_io/io/main.py
def save_video(
frames: np.ndarray | Video,
filename: str | Path,
fps: float = 30,
pixelformat: str = "yuv420p",
codec: str = "libx264",
crf: int = 25,
preset: str = "superfast",
output_params: list | None = None,
):
"""Write a list of frames to a video file.
Args:
frames: Sequence of frames to write to video. Each frame should be a 2D or 3D
numpy array with dimensions (height, width) or (height, width, channels).
filename: Path to output video file.
fps: Frames per second. Defaults to 30.
pixelformat: Pixel format for video. Defaults to "yuv420p".
codec: Codec to use for encoding. Defaults to "libx264".
crf: Constant rate factor to control lossiness of video. Values go from 2 to 32,
with numbers in the 18 to 30 range being most common. Lower values mean less
compressed/higher quality. Defaults to 25. No effect if codec is not
"libx264".
preset: H264 encoding preset. Defaults to "superfast". No effect if codec is not
"libx264".
output_params: Additional output parameters for FFMPEG. This should be a list of
strings corresponding to command line arguments for FFMPEG and libx264. Use
`ffmpeg -h encoder=libx264` to see all options for libx264 output_params.
See also: `sio.VideoWriter`
"""
from sleap_io.io import video_writing
if output_params is None:
output_params = []
with video_writing.VideoWriter(
filename,
fps=fps,
pixelformat=pixelformat,
codec=codec,
crf=crf,
preset=preset,
output_params=output_params,
) as writer:
for frame in frames:
writer(frame)
Format-Specific Functions¶
SLEAP Native Format (.slp)¶
The native SLEAP format stores complete pose tracking projects including videos, skeletons, and annotations.
Detailed Format Specification
For comprehensive documentation of the SLP file format including HDF5 layout, data structures, and version history, see the SLP File Format Reference.
sleap_io.io.main.load_slp(filename, open_videos=True, lazy=False)
¶
Load a SLEAP dataset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to a SLEAP labels file ( |
required |
open_videos
|
bool
|
If |
True
|
lazy
|
bool
|
If |
False
|
Returns:
| Type | Description |
|---|---|
Labels
|
The dataset as a |
See Also
Labels.is_lazy: Check if Labels is lazy-loaded. Labels.materialize: Convert lazy Labels to eager.
Source code in sleap_io/io/main.py
def load_slp(filename: str, open_videos: bool = True, lazy: bool = False) -> Labels:
"""Load a SLEAP dataset.
Args:
filename: Path to a SLEAP labels file (`.slp`).
open_videos: If `True` (the default), attempt to open the video backend for
I/O. If `False`, the backend will not be opened (useful for reading metadata
when the video files are not available).
lazy: If `True`, defer instance materialization for faster loading.
Lazy-loaded Labels support read operations and fast numpy/save.
To modify, call `labels.materialize()` first. Default is `False`.
Returns:
The dataset as a `Labels` object.
See Also:
Labels.is_lazy: Check if Labels is lazy-loaded.
Labels.materialize: Convert lazy Labels to eager.
"""
from sleap_io.io import slp
if lazy:
return slp._read_labels_lazy(filename, open_videos=open_videos)
return slp.read_labels(filename, open_videos=open_videos)
sleap_io.io.main.save_slp(labels, filename, embed=False, restore_original_videos=True, embed_inplace=False, verbose=True, plugin=None, progress_callback=None)
¶
Save a SLEAP dataset to a .slp file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
A SLEAP |
required |
filename
|
str
|
Path to save labels to ending with |
required |
embed
|
bool | str | list[tuple[Video, int]] | None
|
Frames to embed in the saved labels file. One of If If If This argument is only valid for the SLP backend. |
False
|
restore_original_videos
|
bool
|
If |
True
|
embed_inplace
|
bool
|
If |
False
|
verbose
|
bool
|
If |
True
|
plugin
|
str | None
|
Image plugin to use for encoding embedded frames. One of "opencv"
or "imageio". If None, uses the global default from
|
None
|
progress_callback
|
Callable[[int, int], bool] | None
|
Optional callback function called during frame embedding
with |
None
|
Source code in sleap_io/io/main.py
def save_slp(
labels: Labels,
filename: str,
embed: bool | str | list[tuple[Video, int]] | None = False,
restore_original_videos: bool = True,
embed_inplace: bool = False,
verbose: bool = True,
plugin: str | None = None,
progress_callback: Callable[[int, int], bool] | None = None,
):
"""Save a SLEAP dataset to a `.slp` file.
Args:
labels: A SLEAP `Labels` object (see `load_slp`).
filename: Path to save labels to ending with `.slp`.
embed: Frames to embed in the saved labels file. One of `None`, `True`,
`"all"`, `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or list
of tuples of `(video, frame_idx)`.
If `False` is specified (the default), the source video will be restored
if available, otherwise the embedded frames will be re-saved.
If `True` or `"all"`, all labeled frames and suggested frames will be
embedded.
If `"source"` is specified, no images will be embedded and the source video
will be restored if available.
This argument is only valid for the SLP backend.
restore_original_videos: If `True` (default) and `embed=False`, use original
video files. If `False` and `embed=False`, keep references to source
`.pkg.slp` files. Only applies when `embed=False`.
embed_inplace: If `False` (default), a copy of the labels is made before
embedding to avoid modifying the in-memory labels. If `True`, the
labels will be modified in-place to point to the embedded videos,
which is faster but mutates the input. Only applies when embedding.
verbose: If `True` (the default), display a progress bar when embedding frames.
plugin: Image plugin to use for encoding embedded frames. One of "opencv"
or "imageio". If None, uses the global default from
`get_default_image_plugin()`. If no global default is set, auto-detects
based on available packages (opencv preferred, then imageio).
progress_callback: Optional callback function called during frame embedding
with `(current, total)` arguments. If it returns `False`, the operation
is cancelled and `ExportCancelled` is raised. When provided, tqdm
progress bar is disabled in favor of the callback.
"""
from sleap_io.io import slp
return slp.write_labels(
filename,
labels,
embed=embed,
restore_original_videos=restore_original_videos,
embed_inplace=embed_inplace,
verbose=verbose,
plugin=plugin,
progress_callback=progress_callback,
)
Lazy Loading for Large Files¶
When working with large SLP files (hundreds of thousands of frames), loading can be slow due to the creation of many Python objects. sleap-io provides a lazy loading mode that defers object creation until needed, significantly speeding up common workflows.
When to Use Lazy Loading¶
Lazy loading is recommended when:
- You only need to convert data to NumPy arrays (
labels.numpy()) - You're saving to a different file without modifications
- You're accessing a small subset of frames
- You want fast load times for large files
Basic Usage¶
import sleap_io as sio
# Load lazily (up to 90x faster than eager loading!)
labels = sio.load_slp("predictions.slp", lazy=True)
# Check if labels is lazy
print(labels.is_lazy) # True
# Fast path: convert directly to NumPy (no object creation)
poses = labels.numpy()
# Fast path: save without materialization
sio.save_slp(labels, "copy.slp")
Accessing Frames¶
Lazy-loaded Labels support standard read operations:
# These work normally (frames materialized on-demand)
print(len(labels)) # Number of frames
first_frame = labels[0] # Access single frame
last_frame = labels[-1] # Negative indexing
subset = labels[10:20] # Slicing
# Iteration (materializes each frame)
for lf in labels:
print(f"Frame {lf.frame_idx}: {len(lf)} instances")
Modifying Lazy Labels¶
Lazy Labels are read-only. To make modifications, first materialize:
# This raises RuntimeError
labels.append(new_frame) # Error: Cannot append on lazy-loaded Labels
# Materialize first to enable modifications
labels = labels.materialize() # Creates eager copy
labels.append(new_frame) # Now works
Performance Comparison¶
| Operation | Eager | Lazy | Speedup |
|---|---|---|---|
| Load only | 0.47s | 0.005s | ~90x |
| Load + numpy() | 0.86s | 0.38s | ~2x |
| Full iteration | 0.0002s | 0.41s | Eager faster |
Benchmarks on 18,000 frames with ~40,000 instances.
Lazy loading excels at avoiding unnecessary work. If you need to iterate over all frames, eager loading is faster.
API Reference¶
Labels properties and methods for lazy loading:
Labels.is_lazy-Trueif lazy-loadedLabels.materialize()- Convert to eagerLabels(returns self if already eager)Labels.numpy()- Uses fast path when lazy (no object creation)Labels.to_dataframe()- Uses fast path when lazy (no object creation)
Fast statistics (O(1) for lazy-loaded Labels):
Labels.n_user_instances- Total number of user-labeled instancesLabels.n_pred_instances- Total number of predicted instancesLabels.n_frames_per_video()- Dictionary mapping videos to frame countsLabels.n_instances_per_track()- Dictionary mapping tracks to instance counts
NWB Format (.nwb)¶
Neurodata Without Borders (NWB) is a standardized format for neurophysiology data. sleap-io provides comprehensive support for both reading and writing pose tracking data in NWB format.
Harmonized NWB I/O¶
The harmonized API automatically detects and routes to the appropriate NWB backend:
sleap_io.io.main.load_nwb(filename)
¶
Load an NWB dataset as a SLEAP Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to a NWB file ( |
required |
Returns:
| Type | Description |
|---|---|
Labels
|
The dataset as a |
sleap_io.io.main.save_nwb(labels, filename, nwb_format='auto', append=False)
¶
Save a SLEAP dataset to NWB format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
A SLEAP |
required |
filename
|
str | Path
|
Path to NWB file to save to. Must end in |
required |
nwb_format
|
str
|
Format to use for saving. Options are: - "auto" (default): Automatically detect based on data - "annotations": Save training annotations (PoseTraining) - "annotations_export": Export annotations with video frames - "predictions": Save predictions (PoseEstimation) |
'auto'
|
append
|
bool
|
If True, append to existing NWB file. Only supported for predictions format. Defaults to False. |
False
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If an invalid format is specified. |
Source code in sleap_io/io/main.py
def save_nwb(
labels: Labels,
filename: str | Path,
nwb_format: str = "auto",
append: bool = False,
) -> None:
"""Save a SLEAP dataset to NWB format.
Args:
labels: A SLEAP `Labels` object (see `load_slp`).
filename: Path to NWB file to save to. Must end in `.nwb`.
nwb_format: Format to use for saving. Options are:
- "auto" (default): Automatically detect based on data
- "annotations": Save training annotations (PoseTraining)
- "annotations_export": Export annotations with video frames
- "predictions": Save predictions (PoseEstimation)
append: If True, append to existing NWB file. Only supported for
predictions format. Defaults to False.
Raises:
ValueError: If an invalid format is specified.
"""
from sleap_io.io import nwb
from sleap_io.io.nwb import NwbFormat
# Convert string to NwbFormat if needed
if isinstance(nwb_format, str):
nwb_format = NwbFormat(nwb_format)
nwb.save_nwb(labels, filename, nwb_format, append=append)
NWB Format Types¶
sleap-io supports multiple NWB format types through the nwb_format parameter:
"auto"(default): Automatically detect based on data content- Uses
"annotations"if data contains user-labeled instances -
Uses
"predictions"if data contains only predicted instances -
"annotations": Save as PoseTraining format (ndx-pose extension) - Stores manual annotations for training data
- Preserves skeleton structure and node names
-
Includes annotator information
-
"annotations_export": Export annotations with embedded video frames - Creates self-contained NWB file with video data
- Generates MJPEG video with frame provenance tracking
-
Useful for sharing complete datasets
-
"predictions": Save as PoseEstimation format (ndx-pose extension) - Stores predicted pose data from inference
- Includes confidence scores
- Supports multiple animals/tracks
Examples¶
Basic NWB Usage¶
import sleap_io as sio
# Load any NWB file (auto-detects format)
labels = sio.load_nwb("pose_data.nwb")
# Save with auto-detection
sio.save_nwb(labels, "output.nwb")
# Save with specific format
sio.save_nwb(labels, "training.nwb", nwb_format="annotations")
sio.save_nwb(labels, "predictions.nwb", nwb_format="predictions")
Advanced Annotations API¶
For more control over NWB training data, use the annotations module directly:
from sleap_io.io.nwb_annotations import save_labels, load_labels
# Save with custom metadata
save_labels(
labels,
"training.nwb",
session_description="Mouse reaching task",
identifier="mouse_01_session_03",
annotator="researcher_name",
nwb_kwargs={
"session_id": "session_003",
"experimenter": ["John Doe", "Jane Smith"],
"lab": "Motor Control Lab",
"institution": "University",
"experiment_description": "Skilled reaching behavior"
}
)
# Load annotations
labels = load_labels("training.nwb")
Export with Video Frames¶
from sleap_io.io.nwb_annotations import export_labels, export_labeled_frames
# Export complete dataset with videos
export_labels(
labels,
output_dir="export/",
nwb_filename="dataset_with_videos.nwb",
as_training=True, # Include manual annotations
include_videos=True # Embed video frames
)
# Export only labeled frames as video
export_labeled_frames(
labels,
output_path="labeled_frames.avi",
labels_output_path="labels.nwb",
fps=30.0
)
Multi-Subject Support¶
For multi-animal experiments, sleap-io supports the ndx-multisubjects NWB extension. This links each tracked animal to a proper NWB Subject entry.
from sleap_io.io.nwb_annotations import save_labels
# Basic multi-subject export (uses track names as subject IDs)
save_labels(labels, "output.nwb", use_multisubjects=True)
# With detailed subject metadata
subjects_metadata = [
{"sex": "M", "species": "Mus musculus", "age": "P30D"},
{"sex": "F", "species": "Mus musculus", "age": "P45D"},
]
save_labels(
labels,
"output.nwb",
use_multisubjects=True,
subjects_metadata=subjects_metadata
)
Subject metadata fields
The subjects_metadata list should have one entry per track. Each entry can include:
sex: Subject sex (defaults to "U" for unknown)species: Species name (defaults to "unknown")age: Age in ISO 8601 duration format (e.g., "P30D" for 30 days)- Any other fields supported by NWB Subject
Tracked instances required
Multi-subject export requires all instances to have track assignments. A warning is issued if untracked instances are present.
NWB Metadata¶
The NWB format requires certain metadata fields. sleap-io provides sensible defaults:
- Required fields (auto-generated if not provided):
session_description: Defaults to "Processed SLEAP pose data"identifier: Auto-generated UUID string-
session_start_time: Current timestamp -
Optional fields (via
nwb_kwargs): session_id: Unique session identifierexperimenter: List of experimenterslab: Laboratory nameinstitution: Institution nameexperiment_description: Detailed experiment description- Any other valid NWB file fields
JABS Format (.h5)¶
JABS (Janelia Automatic Behavior System) format for behavior classification.
sleap_io.io.main.load_jabs(filename, skeleton=None)
¶
Read JABS-style predictions from a file and return a Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to the jabs h5 pose file. |
required |
skeleton
|
Skeleton | None
|
An optional |
None
|
Returns:
| Type | Description |
|---|---|
Labels
|
Parsed labels as a |
Source code in sleap_io/io/main.py
def load_jabs(filename: str, skeleton: Skeleton | None = None) -> Labels:
"""Read JABS-style predictions from a file and return a `Labels` object.
Args:
filename: Path to the jabs h5 pose file.
skeleton: An optional `Skeleton` object.
Returns:
Parsed labels as a `Labels` instance.
"""
from sleap_io.io import jabs
return jabs.read_labels(filename, skeleton=skeleton)
sleap_io.io.main.save_jabs(labels, pose_version, root_folder=None)
¶
Save a SLEAP dataset to JABS pose file format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
SLEAP |
required |
pose_version
|
int
|
The JABS pose version to write data out. |
required |
root_folder
|
str | None
|
Optional root folder where the files should be saved. |
None
|
Note
Filenames for JABS poses are based on video filenames.
Source code in sleap_io/io/main.py
def save_jabs(labels: Labels, pose_version: int, root_folder: str | None = None):
"""Save a SLEAP dataset to JABS pose file format.
Args:
labels: SLEAP `Labels` object.
pose_version: The JABS pose version to write data out.
root_folder: Optional root folder where the files should be saved.
Note:
Filenames for JABS poses are based on video filenames.
"""
from sleap_io.io import jabs
jabs.write_labels(labels, pose_version, root_folder)
SLEAP Analysis HDF5 Format (.h5)¶
The SLEAP Analysis HDF5 format is a portable format for exporting pose tracking predictions as dense numpy arrays. This is the format produced by SLEAP's "Export Analysis HDF5" feature, designed for easy loading in MATLAB and Python analysis pipelines.
sleap_io.io.main.load_analysis_h5(filename, video=None)
¶
Load SLEAP Analysis HDF5 file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to Analysis HDF5 file. |
required |
video
|
Video | str | None
|
Video to associate with data. If None, uses video_path stored in the file. Can be a Video object or path string. |
None
|
Returns:
| Type | Description |
|---|---|
Labels
|
Labels object with loaded pose data. |
Notes
If the file contains extended metadata (skeleton symmetries, video backend metadata, etc.), it will be used to reconstruct the full Labels context.
See Also
save_analysis_h5: Save Labels to Analysis HDF5 file.
Source code in sleap_io/io/main.py
def load_analysis_h5(
filename: str,
video: "Video | str | None" = None,
) -> Labels:
"""Load SLEAP Analysis HDF5 file.
Args:
filename: Path to Analysis HDF5 file.
video: Video to associate with data. If None, uses video_path stored
in the file. Can be a Video object or path string.
Returns:
Labels object with loaded pose data.
Notes:
If the file contains extended metadata (skeleton symmetries, video
backend metadata, etc.), it will be used to reconstruct the full
Labels context.
See Also:
save_analysis_h5: Save Labels to Analysis HDF5 file.
"""
from sleap_io.io import analysis_h5
return analysis_h5.read_labels(filename, video=video)
sleap_io.io.main.save_analysis_h5(labels, filename, *, video=None, labels_path=None, all_frames=True, min_occupancy=0.0, preset=None, frame_dim=None, track_dim=None, node_dim=None, xy_dim=None, save_metadata=True)
¶
Save Labels to SLEAP Analysis HDF5 file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
Labels to export. |
required |
filename
|
str
|
Output file path. |
required |
video
|
Video | int | None
|
Video to export. If None, uses first video. Can be a Video object or an integer index. |
None
|
labels_path
|
str | None
|
Source labels path (stored as metadata). |
None
|
all_frames
|
bool
|
Include all frames from 0 to last labeled frame. Default True. |
True
|
min_occupancy
|
float
|
Minimum track occupancy ratio (0-1) to keep. 0 = keep all non-empty tracks (SLEAP default). 0.5 = keep tracks with >50% occupancy. |
0.0
|
preset
|
str | None
|
Axis ordering preset. Options: - "matlab" (default): SLEAP-compatible ordering for MATLAB. tracks shape: (n_tracks, 2, n_nodes, n_frames) - "standard": Intuitive Python ordering. tracks shape: (n_frames, n_tracks, n_nodes, 2) Mutually exclusive with explicit dimension parameters. |
None
|
frame_dim
|
int | None
|
Position of the frame dimension (0-3). |
None
|
track_dim
|
int | None
|
Position of the track dimension (0-3). |
None
|
node_dim
|
int | None
|
Position of the node dimension (0-3). |
None
|
xy_dim
|
int | None
|
Position of the xy dimension (0-3). |
None
|
save_metadata
|
bool
|
Store extended metadata for full round-trip. Default True. |
True
|
See Also
load_analysis_h5: Load Labels from Analysis HDF5 file.
Source code in sleap_io/io/main.py
def save_analysis_h5(
labels: Labels,
filename: str,
*,
video: "Video | int | None" = None,
labels_path: str | None = None,
all_frames: bool = True,
min_occupancy: float = 0.0,
preset: str | None = None,
frame_dim: int | None = None,
track_dim: int | None = None,
node_dim: int | None = None,
xy_dim: int | None = None,
save_metadata: bool = True,
) -> None:
"""Save Labels to SLEAP Analysis HDF5 file.
Args:
labels: Labels to export.
filename: Output file path.
video: Video to export. If None, uses first video. Can be a Video
object or an integer index.
labels_path: Source labels path (stored as metadata).
all_frames: Include all frames from 0 to last labeled frame.
Default True.
min_occupancy: Minimum track occupancy ratio (0-1) to keep.
0 = keep all non-empty tracks (SLEAP default).
0.5 = keep tracks with >50% occupancy.
preset: Axis ordering preset. Options:
- "matlab" (default): SLEAP-compatible ordering for MATLAB.
tracks shape: (n_tracks, 2, n_nodes, n_frames)
- "standard": Intuitive Python ordering.
tracks shape: (n_frames, n_tracks, n_nodes, 2)
Mutually exclusive with explicit dimension parameters.
frame_dim: Position of the frame dimension (0-3).
track_dim: Position of the track dimension (0-3).
node_dim: Position of the node dimension (0-3).
xy_dim: Position of the xy dimension (0-3).
save_metadata: Store extended metadata for full round-trip.
Default True.
See Also:
load_analysis_h5: Load Labels from Analysis HDF5 file.
"""
from sleap_io.io import analysis_h5
analysis_h5.write_labels(
labels,
filename,
video=video,
labels_path=labels_path,
all_frames=all_frames,
min_occupancy=min_occupancy,
preset=preset,
frame_dim=frame_dim,
track_dim=track_dim,
node_dim=node_dim,
xy_dim=xy_dim,
save_metadata=save_metadata,
)
Axis Ordering Presets¶
The format supports configurable axis ordering via presets:
| Preset | Description | tracks shape |
|---|---|---|
matlab (default) |
SLEAP-compatible, optimized for MATLAB | (tracks, 2, nodes, frames) |
standard |
Python-native, intuitive indexing | (frames, tracks, nodes, 2) |
import sleap_io as sio
labels = sio.load_slp("predictions.slp")
# Default (MATLAB-compatible) - matches SLEAP's export
sio.save_analysis_h5(labels, "output.h5")
# Python-native ordering for easier numpy indexing
sio.save_analysis_h5(labels, "output.h5", preset="standard")
# Filter tracks with <50% occupancy
sio.save_analysis_h5(labels, "output.h5", min_occupancy=0.5)
# Load back
loaded = sio.load_analysis_h5("output.h5")
Self-Documenting Format¶
Each dataset stores its dimension names in the dims HDF5 attribute, making files self-documenting:
import h5py
with h5py.File("output.h5", "r") as f:
print(f["tracks"].attrs["dims"]) # e.g., '["track", "xy", "node", "frame"]'
print(f.attrs["preset"]) # "matlab", "standard", or "custom"
Label Studio Format (.json)¶
Label Studio is a multi-modal annotation platform. Export annotations from Label Studio and load them into SLEAP.
sleap_io.io.main.load_labelstudio(filename, skeleton=None)
¶
Read Label Studio-style annotations from a file and return a Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to the label-studio annotation file in JSON format. |
required |
skeleton
|
Skeleton | list[str] | None
|
An optional |
None
|
Returns:
| Type | Description |
|---|---|
Labels
|
Parsed labels as a |
Source code in sleap_io/io/main.py
def load_labelstudio(
filename: str, skeleton: Skeleton | list[str] | None = None
) -> Labels:
"""Read Label Studio-style annotations from a file and return a `Labels` object.
Args:
filename: Path to the label-studio annotation file in JSON format.
skeleton: An optional `Skeleton` object or list of node names. If not provided
(the default), skeleton will be inferred from the data. It may be useful to
provide this so the keypoint label types can be filtered to just the ones in
the skeleton.
Returns:
Parsed labels as a `Labels` instance.
"""
from sleap_io.io import labelstudio
return labelstudio.read_labels(filename, skeleton=skeleton)
sleap_io.io.main.save_labelstudio(labels, filename)
¶
Save a SLEAP dataset to Label Studio format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
A SLEAP |
required |
filename
|
str
|
Path to save labels to ending with |
required |
Source code in sleap_io/io/main.py
DeepLabCut Format (.h5, .csv)¶
Load predictions from DeepLabCut, a popular markerless pose estimation tool.
sleap_io.io.main.load_dlc(filename, video_search_paths=None, **kwargs)
¶
Read DeepLabCut annotations from a CSV file and return a Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to DLC CSV file with annotations. |
required |
video_search_paths
|
list[str | Path] | None
|
Optional list of paths to search for video files. |
None
|
**kwargs
|
Additional arguments passed to DLC loader. |
required |
Returns:
| Type | Description |
|---|---|
Labels
|
Parsed labels as a |
Source code in sleap_io/io/main.py
def load_dlc(
filename: str, video_search_paths: list[str | Path] | None = None, **kwargs
) -> Labels:
"""Read DeepLabCut annotations from a CSV file and return a `Labels` object.
Args:
filename: Path to DLC CSV file with annotations.
video_search_paths: Optional list of paths to search for video files.
**kwargs: Additional arguments passed to DLC loader.
Returns:
Parsed labels as a `Labels` instance.
"""
from sleap_io.io import dlc
return dlc.load_dlc(filename, video_search_paths=video_search_paths, **kwargs)
CSV Format (.csv)¶
sleap-io provides comprehensive CSV support for reading and writing pose tracking data, enabling interoperability with spreadsheet tools, custom pipelines, and other pose estimation frameworks.
Supported CSV Formats¶
| Format | Description | Use Case |
|---|---|---|
sleap |
SLEAP Analysis CSV (default) | Native SLEAP exports, one row per instance |
dlc |
DeepLabCut format | DLC compatibility, multi-header structure |
points |
One row per point | Most normalized, database-friendly |
instances |
One row per instance | Compact, analysis-friendly |
frames |
One row per frame | Wide format, all instances in columns |
Basic Usage¶
import sleap_io as sio
# Load CSV (auto-detects format)
labels = sio.load_csv("predictions.csv")
# Save in SLEAP Analysis format (default)
sio.save_csv(labels, "output.csv")
# Save in DLC format
sio.save_csv(labels, "dlc_output.csv", format="dlc", scorer="MyModel")
# Save with metadata for full round-trip support
sio.save_csv(labels, "output.csv", save_metadata=True)
# Creates: output.csv + output.json (metadata)
Round-Trip with Metadata¶
CSV files cannot store all Labels information (skeleton edges, symmetries, suggestions). To enable full round-trip reconstruction, use save_metadata=True:
# Save with metadata sidecar file
sio.save_csv(labels, "data.csv", save_metadata=True)
# Creates: data.csv and data.json
# Load back with full metadata
labels = sio.load_csv("data.csv")
# Automatically loads data.json if present
The metadata JSON file contains:
- Video paths and backend metadata
- Skeleton definitions (nodes, edges, symmetries)
- Track names
- Suggested frames
- Provenance information
Format-Specific Examples¶
SLEAP Analysis Format¶
The default format matches SLEAP's "Export Analysis CSV" output:
Output columns: track, frame_idx, instance.score, {node}.x, {node}.y, {node}.score, ...
DeepLabCut Format¶
For compatibility with DeepLabCut workflows:
# Write DLC format with custom scorer name
sio.save_csv(labels, "dlc_output.csv", format="dlc", scorer="MyNetwork")
# Multi-animal DLC format (auto-detected from tracks)
sio.save_csv(multi_animal_labels, "multi_dlc.csv", format="dlc")
DLC format uses multi-row headers (scorer, bodyparts, coords) and is compatible with DLC's analysis tools.
DataFrame Codec Formats¶
For custom analysis pipelines, use the normalized formats from the DataFrame codec:
# Points format: most normalized (one row per point)
sio.save_csv(labels, "points.csv", format="points")
# Instances format: one row per instance
sio.save_csv(labels, "instances.csv", format="instances")
# Frames format: one row per frame (wide format)
sio.save_csv(labels, "frames.csv", format="frames")
sleap_io.io.main.load_csv(filename, format='auto', video=None, skeleton=None)
¶
Load pose data from a CSV file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to CSV file. |
required |
format
|
str
|
CSV format. One of "auto", "sleap", "dlc", "points", "instances", "frames". Default "auto" detects format from file content. |
'auto'
|
video
|
Video | str | None
|
Video to associate with data. Can be Video object or path string. |
None
|
skeleton
|
Skeleton | None
|
Skeleton to use. If None, inferred from columns or metadata. |
None
|
Returns:
| Type | Description |
|---|---|
Labels
|
Labels object. |
Notes
If a metadata JSON file exists alongside the CSV (same base name with .json extension), it will be automatically loaded to restore full Labels context including skeleton edges, symmetries, and provenance.
See Also
save_csv: Save Labels to CSV file.
Source code in sleap_io/io/main.py
def load_csv(
filename: str,
format: str = "auto",
video: "Video | str | None" = None,
skeleton: "Skeleton | None" = None,
) -> "Labels":
"""Load pose data from a CSV file.
Args:
filename: Path to CSV file.
format: CSV format. One of "auto", "sleap", "dlc", "points", "instances",
"frames". Default "auto" detects format from file content.
video: Video to associate with data. Can be Video object or path string.
skeleton: Skeleton to use. If None, inferred from columns or metadata.
Returns:
Labels object.
Notes:
If a metadata JSON file exists alongside the CSV (same base name with
.json extension), it will be automatically loaded to restore full
Labels context including skeleton edges, symmetries, and provenance.
See Also:
save_csv: Save Labels to CSV file.
"""
from sleap_io.io import csv
return csv.read_labels(filename, format=format, video=video, skeleton=skeleton)
sleap_io.io.main.save_csv(labels, filename, format='sleap', video=None, include_score=True, scorer='sleap-io', save_metadata=False)
¶
Save pose data to a CSV file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
Labels to save. |
required |
filename
|
str
|
Output path. |
required |
format
|
str
|
CSV format. One of "sleap" (default), "dlc", "points", "instances", "frames". |
'sleap'
|
video
|
Video | int | None
|
Video to filter to. Can be Video object or integer index. If None, includes all videos. |
None
|
include_score
|
bool
|
Include confidence scores in output. Default True. |
True
|
scorer
|
str
|
Scorer name for DLC format. Default "sleap-io". |
'sleap-io'
|
save_metadata
|
bool
|
Save JSON metadata file alongside CSV that enables full round-trip reconstruction. Default False. |
False
|
See Also
load_csv: Load Labels from CSV file.
Source code in sleap_io/io/main.py
def save_csv(
labels: "Labels",
filename: str,
format: str = "sleap",
video: "Video | int | None" = None,
include_score: bool = True,
scorer: str = "sleap-io",
save_metadata: bool = False,
) -> None:
"""Save pose data to a CSV file.
Args:
labels: Labels to save.
filename: Output path.
format: CSV format. One of "sleap" (default), "dlc", "points",
"instances", "frames".
video: Video to filter to. Can be Video object or integer index.
If None, includes all videos.
include_score: Include confidence scores in output. Default True.
scorer: Scorer name for DLC format. Default "sleap-io".
save_metadata: Save JSON metadata file alongside CSV that enables
full round-trip reconstruction. Default False.
See Also:
load_csv: Load Labels from CSV file.
"""
from sleap_io.io import csv
csv.write_labels(
labels,
filename,
format=format,
video=video,
include_score=include_score,
scorer=scorer,
save_metadata=save_metadata,
)
AlphaTracker Format¶
Load predictions from AlphaTracker, a tracking system for socially-housed animals.
sleap_io.io.main.load_alphatracker(filename)
¶
Read AlphaTracker annotations from a file and return a Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to the AlphaTracker annotation file in JSON format. |
required |
Returns:
| Type | Description |
|---|---|
Labels
|
Parsed labels as a |
Source code in sleap_io/io/main.py
def load_alphatracker(filename: str) -> Labels:
"""Read AlphaTracker annotations from a file and return a `Labels` object.
Args:
filename: Path to the AlphaTracker annotation file in JSON format.
Returns:
Parsed labels as a `Labels` instance.
"""
from sleap_io.io import alphatracker
return alphatracker.read_labels(filename)
LEAP Format (.mat)¶
Load predictions from LEAP, a SLEAP predecessor. Requires scipy for .mat file support.
sleap_io.io.main.load_leap(filename, skeleton=None, **kwargs)
¶
Load a LEAP dataset from a .mat file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str
|
Path to a LEAP .mat file. |
required |
skeleton
|
Skeleton | None
|
An optional |
None
|
**kwargs
|
Additional arguments (currently unused). |
required |
Returns:
| Type | Description |
|---|---|
Labels
|
The dataset as a |
Source code in sleap_io/io/main.py
def load_leap(
filename: str,
skeleton: Skeleton | None = None,
**kwargs,
) -> Labels:
"""Load a LEAP dataset from a .mat file.
Args:
filename: Path to a LEAP .mat file.
skeleton: An optional `Skeleton` object. If not provided, will be constructed
from the data in the file.
**kwargs: Additional arguments (currently unused).
Returns:
The dataset as a `Labels` object.
"""
from sleap_io.io import leap
return leap.read_labels(filename, skeleton=skeleton)
COCO Format (.json)¶
COCO (Common Objects in Context) format is widely used in computer vision and pose estimation. sleap-io provides full read and write support, making it compatible with tools like mmpose, CVAT, and other COCO-compatible frameworks.
sleap_io.io.main.load_coco(json_path, dataset_root=None, grayscale=False, **kwargs)
¶
Load a COCO-style pose dataset and return a Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
json_path
|
str
|
Path to the COCO annotation JSON file. |
required |
dataset_root
|
str | None
|
Root directory of the dataset. If None, uses parent directory of json_path. |
None
|
grayscale
|
bool
|
If True, load images as grayscale (1 channel). If False, load as RGB (3 channels). Default is False. |
False
|
**kwargs
|
Additional arguments (currently unused). |
required |
Returns:
| Type | Description |
|---|---|
Labels
|
The dataset as a |
Source code in sleap_io/io/main.py
def load_coco(
json_path: str,
dataset_root: str | None = None,
grayscale: bool = False,
**kwargs,
) -> Labels:
"""Load a COCO-style pose dataset and return a Labels object.
Args:
json_path: Path to the COCO annotation JSON file.
dataset_root: Root directory of the dataset. If None, uses parent directory
of json_path.
grayscale: If True, load images as grayscale (1 channel). If False, load as
RGB (3 channels). Default is False.
**kwargs: Additional arguments (currently unused).
Returns:
The dataset as a `Labels` object.
"""
from sleap_io.io import coco
return coco.read_labels(json_path, dataset_root=dataset_root, grayscale=grayscale)
sleap_io.io.main.save_coco(labels, json_path, image_filenames=None, visibility_encoding='ternary')
¶
Save a SLEAP dataset to COCO-style JSON annotation format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
A SLEAP |
required |
json_path
|
str
|
Path to save the COCO annotation JSON file. |
required |
image_filenames
|
str | list[str] | None
|
Optional image filenames to use in the COCO JSON. If provided, must be a single string (for single-frame videos) or a list of strings matching the number of labeled frames. If None, generates filenames from video filenames and frame indices. |
None
|
visibility_encoding
|
str
|
Visibility encoding to use. Either "binary" (0/1) or "ternary" (0/½). Default is "ternary". |
'ternary'
|
Notes
- This function only writes the JSON annotation file. It does not save images.
- The generated JSON can be used with mmpose and other COCO-compatible tools.
- For saving images along with annotations, you would need to extract and save frames separately.
Source code in sleap_io/io/main.py
def save_coco(
labels: Labels,
json_path: str,
image_filenames: str | list[str] | None = None,
visibility_encoding: str = "ternary",
):
"""Save a SLEAP dataset to COCO-style JSON annotation format.
Args:
labels: A SLEAP `Labels` object.
json_path: Path to save the COCO annotation JSON file.
image_filenames: Optional image filenames to use in the COCO JSON. If
provided, must be a single string (for single-frame videos) or
a list of strings matching the number of labeled frames. If
None, generates filenames from video filenames and frame
indices.
visibility_encoding: Visibility encoding to use. Either "binary" (0/1) or
"ternary" (0/1/2). Default is "ternary".
Notes:
- This function only writes the JSON annotation file. It does not save images.
- The generated JSON can be used with mmpose and other COCO-compatible tools.
- For saving images along with annotations, you would need to extract and save
frames separately.
"""
from sleap_io.io import coco
coco.write_labels(labels, json_path, image_filenames, visibility_encoding)
Ultralytics YOLO Format¶
Support for Ultralytics YOLO pose format.
sleap_io.io.main.load_ultralytics(dataset_path, split='train', skeleton=None, **kwargs)
¶
Load an Ultralytics YOLO pose dataset as a SLEAP Labels object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_path
|
str
|
Path to the Ultralytics dataset root directory containing data.yaml. |
required |
split
|
str
|
Dataset split to read ('train', 'val', or 'test'). Defaults to 'train'. |
'train'
|
skeleton
|
Skeleton | None
|
Optional skeleton to use. If not provided, will be inferred from data.yaml. |
None
|
**kwargs
|
Additional arguments passed to |
required |
Returns:
| Type | Description |
|---|---|
Labels
|
The dataset as a |
Source code in sleap_io/io/main.py
def load_ultralytics(
dataset_path: str,
split: str = "train",
skeleton: Skeleton | None = None,
**kwargs,
) -> Labels:
"""Load an Ultralytics YOLO pose dataset as a SLEAP `Labels` object.
Args:
dataset_path: Path to the Ultralytics dataset root directory containing
data.yaml.
split: Dataset split to read ('train', 'val', or 'test'). Defaults to 'train'.
skeleton: Optional skeleton to use. If not provided, will be inferred from
data.yaml.
**kwargs: Additional arguments passed to `ultralytics.read_labels`.
Currently supports:
- image_size: Tuple of (height, width) for coordinate denormalization.
Defaults to
(480, 640). Will attempt to infer from actual images if available.
Returns:
The dataset as a `Labels` object.
"""
from sleap_io.io import ultralytics
return ultralytics.read_labels(
dataset_path, split=split, skeleton=skeleton, **kwargs
)
sleap_io.io.main.save_ultralytics(labels, dataset_path, split_ratios={'train': 0.8, 'val': 0.2}, **kwargs)
¶
Save a SLEAP dataset to Ultralytics YOLO pose format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
A SLEAP |
required |
dataset_path
|
str
|
Path to save the Ultralytics dataset. |
required |
split_ratios
|
dict
|
Dictionary mapping split names to ratios (must sum to 1.0). Defaults to {"train": 0.8, "val": 0.2}. |
{'train': 0.8, 'val': 0.2}
|
**kwargs
|
Additional arguments passed to |
required |
Source code in sleap_io/io/main.py
def save_ultralytics(
labels: Labels,
dataset_path: str,
split_ratios: dict = {"train": 0.8, "val": 0.2},
**kwargs,
):
"""Save a SLEAP dataset to Ultralytics YOLO pose format.
Args:
labels: A SLEAP `Labels` object.
dataset_path: Path to save the Ultralytics dataset.
split_ratios: Dictionary mapping split names to ratios (must sum to 1.0).
Defaults to {"train": 0.8, "val": 0.2}.
**kwargs: Additional arguments passed to `ultralytics.write_labels`.
Currently supports:
- class_id: Class ID to use for all instances (default: 0).
- image_format: Image format to use for saving frames. Either "png"
(default, lossless) or "jpg".
- image_quality: Image quality for JPEG format (1-100). For PNG, this is
the compression
level (0-9). If None, uses default quality settings.
- verbose: If True (default), show progress bars during export.
- use_multiprocessing: If True, use multiprocessing for parallel image
saving. Default is False.
- n_workers: Number of worker processes. If None, uses CPU count - 1.
Only used if
use_multiprocessing=True.
"""
from sleap_io.io import ultralytics
ultralytics.write_labels(labels, dataset_path, split_ratios=split_ratios, **kwargs)
Working with Multiple Datasets¶
Load Multiple Files¶
Load and combine multiple pose tracking files:
sleap_io.io.main.load_labels_set(path, format=None, open_videos=True, **kwargs)
¶
Load a LabelsSet from multiple files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path | list[str | Path] | dict[str, str | Path]
|
Can be one of: - A directory path containing label files - A list of file paths - A dictionary mapping names to file paths |
required |
format
|
str | None
|
Optional format specification. If None, will try to infer from path. Supported formats: "slp", "ultralytics" |
None
|
open_videos
|
bool
|
If |
True
|
**kwargs
|
Additional format-specific arguments. |
required |
Returns:
| Type | Description |
|---|---|
LabelsSet
|
A LabelsSet containing the loaded Labels objects. |
Examples:
Load from SLP directory:
Load from list of SLP files:
Load from Ultralytics dataset:
Source code in sleap_io/io/main.py
def load_labels_set(
path: str | Path | list[str | Path] | dict[str, str | Path],
format: str | None = None,
open_videos: bool = True,
**kwargs,
) -> "LabelsSet":
"""Load a LabelsSet from multiple files.
Args:
path: Can be one of:
- A directory path containing label files
- A list of file paths
- A dictionary mapping names to file paths
format: Optional format specification. If None, will try to infer from path.
Supported formats: "slp", "ultralytics"
open_videos: If `True` (the default), attempt to open video backends.
**kwargs: Additional format-specific arguments.
Returns:
A LabelsSet containing the loaded Labels objects.
Examples:
Load from SLP directory:
>>> labels_set = load_labels_set("path/to/splits/")
Load from list of SLP files:
>>> labels_set = load_labels_set(["train.slp", "val.slp"])
Load from Ultralytics dataset:
>>> labels_set = load_labels_set("path/to/yolo_dataset/", format="ultralytics")
"""
# Try to infer format if not specified
if format is None:
if isinstance(path, (str, Path)):
path_obj = Path(path)
if path_obj.is_dir():
# Check for ultralytics structure
if (path_obj / "data.yaml").exists() or any(
(path_obj / split).exists() for split in ["train", "val", "test"]
):
format = "ultralytics"
else:
# Default to SLP for directories
format = "slp"
else:
# Single file path - check extension
if path_obj.suffix == ".slp":
format = "slp"
elif isinstance(path, list) and len(path) > 0:
# Check first file in list
first_path = Path(path[0])
if first_path.suffix == ".slp":
format = "slp"
elif isinstance(path, dict):
# Dictionary input defaults to SLP
format = "slp"
if format == "slp":
from sleap_io.io import slp
return slp.read_labels_set(path, open_videos=open_videos)
elif format == "ultralytics":
# Extract ultralytics-specific kwargs
splits = kwargs.pop("splits", None)
skeleton = kwargs.pop("skeleton", None)
image_size = kwargs.pop("image_size", (480, 640))
# Remove verbose from kwargs if present (for backward compatibility)
kwargs.pop("verbose", None)
if not isinstance(path, (str, Path)):
raise ValueError(
"Ultralytics format requires a directory path, "
f"got {type(path).__name__}"
)
from sleap_io.io import ultralytics
return ultralytics.read_labels_set(
str(path),
splits=splits,
skeleton=skeleton,
image_size=image_size,
)
else:
raise ValueError(
f"Unknown format: {format}. Supported formats: 'slp', 'ultralytics'"
)
Skeleton Files¶
Load and save skeleton definitions separately:
sleap_io.io.main.load_skeleton(filename)
¶
Load skeleton(s) from a JSON, YAML, or SLP file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str | Path
|
Path to a skeleton file. Supported formats: - JSON: Standalone skeleton or training config with embedded skeletons - YAML: Simplified skeleton format - SLP: SLEAP project file |
required |
Returns:
| Type | Description |
|---|---|
Skeleton | list[Skeleton]
|
A single |
Notes
This function loads skeletons from various file types: - JSON files: Can be standalone skeleton files (jsonpickle format) or training config files with embedded skeletons - YAML files: Use a simplified human-readable format - SLP files: Extracts skeletons from SLEAP project files The format is detected based on the file extension and content.
Source code in sleap_io/io/main.py
def load_skeleton(filename: str | Path) -> Skeleton | list[Skeleton]:
"""Load skeleton(s) from a JSON, YAML, or SLP file.
Args:
filename: Path to a skeleton file. Supported formats:
- JSON: Standalone skeleton or training config with embedded skeletons
- YAML: Simplified skeleton format
- SLP: SLEAP project file
Returns:
A single `Skeleton` or list of `Skeleton` objects.
Notes:
This function loads skeletons from various file types:
- JSON files: Can be standalone skeleton files (jsonpickle format) or training
config files with embedded skeletons
- YAML files: Use a simplified human-readable format
- SLP files: Extracts skeletons from SLEAP project files
The format is detected based on the file extension and content.
"""
if isinstance(filename, Path):
filename = str(filename)
# Detect format based on extension
if filename.lower().endswith(".slp"):
# SLP format - extract skeletons from SLEAP file
from sleap_io.io.slp import read_skeletons
return read_skeletons(filename)
elif filename.lower().endswith((".yaml", ".yml")):
# YAML format
with open(filename, "r") as f:
yaml_data = f.read()
return decode_yaml_skeleton(yaml_data)
else:
# JSON format (default) - could be standalone or training config
with open(filename, "r") as f:
json_data = f.read()
return load_skeleton_from_json(json_data)
sleap_io.io.main.save_skeleton(skeleton, filename)
¶
Save skeleton(s) to a JSON or YAML file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
skeleton
|
Skeleton | list[Skeleton]
|
A single |
required |
filename
|
str | Path
|
Path to save the skeleton file. |
required |
Notes
This function saves skeletons in either JSON or YAML format based on the file extension. JSON files use the jsonpickle format compatible with SLEAP, while YAML files use a simplified human-readable format.
Source code in sleap_io/io/main.py
def save_skeleton(skeleton: Skeleton | list[Skeleton], filename: str | Path):
"""Save skeleton(s) to a JSON or YAML file.
Args:
skeleton: A single `Skeleton` or list of `Skeleton` objects to save.
filename: Path to save the skeleton file.
Notes:
This function saves skeletons in either JSON or YAML format based on the
file extension. JSON files use the jsonpickle format compatible with SLEAP,
while YAML files use a simplified human-readable format.
"""
if isinstance(filename, Path):
filename = str(filename)
# Detect format based on extension
if filename.lower().endswith((".yaml", ".yml")):
# YAML format
yaml_data = encode_yaml_skeleton(skeleton)
with open(filename, "w") as f:
f.write(yaml_data)
else:
# JSON format (default)
json_data = encode_skeleton(skeleton)
with open(filename, "w") as f:
f.write(json_data)
Format Detection¶
sleap-io automatically detects file formats based on:
- File extension:
.slp,.nwb,.h5,.json,.mat,.csv - File content: For ambiguous extensions like
.h5(JABS vs DLC) or.json(Label Studio vs COCO) - Explicit format: Pass
formatparameter to override auto-detection
Format Conversion Examples¶
Convert Between Formats¶
import sleap_io as sio
# Load from any supported format
labels = sio.load_file("data.slp")
# Save to different formats
labels.save("data.nwb") # NWB format
labels.save("data.labelstudio.json") # Label Studio
labels.save("data_yolo/") # Ultralytics YOLO
Batch Conversion¶
import sleap_io as sio
from pathlib import Path
# Convert all SLEAP files to NWB
for slp_file in Path("data/").glob("*.slp"):
labels = sio.load_file(slp_file)
nwb_file = slp_file.with_suffix(".nwb")
labels.save(nwb_file)
Round-Trip Preservation¶
Most formats preserve data during round-trip conversion:
import sleap_io as sio
# Load original
labels_original = sio.load_file("data.slp")
# Save and reload
labels_original.save("temp.nwb")
labels_reloaded = sio.load_file("temp.nwb")
# Data is preserved
assert len(labels_original) == len(labels_reloaded)
assert labels_original.skeleton == labels_reloaded.skeleton
Format Limitations¶
Different formats have varying capabilities:
| Format | Read | Write | Videos | Skeletons | Tracks | Confidence | User/Predicted |
|---|---|---|---|---|---|---|---|
| SLEAP (.slp) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| NWB (.nwb) | ✅ | ✅ | ✅* | ✅ | ✅ | ✅ | ✅ |
| JABS (.h5) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| Analysis HDF5 | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ |
| Label Studio | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |
| CSV (.csv) | ✅ | ✅ | ❌ | ✅** | ✅ | ✅ | ❌ |
| DeepLabCut | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |
| AlphaTracker | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ |
| LEAP (.mat) | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| COCO (.json) | ✅ | ✅ | ❌ | ✅ | ✅*** | ❌ | ✅ |
| Ultralytics | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ❌ |
*NWB can embed videos with annotations_export format
**CSV skeleton edges/symmetries preserved via optional metadata JSON sidecar
***COCO tracks are stored via attributes.object_id (CVAT-compatible)
See Also¶
- Data Model: Understanding the core data structures
- Examples: More usage examples and recipes
- Merging: Combining data from multiple sources