Rendering¶
sleap-io provides high-performance pose visualization using skia-python, a production-quality 2D graphics library.
Quick Start¶
Render a single frame to a numpy array:
import sleap_io as sio
labels = sio.load_slp("predictions.slp")
img = sio.render_image(labels.labeled_frames[0])
Render a full video:
This produces an MP4 video file with skeleton overlays on all labeled frames.
Color Schemes¶
Color scheme determines how poses are colored across instances and frames. The
color_by parameter defaults to "auto", which resolves to "track" when tracks
are present, otherwise "instance" for a single image and "node" for a video or
multi-image render. Pass an explicit value to override:
Color by track¶
Each tracked animal gets a consistent color across all frames:
Color by instance¶
Each animal within a frame gets a unique color (colors may change between frames):
Color by node¶
Each body part gets a unique color (same across all animals):
Color Palettes¶
Built-in palettes¶
9 palettes are included with no additional dependencies.
standard¶
MATLAB default colors (default palette):
distinct¶
High-contrast colors for instances/tracks:
rainbow¶
Spectrum colors for node types:
warm¶
Orange/red tones:
cool¶
Blue/purple tones:
pastel¶
Subtle colors for overlays:
seaborn¶
Professional look for publications:
tableau10¶
Data visualization standard:
viridis¶
Perceptually uniform scientific:
Colorcet palettes¶
sleap-io includes colorcet palettes:
glasbey¶
256 maximally distinct colors:
glasbey_hv¶
High visibility variant:
glasbey_cool¶
Cool-toned variant:
glasbey_warm¶
Warm-toned variant:
Getting palette colors programmatically¶
colors = sio.rendering.get_palette("tableau10", 10)
# Returns: [(31, 119, 180), (255, 127, 14), ...]
Marker Shapes¶
Five marker shapes are available for node visualization.
circle¶
Filled circle (default):
square¶
Filled square:
diamond¶
Rotated square:
triangle¶
Upward-pointing triangle:
cross¶
Plus sign:
Styling Options¶
Marker and line sizes¶
Small markers and thin lines for detailed work:
Medium markers and lines:
Large markers and thick lines for visibility:
Transparency¶
Full opacity (default):
Semi-transparent overlay:
Subtle overlay:
Toggle elements¶
Both nodes and edges (default):
Edges only:
Nodes only:
Motion trails¶
Motion trails draw the trajectory of a node or centroid over the last
trail_length frames, so you can see how each animal moved through the
rendered output. Trails are drawn behind the poses and colored to match them
(by track when tracks are present, otherwise by instance).
sio.render_video(
labels,
save_path="output.mp4",
show_trails=True, # enable trail drawing
trail_length=10, # number of past frames to trace
trail_node="centroid", # node name, list of node names, or "centroid"
trail_width=2.0, # trail line width in pixels
trail_alpha_fade=True, # fade from faint (oldest) to opaque (newest)
trail_alpha=1.0, # global trail opacity
trail_color=None, # uniform color, or None to match pose colors
)
trail_node accepts:
"centroid"(default): trail the mean of each instance's visible nodes.- A node name (e.g.
"head"): trail that single node. - A list of node names (e.g.
["head", "thorax"]): draw one trail per node.
By default trails are colored to match the poses (by track when tracks are
present, otherwise by instance). Pass trail_color to override this with a
single uniform color for all trails — it accepts any color spec (RGB tuple,
named color, hex, or palette index). trail_alpha scales the overall opacity
and combines with trail_alpha_fade.
# Faint white trails, uniform color, no fade.
sio.render_video(
labels,
save_path="output.mp4",
show_trails=True,
trail_color="white",
trail_alpha=0.5,
trail_alpha_fade=False,
)
Trails need temporal context, so they are only drawn for videos or for
render_image when the source is a Labels object (so past frames are
available). They are silently skipped for a single LabeledFrame or a list of
instances.
# Trails also work on a single rendered frame from a Labels object.
img = sio.render_image(labels, lf_ind=100, show_trails=True, trail_length=20)
Note
For untracked data, trails are matched by instance index across frames, which can jump between animals if the per-frame ordering changes. Trails are most reliable on tracked data.
Scaling and Cropping¶
Control output resolution and focus on regions of interest.
Output scaling¶
The scale parameter resizes the output. Graphics (markers, lines) scale proportionally:
# Full resolution (default)
img = sio.render_image(lf, scale=1.0)
# Half resolution - faster, smaller files
img = sio.render_image(lf, scale=0.5)
# Quarter resolution - quick preview
img = sio.render_image(lf, scale=0.25)
Cropping to a region¶
Use the crop parameter to render a specific region. Bounds are (x1, y1, x2, y2) where (x1, y1) is the top-left corner and (x2, y2) is the bottom-right (exclusive). Origin (0, 0) is at the image top-left.
Cropping works for both single images and videos. For videos, the same crop region is applied uniformly to all frames.
Pixel coordinates (integer tuple):
import sleap_io as sio
labels = sio.load_slp("predictions.slp")
lf = labels.labeled_frames[0]
# Crop to region (x1, y1, x2, y2) in pixels
img = sio.render_image(lf, crop=(100, 100, 300, 300))
# Same for video rendering
sio.render_video(labels, "cropped.mp4", crop=(100, 100, 300, 300))
Normalized coordinates (float tuple in [0.0, 1.0]):
# Crop center 50% of the frame
img = sio.render_image(lf, crop=(0.25, 0.25, 0.75, 0.75))
# Crop right half of the frame
img = sio.render_image(lf, crop=(0.5, 0.0, 1.0, 1.0))
Detection is type-based: all values must be Python float type and in [0.0, 1.0] range. Values outside this range (e.g., (100.0, 100.0, 300.0, 300.0)) are treated as pixel coordinates.
Zoomed crop¶
Combine cropping with scaling for zoomed-in views:
# Crop a small region and scale up for zoom effect
img = sio.render_image(lf, crop=(140, 120, 240, 220), scale=2.0)
Background Control¶
Control the background when rendering poses. Use background="video" (default) to load video frames, or specify a color to render with a solid background.
Remote video backgrounds
With background="video", frames are read from the underlying Video backend, which can be a remote/cloud/Google-Drive source (see Remote loading). Rendering then incurs the same network fetch, caching, and RemoteIOError behavior as any other frame access.
Named color¶
RGB tuple¶
Hex color¶
Palette color¶
Color specification formats¶
The background parameter accepts many formats:
| Format | Example | Description |
|---|---|---|
| Named color | "black", "white", "gray" |
Predefined color names |
| Hex (6-digit) | "#ff8000" |
Standard hex color |
| Hex (3-digit) | "#f80" |
Shorthand hex |
| RGB int tuple | (255, 128, 0) |
Values 0-255 |
| RGB float tuple | (1.0, 0.5, 0.0) |
Values 0.0-1.0 |
| Grayscale int | 40 |
Single value 0-255 |
| Grayscale float | 0.15 |
Single value 0.0-1.0 |
| Palette index | "tableau10[0]" |
Color from palette |
Available named colors: black, white, red, green, blue, yellow, cyan, magenta, gray/grey, orange, purple, pink, brown.
Creating Montages¶
Render multiple frames and combine them:
import numpy as np
import sleap_io as sio
labels = sio.load_slp("predictions.slp")
frame_indices = [0, 100, 200, 300, 400]
frames = []
for i in frame_indices:
img = sio.render_image(labels.labeled_frames[i], color_by="track")
frames.append(img)
montage = np.concatenate(frames, axis=1)
Custom Rendering with Callbacks¶
Callbacks let you add custom graphics. You get direct access to the Skia canvas.
There are three callback types:
| Callback | Context Type | When Called |
|---|---|---|
pre_render_callback |
RenderContext |
Before poses are drawn |
post_render_callback |
RenderContext |
After all poses are drawn |
per_instance_callback |
InstanceContext |
After each instance is drawn |
Instance labels¶
Draw track names above each instance:
import skia
from sleap_io.rendering import InstanceContext
def draw_labels(ctx: InstanceContext):
centroid = ctx.get_centroid()
if centroid is None:
return
cx, cy = ctx.world_to_canvas(*centroid)
font = skia.Font(skia.Typeface("Arial"), 14)
label = ctx.track_name or f"Instance {ctx.instance_idx}"
blob = skia.TextBlob(label, font)
# Background
bounds = font.measureText(label)
bg = skia.Paint(Color=skia.Color4f(0, 0, 0, 0.6))
ctx.canvas.drawRect(skia.Rect(cx - 2, cy - 18, cx + bounds + 2, cy - 4), bg)
# Text
paint = skia.Paint(Color=skia.ColorWHITE, AntiAlias=True)
ctx.canvas.drawTextBlob(blob, cx, cy - 6, paint)
img = sio.render_image(lf, per_instance_callback=draw_labels)
Bounding boxes¶
Draw dashed bounding boxes around instances:
import skia
from sleap_io.rendering import InstanceContext
def draw_bbox(ctx: InstanceContext):
bbox = ctx.get_bbox()
if bbox is None:
return
x1, y1, x2, y2 = bbox
x1, y1 = ctx.world_to_canvas(x1, y1)
x2, y2 = ctx.world_to_canvas(x2, y2)
pad = 8
rect = skia.Rect(x1 - pad, y1 - pad, x2 + pad, y2 + pad)
dash = skia.DashPathEffect.Make([6, 3], 0)
paint = skia.Paint(
Color=skia.ColorWHITE,
Style=skia.Paint.kStroke_Style,
StrokeWidth=2,
PathEffect=dash,
)
ctx.canvas.drawRect(rect, paint)
img = sio.render_image(lf, per_instance_callback=draw_bbox)
Frame info overlay¶
Add frame number and instance count:
import skia
from sleap_io.rendering import RenderContext
def draw_frame_info(ctx: RenderContext):
font = skia.Font(skia.Typeface("Arial"), 14)
text = f"Frame: {ctx.frame_idx} Instances: {len(ctx.instances)}"
blob = skia.TextBlob(text, font)
bg = skia.Paint(Color=skia.Color4f(0, 0, 0, 0.7))
ctx.canvas.drawRect(skia.Rect(4, 4, 200, 24), bg)
paint = skia.Paint(Color=skia.ColorWHITE, AntiAlias=True)
ctx.canvas.drawTextBlob(blob, 8, 18, paint)
img = sio.render_image(lf, post_render_callback=draw_frame_info)
Combining callbacks¶
from sleap_io.rendering import InstanceContext
def combined_per_instance(ctx: InstanceContext):
draw_bbox(ctx)
draw_labels(ctx)
img = sio.render_image(
lf,
per_instance_callback=combined_per_instance,
post_render_callback=draw_frame_info,
)
Segmentation Overlays¶
Overlay segmentation masks on images via render_image/render_video or standalone functions.
Masks and label images are drawn automatically
When no explicit overlay is passed and the labels carry annotations,
render_video (and sio render) auto-draw them: label images take
precedence, falling back to SegmentationMask annotations
(labels.get_masks(video=...), routed to the correct frame). render_image
auto-draws the frame's masks when no overlay is given. So
sio render preds.slp -o out.mp4 draws segmentation predictions out of the
box. Pass an explicit overlay to override. Auto-drawing works on both RGB
and single-channel grayscale (H, W, 1) video.
Coloring overlays by track
With color_by="track" (the default "auto" when the labels carry tracks),
SegmentationMask, ROI, and BoundingBox overlays are colored by their
.track identity using the pose palette, so a tracked object keeps a
stable color across frames and matches its poses, centroids, and trails.
Untracked elements use the first palette color. When color_by is not
"track" (or the overlay carries no tracks), elements fall back to
positional coloring from overlay_palette (default "distinct").
Using render_image / render_video¶
The overlay parameter on render_image and render_video accepts label images, SegmentationMask, ROI, or BoundingBox objects:
import numpy as np
import sleap_io as sio
# Label image overlay (no poses)
img = sio.render_image(image=frame, overlay=label_mask, overlay_alpha=0.4)
# With outlines
img = sio.render_image(
image=frame,
overlay=label_mask,
overlay_alpha=0.4,
overlay_outline=True,
)
# SegmentationMask, ROI, or BoundingBox objects (a list, or a single object)
img = sio.render_image(image=frame, overlay=masks, overlay_alpha=0.3)
img = sio.render_image(image=frame, overlay=rois, overlay_alpha=0.3)
img = sio.render_image(image=frame, overlay=bboxes, overlay_alpha=0.3)
# A single annotation object also works (treated like a one-element list)
img = sio.render_image(image=frame, overlay=masks[0], overlay_alpha=0.3)
# Overlay on a labeled frame (poses render on top)
img = sio.render_image(lf, overlay=label_mask, overlay_alpha=0.4)
For video rendering, overlay can be a 3-D array (T, H, W), a callable, or a list of objects with frame_idx:
# 3-D label image stack
sio.render_video(labels, "output.mp4", overlay=label_stack)
# Callable for lazy per-frame loading
sio.render_video(labels, "output.mp4", overlay=lambda idx: load_mask(idx))
# SegmentationMask objects (filtered per-frame by frame_idx)
sio.render_video(labels, "output.mp4", overlay=masks)
Standalone functions¶
For direct control, use sio.draw_label_image, sio.draw_masks, sio.draw_rois, sio.draw_bboxes, or sio.draw_centroids:
Label images (instance/panoptic segmentation)¶
# label_image: (H, W) int array, 0=background, 1..N=object IDs
sio.draw_label_image(image, label_image, alpha=0.4, palette="distinct")
Add outlines around each segment:
sio.draw_label_image(
image,
label_image,
alpha=0.4,
palette="distinct",
outline=True,
outline_width=2,
)
Use a uniform outline color:
Binary segmentation masks¶
For SegmentationMask objects, use sio.draw_masks with per-mask coloring:
# Single color for all masks
sio.draw_masks(image, masks, color=(255, 0, 0), alpha=0.3)
# Per-mask colors
colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255)]
sio.draw_masks(image, masks, colors=colors, alpha=0.3)
ROI geometries¶
For ROI objects (polygons, points, lines), use sio.draw_rois:
Bounding boxes¶
For BoundingBox objects, use sio.draw_bboxes:
Centroids¶
For Centroid objects, use sio.draw_centroids to draw a filled marker at each
position:
Video Rendering¶
Basic video rendering¶
Render a clip¶
Quality presets¶
# Fast preview (0.25x resolution)
sio.render_video(labels, "preview.mp4", preset="preview")
# Draft quality (0.5x resolution)
sio.render_video(labels, "draft.mp4", preset="draft")
# Full quality (default)
sio.render_video(labels, "final.mp4", preset="final")
Encoding options¶
sio.render_video(
labels,
"output.mp4",
fps=30.0, # Output frame rate
crf=18, # Quality (2-32, lower=better)
x264_preset="slow", # Encoding speed
)
CLI Reference¶
For CLI usage, see the CLI Guide.
# Basic rendering
sio render -i predictions.slp -o output.mp4
# Fast preview
sio render -i predictions.slp --preset preview
# Single frame to PNG
sio render -i predictions.slp --lf 0 -o frame.png
# Custom styling
sio render -i predictions.slp -o styled.mp4 \
--color-by track --palette tableau10 --marker-shape diamond
# Motion trails
sio render -i predictions.slp -o output.mp4 --trails --trail-length 10
# Trails for specific nodes
sio render -i predictions.slp -o output.mp4 --trails --trail-node head,thorax
# Styled trails (uniform color, faint, no fade)
sio render -i predictions.slp -o output.mp4 --trails \
--trail-color white --trail-alpha 0.5 --no-trail-fade
# Disable the progress bar
sio render -i predictions.slp -o output.mp4 --no-progress
# Segmentation overlay from TIFF stack
sio render -i predictions.slp --overlay masks.tif --overlay-alpha 0.4
# Overlay from directory of per-frame TIFFs
sio render -i predictions.slp --overlay masks/
# Single frame with overlay and outlines
sio render -i predictions.slp --lf 0 --overlay masks.tif \
--overlay-outline --overlay-outline-color white
# Overlay-only mode (no labels file needed)
sio render --images frames/ --overlay masks.tif -o output.mp4
# Overlay-only with TIFF stack images
sio render --images frames.tif --overlay masks/ --overlay-outline -o output.mp4
# Color overlay with a specific palette
sio render -i predictions.slp --overlay masks.tif --overlay-palette tableau10
# Discover palettes and named colors
sio render --list-palettes
sio render --list-colors
API Reference¶
sleap_io.render_video(source, save_path=None, *, video=None, frame_inds=None, start=None, end=None, include_unlabeled=None, overlay=None, overlay_alpha=0.3, overlay_palette='distinct', overlay_outline=False, overlay_outline_width=1, overlay_outline_color=None, crop=None, preset=None, scale=1.0, color_by='auto', palette='standard', marker_shape='circle', marker_size=4.0, line_width=2.0, alpha=1.0, show_nodes=True, show_edges=True, show_centroids=True, centroid_marker_size=5.0, show_trails=False, trail_length=10, trail_node='centroid', trail_width=2.0, trail_alpha_fade=True, trail_alpha=1.0, trail_color=None, fps=None, codec='libx264', crf=25, x264_preset='superfast', background='video', pre_render_callback=None, post_render_callback=None, per_instance_callback=None, progress_callback=None, show_progress=True)
¶
Render video with pose overlays.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Labels | list[LabeledFrame]
|
Labels object or list of LabeledFrames to render. |
required |
save_path
|
str | Path | None
|
Output video path. If None, returns list of rendered arrays. |
None
|
video
|
Video | int | None
|
Video to render from (default: first video in Labels). |
None
|
frame_inds
|
list[int] | None
|
Specific frame indices to render. |
None
|
start
|
int | None
|
Start frame index (inclusive). |
None
|
end
|
int | None
|
End frame index (exclusive). |
None
|
include_unlabeled
|
bool | None
|
If True, render all frames in range even if they have no LabeledFrame (just shows video frame without poses). Default None, which resolves to False unless auto-overlay detection enables it (when label_images exist for the target video and no explicit overlay is given). |
None
|
overlay
|
ndarray | list[LabelImage] | list[SegmentationMask] | list[ROI] | list[BoundingBox] | Callable[[int], ndarray] | None
|
Per-frame annotation overlay. Accepts:
|
None
|
overlay_alpha
|
float
|
Opacity for the annotation overlay (0.0 to 1.0). |
0.3
|
overlay_palette
|
Literal | str
|
Color palette for overlay coloring. |
'distinct'
|
overlay_outline
|
bool
|
Draw outlines around segmented regions (label images). |
False
|
overlay_outline_width
|
int
|
Outline width in pixels. |
1
|
overlay_outline_color
|
tuple[int, int, int] | None
|
RGB outline color, or |
None
|
crop
|
CropSpec
|
Static crop applied uniformly to all frames. Bounds are (x1, y1, x2, y2) where (x1, y1) is the top-left corner and (x2, y2) is the bottom-right (exclusive). Supports:
|
None
|
preset
|
Literal['preview', 'draft', 'final'] | None
|
Quality preset ('preview'=0.25x, 'draft'=0.5x, 'final'=1.0x). |
None
|
scale
|
float
|
Scale factor (overrides preset if both provided). |
1.0
|
color_by
|
Literal
|
Color scheme - 'track', 'instance', 'node', 'identity', 'category', or 'auto'. |
'auto'
|
palette
|
Literal | str
|
Color palette name. |
'standard'
|
marker_shape
|
Literal
|
Node marker shape. |
'circle'
|
marker_size
|
float
|
Node marker radius in pixels. |
4.0
|
line_width
|
float
|
Edge line width in pixels. |
2.0
|
alpha
|
float
|
Global transparency (0.0-1.0). |
1.0
|
show_nodes
|
bool
|
Whether to draw node markers. |
True
|
show_edges
|
bool
|
Whether to draw skeleton edges. |
True
|
show_centroids
|
bool
|
Whether to draw centroid markers from
|
True
|
centroid_marker_size
|
float
|
Radius of centroid markers in pixels. |
5.0
|
show_trails
|
bool
|
Whether to draw motion trails tracing node or centroid positions over past frames. |
False
|
trail_length
|
int
|
Number of past frames behind each frame to include in the trail. |
10
|
trail_node
|
str | list[str]
|
Which point to trail. One of |
'centroid'
|
trail_width
|
float
|
Trail line width in pixels. |
2.0
|
trail_alpha_fade
|
bool
|
If |
True
|
trail_alpha
|
float
|
Global opacity multiplier for trails (0.0 to 1.0). Combines
with |
1.0
|
trail_color
|
ColorSpec | None
|
Uniform color for all trails. If |
None
|
fps
|
float | None
|
Output frame rate (default: source video fps). |
None
|
codec
|
str
|
Video codec for encoding. |
'libx264'
|
crf
|
int
|
Constant rate factor for quality (2-32, lower=better). Default 25. |
25
|
x264_preset
|
str
|
H.264 encoding preset (ultrafast, superfast, fast, medium, slow). |
'superfast'
|
background
|
Literal['video'] | ColorSpec
|
Background control. Can be:
- |
'video'
|
pre_render_callback
|
Callable[[RenderContext], None] | None
|
Called before each frame's poses are drawn. |
None
|
post_render_callback
|
Callable[[RenderContext], None] | None
|
Called after each frame's poses are drawn. |
None
|
per_instance_callback
|
Callable[[InstanceContext], None] | None
|
Called after each instance is drawn. |
None
|
progress_callback
|
Callable[[int, int], bool] | None
|
Called with (current, total), return False to cancel. |
None
|
show_progress
|
bool
|
Show tqdm progress bar. |
True
|
Returns:
| Type | Description |
|---|---|
Video | list[ndarray]
|
If save_path provided: Video object pointing to output file. If save_path is None: List of rendered numpy arrays (H, W, 3) uint8. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If background="video" and video unavailable. |
Examples:
Render full video with pose overlays:
>>> import sleap_io as sio
>>> labels = sio.load_slp("predictions.slp")
>>> sio.render_video(labels, "output.mp4")
Fast preview at reduced resolution:
Get rendered frames as numpy arrays:
Source code in sleap_io/rendering/core.py
def render_video(
source: "Labels | list[LabeledFrame]",
save_path: str | Path | None = None,
*,
# Video selection
video: "Video | int | None" = None,
# Frame selection
frame_inds: list[int] | None = None,
start: int | None = None,
end: int | None = None,
include_unlabeled: bool | None = None,
# Annotation overlay
overlay: (
"np.ndarray"
" | list[LabelImage] | list[SegmentationMask] | list[ROI] | list[BoundingBox]"
" | Callable[[int], np.ndarray] | None"
) = None,
overlay_alpha: float = 0.3,
overlay_palette: PaletteName | str = "distinct",
overlay_outline: bool = False,
overlay_outline_width: int = 1,
overlay_outline_color: tuple[int, int, int] | None = None,
# Cropping
crop: CropSpec = None,
# Quality/scale
preset: Literal["preview", "draft", "final"] | None = None,
scale: float = 1.0,
# Appearance
color_by: ColorScheme = "auto",
palette: PaletteName | str = "standard",
marker_shape: MarkerShape = "circle",
marker_size: float = 4.0,
line_width: float = 2.0,
alpha: float = 1.0,
show_nodes: bool = True,
show_edges: bool = True,
show_centroids: bool = True,
centroid_marker_size: float = 5.0,
# Motion trails
show_trails: bool = False,
trail_length: int = 10,
trail_node: str | list[str] = "centroid",
trail_width: float = 2.0,
trail_alpha_fade: bool = True,
trail_alpha: float = 1.0,
trail_color: ColorSpec | None = None,
# Video encoding
fps: float | None = None,
codec: str = "libx264",
crf: int = 25,
x264_preset: str = "superfast",
# Background control
background: Literal["video"] | ColorSpec = "video",
# Callbacks
pre_render_callback: Callable[[RenderContext], None] | None = None,
post_render_callback: Callable[[RenderContext], None] | None = None,
per_instance_callback: Callable[[InstanceContext], None] | None = None,
# Progress
progress_callback: Callable[[int, int], bool] | None = None,
show_progress: bool = True,
) -> "Video | list[np.ndarray]":
"""Render video with pose overlays.
Args:
source: Labels object or list of LabeledFrames to render.
save_path: Output video path. If None, returns list of rendered arrays.
video: Video to render from (default: first video in Labels).
frame_inds: Specific frame indices to render.
start: Start frame index (inclusive).
end: End frame index (exclusive).
include_unlabeled: If True, render all frames in range even if they have
no LabeledFrame (just shows video frame without poses). Default None,
which resolves to False unless auto-overlay detection enables it (when
label_images exist for the target video and no explicit overlay is given).
overlay: Per-frame annotation overlay. Accepts:
- ``np.ndarray``: 3-D array ``(T, H, W)`` of integer label images
indexed by frame number, or 2-D ``(H, W)`` for a static overlay.
- ``list[SegmentationMask | ROI | BoundingBox]``: Indexed by
position — the item at list index ``i`` is applied to frame
``i``. One overlay per frame.
- ``Callable[[int], np.ndarray]``: Called with the frame index,
returns an ``(H, W)`` label image for that frame.
overlay_alpha: Opacity for the annotation overlay (0.0 to 1.0).
overlay_palette: Color palette for overlay coloring.
overlay_outline: Draw outlines around segmented regions (label images).
overlay_outline_width: Outline width in pixels.
overlay_outline_color: RGB outline color, or ``None`` for auto-darkened.
crop: Static crop applied uniformly to all frames. Bounds are
(x1, y1, x2, y2) where (x1, y1) is the top-left corner and (x2, y2)
is the bottom-right (exclusive). Supports:
- **Pixel coordinates** (int tuple): ``(100, 100, 300, 300)``
- **Normalized coordinates** (float tuple in [0.0, 1.0]):
``(0.25, 0.25, 0.75, 0.75)`` crops the center 50%.
- ``None``: No cropping (default).
preset: Quality preset ('preview'=0.25x, 'draft'=0.5x, 'final'=1.0x).
scale: Scale factor (overrides preset if both provided).
color_by: Color scheme - 'track', 'instance', 'node', 'identity',
'category', or 'auto'.
palette: Color palette name.
marker_shape: Node marker shape.
marker_size: Node marker radius in pixels.
line_width: Edge line width in pixels.
alpha: Global transparency (0.0-1.0).
show_nodes: Whether to draw node markers.
show_edges: Whether to draw skeleton edges.
show_centroids: Whether to draw centroid markers from
``Labels.centroids``. Centroids are colored by track.
centroid_marker_size: Radius of centroid markers in pixels.
show_trails: Whether to draw motion trails tracing node or centroid
positions over past frames.
trail_length: Number of past frames behind each frame to include in the
trail.
trail_node: Which point to trail. One of ``"centroid"`` (default), a
node name, or a list of node names (one trail per node).
trail_width: Trail line width in pixels.
trail_alpha_fade: If ``True``, fade trails from faint (oldest) to opaque
(newest).
trail_alpha: Global opacity multiplier for trails (0.0 to 1.0). Combines
with ``trail_alpha_fade``.
trail_color: Uniform color for all trails. If ``None`` (default), trails
are colored to match the poses (by track or instance). Accepts any
color spec (RGB tuple, named color, hex, or palette index).
fps: Output frame rate (default: source video fps).
codec: Video codec for encoding.
crf: Constant rate factor for quality (2-32, lower=better). Default 25.
x264_preset: H.264 encoding preset (ultrafast, superfast, fast, medium, slow).
background: Background control. Can be:
- ``"video"``: Load video frame (default). Raises error if unavailable.
- Any color spec: Use solid color background, skip video loading entirely.
Supports RGB tuples ``(255, 128, 0)``, float tuples ``(1.0, 0.5, 0.0)``,
grayscale ``128`` or ``0.5``, named colors ``"black"``, hex ``"#ff8000"``,
or palette index ``"tableau10[2]"``.
pre_render_callback: Called before each frame's poses are drawn.
post_render_callback: Called after each frame's poses are drawn.
per_instance_callback: Called after each instance is drawn.
progress_callback: Called with (current, total), return False to cancel.
show_progress: Show tqdm progress bar.
Returns:
If save_path provided: Video object pointing to output file.
If save_path is None: List of rendered numpy arrays (H, W, 3) uint8.
Raises:
ValueError: If background="video" and video unavailable.
Examples:
Render full video with pose overlays:
>>> import sleap_io as sio
>>> labels = sio.load_slp("predictions.slp")
>>> sio.render_video(labels, "output.mp4")
Fast preview at reduced resolution:
>>> sio.render_video(labels, "preview.mp4", preset="preview")
Get rendered frames as numpy arrays:
>>> frames = sio.render_video(labels)
"""
import skia # noqa: F401
from sleap_io.model.labeled_frame import LabeledFrame
from sleap_io.model.labels import Labels
from sleap_io.model.video import Video as VideoModel
# Handle background parameter
use_video = background == "video"
background_color: tuple[int, int, int] | None = None
if not use_video:
background_color = resolve_color(background)
# Handle preset
if preset is not None and preset in PRESETS:
scale = PRESETS[preset]["scale"]
# Whether this video has centroids to render (populated below).
_has_video_centroids: bool = False
# Resolve source
if isinstance(source, Labels):
labels = source
# Resolve video
if video is None:
if not labels.videos:
raise ValueError("Labels has no videos")
target_video = labels.videos[0]
elif isinstance(video, int):
target_video = labels.videos[video]
else:
target_video = video
# Get labeled frames for this video
labeled_frames = labels.find(target_video)
# Sort by frame index
labeled_frames = sorted(labeled_frames, key=lambda lf: lf.frame_idx)
# Check for spatial annotations (label_images, masks, bboxes, rois,
# centroids) that can be rendered even without labeled frames (poses).
has_spatial = bool(
labels.get_label_images(video=target_video)
or labels.get_masks(video=target_video)
or labels.get_bboxes(video=target_video)
or labels.get_rois(video=target_video)
or labels.get_centroids(video=target_video)
)
if not labeled_frames and not has_spatial:
raise ValueError(f"No labeled frames found for video {target_video}")
# Get skeleton info (not required when rendering only spatial
# annotations)
skeleton = labels.skeletons[0] if labels.skeletons else None
if skeleton is None and labeled_frames:
for lf in labeled_frames:
for inst in lf.instances:
skeleton = inst.skeleton
break
if skeleton:
break
if skeleton is not None:
edge_inds = skeleton.edge_inds
node_names = [n.name for n in skeleton.nodes]
else:
# Only raise if frames have instances (which need a skeleton)
has_instances = any(len(lf.instances) > 0 for lf in labeled_frames)
if has_instances:
raise ValueError("No skeleton found in labels")
edge_inds = []
node_names = []
n_tracks = len(labels.tracks)
has_tracks = n_tracks > 0
# Auto-use label_images as overlay when no explicit overlay is
# provided and the file has label images for this video.
if overlay is None and labels.label_images:
video_label_images = labels.get_label_images(video=target_video)
if video_label_images:
overlay = video_label_images
if include_unlabeled is None:
include_unlabeled = True
# Auto-use segmentation masks as overlay when no explicit overlay (and
# no label images) resolved. Masks live on specific frames at arbitrary
# frame indices, so resolve them per-frame via a callable rather than a
# positional list. label_images take precedence (resolved above).
if overlay is None and labels.masks:
video_masks = labels.get_masks(video=target_video)
if video_masks:
_auto_labels = labels
_auto_video = target_video
def overlay(fidx: int) -> list["SegmentationMask"]:
"""Resolve segmentation masks for a single frame index."""
return _auto_labels.get_masks(video=_auto_video, frame_idx=fidx)
if include_unlabeled is None:
include_unlabeled = True
# Check if centroids exist for this video (per-frame access via lf).
if show_centroids and labels.centroids:
_has_video_centroids = bool(labels.get_centroids(video=target_video))
if _has_video_centroids and include_unlabeled is None:
include_unlabeled = True
# Resolve None to default after auto-overlay logic
if include_unlabeled is None:
include_unlabeled = False
elif isinstance(source, list) and all(isinstance(x, LabeledFrame) for x in source):
labeled_frames = source
if not labeled_frames:
raise ValueError("Empty labeled frames list")
target_video = labeled_frames[0].video
skeleton = None
for lf in labeled_frames:
for inst in lf.instances:
skeleton = inst.skeleton
break
if skeleton:
break
if skeleton is None:
raise ValueError("No skeleton found in labeled frames")
edge_inds = skeleton.edge_inds
node_names = [n.name for n in skeleton.nodes]
n_tracks = 0
has_tracks = False
labels = None
else:
raise TypeError(
f"source must be Labels or list of LabeledFrame, got {type(source)}"
)
# Create frame index mapping
frame_idx_to_lf = {lf.frame_idx: lf for lf in labeled_frames}
# Get video frame count for include_unlabeled mode
n_video_frames = None
if include_unlabeled:
if hasattr(target_video, "shape") and target_video.shape is not None:
n_video_frames = target_video.shape[0]
# Determine frame indices to render
if frame_inds is not None:
render_indices = frame_inds
elif start is not None or end is not None:
labeled_indices = sorted(frame_idx_to_lf.keys())
if include_unlabeled and n_video_frames is not None:
# Render all frames in range, not just labeled ones
start_idx = start if start is not None else 0
end_idx = end if end is not None else n_video_frames
render_indices = list(range(start_idx, end_idx))
else:
# Only render labeled frames in range
start_idx = start if start is not None else min(labeled_indices, default=0)
end_idx = end if end is not None else max(labeled_indices, default=0) + 1
render_indices = [i for i in labeled_indices if start_idx <= i < end_idx]
else:
if include_unlabeled and n_video_frames is not None:
# Render entire video
render_indices = list(range(n_video_frames))
else:
# Only render labeled frames
render_indices = sorted(frame_idx_to_lf.keys())
if not render_indices and isinstance(overlay, list) and overlay:
# Derive frame indices from overlay list (use list indices as frame indices)
render_indices = list(range(len(overlay)))
if not render_indices and _has_video_centroids:
# Derive frame indices from frames that have centroids
render_indices = sorted(lf.frame_idx for lf in labeled_frames if lf.centroids)
if not render_indices:
raise ValueError("No frames to render")
# Determine FPS
if fps is None:
# Try to get from video
if hasattr(target_video, "backend") and target_video.backend is not None:
try:
fps = target_video.backend.fps
except Exception:
fps = 30.0
else:
fps = 30.0
# Determine color scheme
resolved_scheme = determine_color_scheme(
has_tracks=has_tracks,
is_single_image=False,
scheme=color_by,
)
# Resolve crop bounds once (before the loop)
# We need the video shape to resolve normalized coordinates
crop_bounds: tuple[int, int, int, int] | None = None
crop_offset: tuple[float, float] = (0.0, 0.0)
if crop is not None:
if hasattr(target_video, "shape") and target_video.shape is not None:
h, w = target_video.shape[1:3]
else:
# Fallback: try to get from first frame
h, w = 480, 640 # reasonable default
crop_bounds = _resolve_crop(crop, (h, w))
crop_offset = (float(crop_bounds[0]), float(crop_bounds[1]))
# Setup progress
if show_progress:
try:
from tqdm import tqdm
iterator = tqdm(render_indices, desc="Rendering", unit="frame")
except ImportError:
iterator = render_indices
else:
iterator = render_indices
# Setup video writer for streaming output (memory optimization)
# When save_path is provided, write frames directly instead of accumulating
writer = None
if save_path is not None:
from sleap_io.io.video_writing import VideoWriter
save_path_ = Path(save_path)
save_path_.parent.mkdir(parents=True, exist_ok=True)
writer = VideoWriter(
filename=save_path_,
fps=fps,
codec=codec,
crf=crf,
preset=x264_preset,
)
# Build centroid color palette (by track).
_centroid_palette: list[tuple[int, int, int]] = []
if _has_video_centroids and labels is not None:
_centroid_palette = get_palette(palette, max(len(labels.tracks), 1))
# Set up motion trails (drawn behind poses). Trails trace instances, and
# every instance carries a skeleton, so a missing skeleton means there are
# no instances to trail.
_do_trails = show_trails and trail_length > 0 and skeleton is not None
_trail_targets: list[int | None] = []
_trail_palette: list[tuple[int, int, int]] = []
_trail_pts_cache: dict[int, np.ndarray] = {}
if _do_trails:
_trail_targets = _resolve_trail_node(trail_node, skeleton)
_trail_palette = get_palette(
palette, _n_trail_palette_colors(has_tracks, n_tracks, labeled_frames)
)
# A uniform trail_color overrides the per-track palette colors.
_trail_color_resolved = (
resolve_color(trail_color) if trail_color is not None else None
)
# Pre-process overlay: determine type for per-frame dispatch
_overlay_is_3d = (
overlay is not None and isinstance(overlay, np.ndarray) and overlay.ndim == 3
)
_overlay_is_2d = (
overlay is not None and isinstance(overlay, np.ndarray) and overlay.ndim == 2
)
_overlay_is_callable = callable(overlay) if overlay is not None else False
_overlay_is_list = (
overlay is not None and isinstance(overlay, list) and len(overlay) > 0
)
_overlay_is_label_image_list = _overlay_is_list and _is_label_image(overlay[0])
def _get_frame_overlay(fidx: int):
"""Resolve overlay data for a single frame."""
if overlay is None:
return None
if _overlay_is_3d:
if fidx < overlay.shape[0]:
return overlay[fidx]
return None
if _overlay_is_2d:
return overlay
if _overlay_is_callable:
return overlay(fidx)
if _overlay_is_label_image_list:
# Match by list index (overlays ordered by frame sequence)
if fidx < len(overlay):
return overlay[fidx]
return None
if _overlay_is_list:
# Match by list index
if fidx < len(overlay):
return [overlay[fidx]]
return []
return None
def _apply_frame_overlay(image: np.ndarray, fidx: int) -> np.ndarray:
"""Resolve and apply overlay for a single frame.
Returns the (possibly new) image array — the caller must use the
returned value since grayscale-to-RGB conversion creates a new array.
"""
frame_overlay = _get_frame_overlay(fidx)
if frame_overlay is None:
return image
if image.ndim == 2:
image = np.stack([image] * 3, axis=-1)
elif image.ndim == 3 and image.shape[-1] == 1:
image = np.repeat(image, 3, axis=-1)
# Crop overlay to match cropped image region
cropped_overlay = frame_overlay
if crop_bounds is not None:
if isinstance(frame_overlay, np.ndarray):
x1, y1, x2, y2 = crop_bounds
oh, ow = frame_overlay.shape[:2]
cropped_overlay = frame_overlay[
max(0, y1) : min(oh, y2), max(0, x1) : min(ow, x2)
]
elif _is_label_image(frame_overlay):
from sleap_io.model.label_image import PredictedLabelImage
x1, y1, x2, y2 = crop_bounds
oh, ow = frame_overlay.data.shape[:2]
cropped_data = frame_overlay.data[
max(0, y1) : min(oh, y2), max(0, x1) : min(ow, x2)
]
kwargs = dict(
data=cropped_data,
objects=frame_overlay.objects,
)
if isinstance(frame_overlay, PredictedLabelImage):
kwargs["score"] = frame_overlay.score
kwargs["score_map"] = frame_overlay.score_map
cropped_overlay = type(frame_overlay)(**kwargs)
# Color overlay elements (masks/ROIs/bboxes) by track identity when
# color_by resolves to "track", matching poses/centroids/trails (same
# `palette`). Otherwise fall through to positional `overlay_palette`.
# Untracked elements fall back to the first palette color.
overlay_colors = None
if (
resolved_scheme == "track"
and _overlay_palette_by_track
and isinstance(cropped_overlay, list)
and cropped_overlay
and not _is_label_image(cropped_overlay[0])
):
overlay_colors = []
for el in cropped_overlay:
t = getattr(el, "track", None)
tidx = _track_idx_map.get(id(t)) if t is not None else None
overlay_colors.append(
_overlay_palette_by_track[tidx % len(_overlay_palette_by_track)]
if tidx is not None
else _overlay_palette_by_track[0]
)
_apply_overlay(
image,
cropped_overlay,
alpha=overlay_alpha,
palette=overlay_palette,
outline=overlay_outline,
outline_width=overlay_outline_width,
outline_color=overlay_outline_color,
colors=overlay_colors,
)
return image
# Pre-build track index map for O(1) track color lookups.
_track_idx_map: dict[int, int] = {}
if labels is not None and has_tracks:
_track_idx_map = {id(t): i for i, t in enumerate(labels.tracks)}
# Identity catalog for stable identity coloring across frames (mirrors the
# track index map). Only built when coloring by identity.
_identity_catalog: list | None = None
if resolved_scheme == "identity":
_identity_catalog = list(labels.identities) if labels is not None else []
# Category catalog for stable category coloring across frames (mirrors the
# identity index map). Only built when coloring by category.
_category_catalog: list | None = None
if resolved_scheme == "category":
_category_catalog = list(labels.categories) if labels is not None else []
# Track-keyed palette for overlay (mask/ROI/bbox) coloring under
# color_by="track", matching centroids/poses/trails (same `palette`).
_overlay_palette_by_track: list[tuple[int, int, int]] = []
if labels is not None and has_tracks:
_overlay_palette_by_track = get_palette(palette, max(len(labels.tracks), 1))
def _draw_frame_centroids(
image: np.ndarray, fidx: int, crop_off: tuple[float, float] = (0.0, 0.0)
) -> np.ndarray:
"""Draw centroids for a single frame onto the image."""
if not _has_video_centroids:
return image
# Use frame-level centroid access instead of linear scan.
lf = frame_idx_to_lf.get(fidx)
frame_centroids = lf.centroids if lf is not None else []
if not frame_centroids:
return image
from sleap_io.rendering.overlays import draw_centroids
# Ensure RGB.
if image.ndim == 2:
image = np.stack([image] * 3, axis=-1)
elif image.ndim == 3 and image.shape[-1] == 1:
image = np.repeat(image, 3, axis=-1)
# Assign colors by track using pre-built index map.
centroid_colors = []
for c in frame_centroids:
if c.track is not None and id(c.track) in _track_idx_map:
tidx = _track_idx_map[id(c.track)]
centroid_colors.append(_centroid_palette[tidx % len(_centroid_palette)])
else:
centroid_colors.append(
_centroid_palette[0] if _centroid_palette else (0, 255, 0)
)
# centroid_marker_size is NOT pre-scaled: the centroids are drawn here,
# then the whole image is upscaled once by `scale` inside render_frame,
# so the final radius matches pose nodes (marker_size * scale).
draw_centroids(
image,
frame_centroids,
colors=centroid_colors,
marker_size=centroid_marker_size,
offset=crop_off,
)
return image
def _draw_frame_trails(
image: np.ndarray, fidx: int, crop_off: tuple[float, float] = (0.0, 0.0)
) -> np.ndarray:
"""Draw motion trails for a single frame onto the image."""
if not _do_trails:
return image
trails, trail_colors = _compute_trails(
fidx=fidx,
frame_idx_to_lf=frame_idx_to_lf,
trail_length=trail_length,
trail_targets=_trail_targets,
track_idx_map=_track_idx_map,
palette_colors=_trail_palette,
has_tracks=has_tracks,
pts_cache=_trail_pts_cache,
)
if not trails:
return image
from sleap_io.rendering.overlays import draw_trails
# A uniform trail_color overrides the per-track palette colors.
if _trail_color_resolved is not None:
color_kwargs: dict = {"color": _trail_color_resolved}
else:
color_kwargs = {"colors": trail_colors}
# trail_width is NOT pre-scaled: the trail is drawn here, then the whole
# image is upscaled once by `scale` inside render_frame, so the final
# width matches pose edges (line_width * scale).
return draw_trails(
image,
trails,
line_width=trail_width,
alpha_fade=trail_alpha_fade,
alpha=trail_alpha,
offset=crop_off,
**color_kwargs,
)
# Only accumulate frames if returning as list (no save_path)
rendered_frames: list[np.ndarray] = []
total_frames = len(render_indices)
try:
for i, fidx in enumerate(iterator):
# Check for cancellation
if progress_callback is not None:
if progress_callback(i, total_frames) is False:
break
lf = frame_idx_to_lf.get(fidx)
# Handle frames without LabeledFrame
if lf is None:
if not include_unlabeled:
continue
# Render just the video frame without poses
if background_color is not None:
# Solid color background - skip video loading entirely
if (
hasattr(target_video, "shape")
and target_video.shape is not None
):
h, w = target_video.shape[1:3]
else:
# No video metadata and no points - use minimum default
h, w = 64, 64
image = _create_blank_frame(h, w, background_color)[:, :, :3]
else:
try:
image = target_video[fidx]
except Exception:
image = None
if image is None:
raise ValueError(
f"Video unavailable at frame {fidx}. "
"Specify a background color to render without video."
)
# Ensure RGB for overlay compositing
if image.ndim == 2:
image = np.stack([image] * 3, axis=-1)
elif image.ndim == 3 and image.shape[2] == 1:
image = np.concatenate([image] * 3, axis=-1)
# Apply cropping if specified
render_image_data = image
if crop_bounds is not None:
render_image_data, _, _ = _apply_crop(image, [], crop_bounds)
# Apply overlay
render_image_data = _apply_frame_overlay(render_image_data, fidx)
# Draw centroids
crop_off = (
(float(crop_bounds[0]), float(crop_bounds[1]))
if crop_bounds is not None
else (0.0, 0.0)
)
render_image_data = _draw_frame_centroids(
render_image_data, fidx, crop_off
)
# Draw motion trails
render_image_data = _draw_frame_trails(
render_image_data, fidx, crop_off
)
# Render frame without poses
rendered = render_frame(
frame=render_image_data,
instances_points=[],
edge_inds=edge_inds,
node_names=node_names,
color_by=resolved_scheme,
palette=palette,
marker_shape=marker_shape,
marker_size=marker_size,
line_width=line_width,
alpha=alpha,
show_nodes=show_nodes,
show_edges=show_edges,
scale=scale,
track_indices=None,
n_tracks=n_tracks,
n_identities=(
len(_identity_catalog) if _identity_catalog is not None else 0
),
n_categories=(
len(_category_catalog) if _category_catalog is not None else 0
),
pre_render_callback=pre_render_callback,
post_render_callback=post_render_callback,
per_instance_callback=None,
frame_idx=fidx,
instance_metadata=[],
crop_offset=crop_offset,
)
# Stream to file or accumulate for return
if writer is not None:
writer(rendered)
else:
rendered_frames.append(rendered)
continue
instances = list(lf.instances)
instances_points = [inst.numpy() for inst in instances]
# Get track indices using pre-built map
track_indices = None
if labels is not None and has_tracks:
track_indices = []
for inst in instances:
tidx = _track_idx_map.get(id(inst.track)) if inst.track else None
track_indices.append(tidx if tidx is not None else 0)
# Get identity indices when coloring by identity
identity_indices = None
n_identities = 0
if resolved_scheme == "identity":
identity_indices, n_identities = _compute_identity_coloring(
instances, _identity_catalog
)
# Get category indices when coloring by category
category_indices = None
n_categories = 0
if resolved_scheme == "category":
category_indices, n_categories = _compute_category_coloring(
instances, _category_catalog
)
# Build instance metadata
instance_metadata = []
for inst in instances:
meta = {}
if hasattr(inst, "track") and inst.track is not None:
meta["track_name"] = inst.track.name
if hasattr(inst, "score"):
meta["confidence"] = inst.score
instance_metadata.append(meta)
# Get image
if background_color is not None:
# Solid color background - skip video loading entirely
if hasattr(target_video, "shape") and target_video.shape is not None:
h, w = target_video.shape[1:3]
else:
# Estimate from points
h, w = _estimate_frame_size(instances_points)
image = _create_blank_frame(h, w, background_color)[:, :, :3]
else:
try:
image = lf.image
except Exception:
image = None
if image is None:
raise ValueError(
f"Video unavailable at frame {fidx}. "
"Specify a background color to render without video."
)
# Apply cropping if specified
render_image_data = image
render_points = instances_points
if crop_bounds is not None:
render_image_data, render_points, _ = _apply_crop(
image, instances_points, crop_bounds
)
# Apply overlay
render_image_data = _apply_frame_overlay(render_image_data, fidx)
# Draw centroids
crop_off = (
(float(crop_bounds[0]), float(crop_bounds[1]))
if crop_bounds is not None
else (0.0, 0.0)
)
render_image_data = _draw_frame_centroids(render_image_data, fidx, crop_off)
# Draw motion trails
render_image_data = _draw_frame_trails(render_image_data, fidx, crop_off)
# Render frame
rendered = render_frame(
frame=render_image_data,
instances_points=render_points,
edge_inds=edge_inds,
node_names=node_names,
color_by=resolved_scheme,
palette=palette,
marker_shape=marker_shape,
marker_size=marker_size,
line_width=line_width,
alpha=alpha,
show_nodes=show_nodes,
show_edges=show_edges,
scale=scale,
track_indices=track_indices,
n_tracks=n_tracks,
identity_indices=identity_indices,
n_identities=n_identities,
category_indices=category_indices,
n_categories=n_categories,
pre_render_callback=pre_render_callback,
post_render_callback=post_render_callback,
per_instance_callback=per_instance_callback,
frame_idx=fidx,
instance_metadata=instance_metadata,
crop_offset=crop_offset,
)
# Stream to file or accumulate for return
if writer is not None:
writer(rendered)
else:
rendered_frames.append(rendered)
finally:
# Ensure writer is closed even if an exception occurs
if writer is not None:
writer.close()
# Return Video object or frame list
if save_path is not None:
return VideoModel.from_filename(str(save_path_))
return rendered_frames
sleap_io.render_image(source=None, save_path=None, *, lf_ind=None, video=None, frame_idx=None, image=None, overlay=None, overlay_alpha=0.3, overlay_palette='distinct', overlay_outline=False, overlay_outline_width=1, overlay_outline_color=None, crop=None, color_by='auto', palette='standard', marker_shape='circle', marker_size=4.0, line_width=2.0, alpha=1.0, show_nodes=True, show_edges=True, show_centroids=True, centroid_marker_size=5.0, scale=1.0, show_trails=False, trail_length=10, trail_node='centroid', trail_width=2.0, trail_alpha_fade=True, trail_alpha=1.0, trail_color=None, background='video', pre_render_callback=None, post_render_callback=None, per_instance_callback=None)
¶
Render single frame with pose and/or segmentation overlays.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Labels | LabeledFrame | list[Instance | PredictedInstance] | None
|
LabeledFrame, Labels (with frame specifier), list of instances,
or |
None
|
save_path
|
str | Path | None
|
Output image path (PNG/JPEG). If None, only returns array. |
None
|
lf_ind
|
int | None
|
LabeledFrame index within Labels.labeled_frames (when source is Labels). |
None
|
video
|
Video | int | None
|
Video object or video index (used with frame_idx when source is Labels). |
None
|
frame_idx
|
int | None
|
Video frame index (0-based, used with video when source is Labels). |
None
|
image
|
ndarray | None
|
Override image array (H, W) or (H, W, C) uint8. Fetched from LabeledFrame if not provided. |
None
|
overlay
|
ndarray | LabelImage | SegmentationMask | ROI | BoundingBox | list[SegmentationMask] | list[ROI] | list[BoundingBox] | None
|
Annotation data to render on the image before poses. Accepts:
Applied before pose rendering so poses draw on top. |
None
|
overlay_alpha
|
float
|
Opacity for the segmentation overlay (0.0 to 1.0). |
0.3
|
overlay_palette
|
Literal | str
|
Color palette for segmentation overlay. |
'distinct'
|
overlay_outline
|
bool
|
Whether to draw outlines around segmented regions. |
False
|
overlay_outline_width
|
int
|
Outline width in pixels. |
1
|
overlay_outline_color
|
tuple[int, int, int] | None
|
RGB outline color, or |
None
|
crop
|
CropSpec
|
Crop specification. Bounds are (x1, y1, x2, y2) where (x1, y1) is the top-left corner and (x2, y2) is the bottom-right (exclusive). Origin (0, 0) is at the image top-left. Can be:
|
None
|
color_by
|
Literal
|
Color scheme - 'track', 'instance', 'node', 'identity', 'category', or 'auto'. |
'auto'
|
palette
|
Literal | str
|
Color palette name. |
'standard'
|
marker_shape
|
Literal
|
Node marker shape. |
'circle'
|
marker_size
|
float
|
Node marker radius in pixels. |
4.0
|
line_width
|
float
|
Edge line width in pixels. |
2.0
|
alpha
|
float
|
Global transparency (0.0-1.0). |
1.0
|
show_nodes
|
bool
|
Whether to draw node markers. |
True
|
show_edges
|
bool
|
Whether to draw skeleton edges. |
True
|
show_centroids
|
bool
|
Whether to draw centroid markers from
|
True
|
centroid_marker_size
|
float
|
Radius of centroid markers in pixels. |
5.0
|
scale
|
float
|
Output scale factor. Applied after cropping. |
1.0
|
show_trails
|
bool
|
Whether to draw motion trails tracing node or centroid
positions over past frames. Only takes effect when |
False
|
trail_length
|
int
|
Number of past frames behind the current frame to include in each trail. |
10
|
trail_node
|
str | list[str]
|
Which point to trail. One of |
'centroid'
|
trail_width
|
float
|
Trail line width in pixels. |
2.0
|
trail_alpha_fade
|
bool
|
If |
True
|
trail_alpha
|
float
|
Global opacity multiplier for trails (0.0 to 1.0). Combines
with |
1.0
|
trail_color
|
ColorSpec | None
|
Uniform color for all trails. If |
None
|
background
|
Literal['video'] | ColorSpec
|
Background control. Can be:
- |
'video'
|
pre_render_callback
|
Callable[[RenderContext], None] | None
|
Called before poses are drawn. |
None
|
post_render_callback
|
Callable[[RenderContext], None] | None
|
Called after poses are drawn. |
None
|
per_instance_callback
|
Callable[[InstanceContext], None] | None
|
Called after each instance is drawn. |
None
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Rendered numpy array (H, W, 3) uint8. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If background="video" and video unavailable. |
Examples:
Render a single labeled frame:
>>> import sleap_io as sio
>>> labels = sio.load_slp("predictions.slp")
>>> lf = labels.labeled_frames[0]
>>> img = sio.render_image(lf)
Render with solid color background (no video required):
>>> img = sio.render_image(lf, background="black")
>>> img = sio.render_image(lf, background=(40, 40, 40))
>>> img = sio.render_image(lf, background="#404040")
>>> img = sio.render_image(lf, background=0.25)
Crop to a region (pixel coordinates):
Normalized crop (center 50% of frame):
Render and save to file:
>>> sio.render_image(labels, lf_ind=0, save_path="frame.png")
>>> sio.render_image(labels, video=0, frame_idx=42, save_path="frame.png")
Overlay a segmentation mask on a raw image (no poses):
Overlay segmentation on a labeled frame (poses draw on top):
Source code in sleap_io/rendering/core.py
def render_image(
source: "Labels | LabeledFrame | list[Instance | PredictedInstance] | None" = None,
save_path: str | Path | None = None,
*,
# Frame specification (for Labels input)
lf_ind: int | None = None,
video: "Video | int | None" = None,
frame_idx: int | None = None,
# Image override
image: np.ndarray | None = None,
# Annotation overlay
overlay: (
"np.ndarray | LabelImage"
" | SegmentationMask | ROI | BoundingBox"
" | list[SegmentationMask] | list[ROI] | list[BoundingBox]"
" | None"
) = None,
overlay_alpha: float = 0.3,
overlay_palette: PaletteName | str = "distinct",
overlay_outline: bool = False,
overlay_outline_width: int = 1,
overlay_outline_color: tuple[int, int, int] | None = None,
# Cropping
crop: CropSpec = None,
# Appearance
color_by: ColorScheme = "auto",
palette: PaletteName | str = "standard",
marker_shape: MarkerShape = "circle",
marker_size: float = 4.0,
line_width: float = 2.0,
alpha: float = 1.0,
show_nodes: bool = True,
show_edges: bool = True,
show_centroids: bool = True,
centroid_marker_size: float = 5.0,
scale: float = 1.0,
# Motion trails
show_trails: bool = False,
trail_length: int = 10,
trail_node: str | list[str] = "centroid",
trail_width: float = 2.0,
trail_alpha_fade: bool = True,
trail_alpha: float = 1.0,
trail_color: ColorSpec | None = None,
# Background control
background: Literal["video"] | ColorSpec = "video",
# Callbacks
pre_render_callback: Callable[[RenderContext], None] | None = None,
post_render_callback: Callable[[RenderContext], None] | None = None,
per_instance_callback: Callable[[InstanceContext], None] | None = None,
) -> np.ndarray:
"""Render single frame with pose and/or segmentation overlays.
Args:
source: LabeledFrame, Labels (with frame specifier), list of instances,
or ``None``. When ``None``, ``image`` must be provided and only
segmentation overlays are rendered (no poses).
save_path: Output image path (PNG/JPEG). If None, only returns array.
lf_ind: LabeledFrame index within Labels.labeled_frames (when source is Labels).
video: Video object or video index (used with frame_idx when source is Labels).
frame_idx: Video frame index (0-based, used with video when source is Labels).
image: Override image array (H, W) or (H, W, C) uint8. Fetched from
LabeledFrame if not provided.
overlay: Annotation data to render on the image before poses. Accepts:
- ``np.ndarray``: Integer label image ``(H, W)`` where 0 is
background and positive values are object IDs.
- ``SegmentationMask``, ``ROI``, or ``BoundingBox``: A single
annotation object (treated like a one-element list).
- ``list[SegmentationMask]``: Binary segmentation masks.
- ``list[ROI]``: Vector geometries (polygons, points, etc.).
- ``list[BoundingBox]``: Bounding boxes.
Applied before pose rendering so poses draw on top.
overlay_alpha: Opacity for the segmentation overlay (0.0 to 1.0).
overlay_palette: Color palette for segmentation overlay.
overlay_outline: Whether to draw outlines around segmented regions.
overlay_outline_width: Outline width in pixels.
overlay_outline_color: RGB outline color, or ``None`` for auto-darkened.
crop: Crop specification. Bounds are (x1, y1, x2, y2) where (x1, y1) is
the top-left corner and (x2, y2) is the bottom-right (exclusive).
Origin (0, 0) is at the image top-left. Can be:
- **Pixel coordinates** (int tuple): ``(100, 100, 300, 300)`` crops
from pixel (100, 100) to (300, 300).
- **Normalized coordinates** (float tuple in [0.0, 1.0]):
``(0.25, 0.25, 0.75, 0.75)`` crops the center 50% of the frame.
Detection is type-based: all values must be ``float`` and in range.
- ``None``: No cropping (default).
color_by: Color scheme - 'track', 'instance', 'node', 'identity',
'category', or 'auto'.
palette: Color palette name.
marker_shape: Node marker shape.
marker_size: Node marker radius in pixels.
line_width: Edge line width in pixels.
alpha: Global transparency (0.0-1.0).
show_nodes: Whether to draw node markers.
show_edges: Whether to draw skeleton edges.
show_centroids: Whether to draw centroid markers from
``Labels.centroids``. Centroids are colored by track.
centroid_marker_size: Radius of centroid markers in pixels.
scale: Output scale factor. Applied after cropping.
show_trails: Whether to draw motion trails tracing node or centroid
positions over past frames. Only takes effect when ``source`` is a
``Labels`` object (trails need temporal context); ignored otherwise.
trail_length: Number of past frames behind the current frame to include
in each trail.
trail_node: Which point to trail. One of ``"centroid"`` (default), a
node name, or a list of node names (one trail per node).
trail_width: Trail line width in pixels.
trail_alpha_fade: If ``True``, fade trails from faint (oldest) to opaque
(newest).
trail_alpha: Global opacity multiplier for trails (0.0 to 1.0). Combines
with ``trail_alpha_fade``.
trail_color: Uniform color for all trails. If ``None`` (default), trails
are colored to match the poses (by track or instance). Accepts any
color spec (RGB tuple, named color, hex, or palette index).
background: Background control. Can be:
- ``"video"``: Load video frame (default). Raises error if unavailable.
- Any color spec: Use solid color background, skip video loading entirely.
Supports RGB tuples ``(255, 128, 0)``, float tuples ``(1.0, 0.5, 0.0)``,
grayscale ``128`` or ``0.5``, named colors ``"black"``, hex ``"#ff8000"``,
or palette index ``"tableau10[2]"``.
pre_render_callback: Called before poses are drawn.
post_render_callback: Called after poses are drawn.
per_instance_callback: Called after each instance is drawn.
Returns:
Rendered numpy array (H, W, 3) uint8.
Raises:
ValueError: If background="video" and video unavailable.
Examples:
Render a single labeled frame:
>>> import sleap_io as sio
>>> labels = sio.load_slp("predictions.slp")
>>> lf = labels.labeled_frames[0]
>>> img = sio.render_image(lf)
Render with solid color background (no video required):
>>> img = sio.render_image(lf, background="black")
>>> img = sio.render_image(lf, background=(40, 40, 40))
>>> img = sio.render_image(lf, background="#404040")
>>> img = sio.render_image(lf, background=0.25)
Crop to a region (pixel coordinates):
>>> img = sio.render_image(lf, crop=(100, 100, 300, 300))
Normalized crop (center 50% of frame):
>>> img = sio.render_image(lf, crop=(0.25, 0.25, 0.75, 0.75))
Render and save to file:
>>> sio.render_image(labels, lf_ind=0, save_path="frame.png")
>>> sio.render_image(labels, video=0, frame_idx=42, save_path="frame.png")
Overlay a segmentation mask on a raw image (no poses):
>>> img = sio.render_image(image=frame, overlay=label_mask)
Overlay segmentation on a labeled frame (poses draw on top):
>>> img = sio.render_image(lf, overlay=label_mask, overlay_alpha=0.4)
"""
import skia # noqa: F401
from sleap_io.model.instance import Instance, PredictedInstance
from sleap_io.model.labeled_frame import LabeledFrame
from sleap_io.model.labels import Labels
# Handle background parameter
use_video = background == "video"
background_color: tuple[int, int, int] | None = None
if not use_video:
background_color = resolve_color(background)
# Resolve source to LabeledFrame or instances
if isinstance(source, Labels):
lf = None
has_centroids = show_centroids and bool(source.centroids)
if video is not None and frame_idx is not None:
# Render by video + frame_idx
target_video = source.videos[video] if isinstance(video, int) else video
lf_list = source.find(target_video, frame_idx)
if lf_list:
lf = lf_list[0]
elif not has_centroids:
raise ValueError(
f"No labeled frame found for video {target_video} "
f"at frame {frame_idx}"
)
elif lf_ind is not None:
# Render by labeled frame index
lf = source.labeled_frames[lf_ind]
elif source.labeled_frames:
# Default to first labeled frame
lf = source.labeled_frames[0]
elif not has_centroids:
raise ValueError("No labeled frames to render")
if lf is not None:
instances = list(lf.instances)
if instances:
skeleton = instances[0].skeleton
elif source.skeletons:
# No instances but skeletons exist: use the first skeleton for
# pose-rendering metadata (e.g. trail node resolution).
skeleton = source.skeletons[0]
else:
# No instances and no skeletons — segmentation/overlay-only
# frame (e.g. bottom-up mask tracking). Fall through with empty
# pose state so the background and any overlay (masks, label
# images, ROIs, bboxes) still render instead of crashing.
# Mirrors the LabeledFrame branch below.
skeleton = None
edge_inds = skeleton.edge_inds if skeleton is not None else []
node_names = (
[n.name for n in skeleton.nodes] if skeleton is not None else []
)
fidx_for_callback = lf.frame_idx
else:
# Centroid-only / spatial-only mode: no labeled frames.
instances = []
skeleton = None
edge_inds = []
node_names = []
fidx_for_callback = frame_idx if frame_idx is not None else 0
# Get track info using O(1) lookup map
n_tracks = len(source.tracks)
has_tracks = n_tracks > 0
img_track_idx_map = {id(t): i for i, t in enumerate(source.tracks)}
track_indices = []
for inst in instances:
tidx = img_track_idx_map.get(id(inst.track)) if inst.track else None
track_indices.append(tidx if tidx is not None else 0)
# Convert instances to point arrays (needed for both image size and rendering)
instances_points = [inst.numpy() for inst in instances]
# Get image if not provided
if image is None:
video_obj = (
lf.video
if lf is not None
else (source.videos[0] if source.videos else None)
)
if background_color is not None:
# Solid color background - skip video loading entirely
if (
video_obj is not None
and hasattr(video_obj, "shape")
and video_obj.shape is not None
):
h, w = video_obj.shape[1:3]
else:
# Estimate from points or default
if instances_points:
h, w = _estimate_frame_size(instances_points)
else:
h, w = 512, 512
image = _create_blank_frame(h, w, background_color)[:, :, :3]
else:
# Load video frame
try:
if lf is not None:
image = lf.image
elif video_obj is not None and frame_idx is not None:
image = video_obj[frame_idx]
else:
image = None
if image is None:
raise ValueError("No image available")
except Exception:
raise ValueError(
"Video unavailable. Specify a background color to render "
"without video, e.g., background='black' or "
"background=(40, 40, 40)."
)
elif isinstance(source, LabeledFrame):
lf = source
instances = list(lf.instances)
if instances:
skeleton = instances[0].skeleton
edge_inds = skeleton.edge_inds
node_names = [n.name for n in skeleton.nodes]
else:
# No instances — segmentation/overlay-only mode. Fall through with
# empty pose-rendering state so the video frame and any overlay
# (masks, label images, ROIs, bboxes) still render. Mirrors the
# centroid-only path in the `Labels` branch above.
edge_inds = []
node_names = []
fidx_for_callback = lf.frame_idx
track_indices = None
n_tracks = 0
has_tracks = False
# Convert instances to point arrays (needed for both image size and rendering)
instances_points = [inst.numpy() for inst in instances]
# Get image if not provided
if image is None:
if background_color is not None:
# Solid color background - skip video loading entirely
video_obj = lf.video
if hasattr(video_obj, "shape") and video_obj.shape is not None:
h, w = video_obj.shape[1:3]
else:
# Estimate from points
h, w = _estimate_frame_size(instances_points)
image = _create_blank_frame(h, w, background_color)[:, :, :3]
else:
# Load video frame
try:
image = lf.image
if image is None:
raise ValueError("No image available")
except Exception:
raise ValueError(
"Video unavailable. Specify a background color to render "
"without video, e.g., background='black' or "
"background=(40, 40, 40)."
)
elif isinstance(source, list) and all(
isinstance(x, (Instance, PredictedInstance)) for x in source
):
instances = source
if not instances:
raise ValueError("Empty instances list")
skeleton = instances[0].skeleton
edge_inds = skeleton.edge_inds
node_names = [n.name for n in skeleton.nodes]
fidx_for_callback = 0
track_indices = None
n_tracks = 0
has_tracks = False
# Convert instances to point arrays
instances_points = [inst.numpy() for inst in instances]
if image is None:
raise ValueError(
"image parameter required when source is list of instances"
)
elif source is None:
# No poses — overlay-only or image-only mode
if image is None:
raise ValueError("image parameter required when source is None")
instances = []
instances_points = []
edge_inds = []
node_names = []
fidx_for_callback = 0
track_indices = None
n_tracks = 0
has_tracks = False
else:
raise TypeError(
f"source must be Labels, LabeledFrame, list of instances, "
f"or None, got {type(source)}"
)
# Auto-use the frame's segmentation masks as overlay when no explicit
# overlay is given and the frame carries masks. Mirrors render_video's
# auto-overlay behavior so a segmentation-only frame still draws its masks.
# Only masks are auto-resolved here (not label_images): label_image overlays
# are a Labels/render_video-level concept and _apply_overlay does not accept a
# list[LabelImage] in the single-frame path. An explicit overlay always wins.
if (
overlay is None
and isinstance(source, (Labels, LabeledFrame))
and lf is not None
and lf.masks
):
overlay = list(lf.masks)
# Determine color scheme up front so track-colored overlays (masks/ROIs/
# bboxes) can match the pose/centroid/trail track colors. Consumed below by
# both the overlay block and the pose render_frame call.
resolved_scheme = determine_color_scheme(
has_tracks=has_tracks,
is_single_image=True,
scheme=color_by,
)
# Apply cropping if specified
render_image_data = image
render_points = instances_points
crop_offset: tuple[float, float] = (0.0, 0.0)
if crop is not None:
h, w = image.shape[:2]
# Resolve normalized or pixel coordinates
crop_bounds = _resolve_crop(crop, (h, w))
crop_offset = (float(crop_bounds[0]), float(crop_bounds[1]))
render_image_data, render_points, _ = _apply_crop(
image, instances_points, crop_bounds
)
# Apply annotation overlay before pose rendering
if overlay is not None:
# Ensure image is RGB for color blending
if render_image_data.ndim == 2:
render_image_data = np.stack([render_image_data] * 3, axis=-1)
elif render_image_data.ndim == 3 and render_image_data.shape[2] == 1:
render_image_data = np.repeat(render_image_data, 3, axis=2)
# Crop overlay to match cropped image region
render_overlay = overlay
if crop is not None and isinstance(overlay, np.ndarray):
x1, y1, x2, y2 = crop_bounds
oh, ow = overlay.shape[:2]
render_overlay = overlay[max(0, y1) : min(oh, y2), max(0, x1) : min(ow, x2)]
# Color overlay elements (masks/ROIs/bboxes) by track identity when
# color_by resolves to "track", matching poses/centroids/trails (same
# `palette`). Otherwise fall through to positional `overlay_palette`
# coloring. Gated on a Labels source with tracks (only that branch builds
# the track index map; `has_tracks` mirrors render_video so track-less
# labels stay positional). Untracked elements fall back to the first
# color.
overlay_colors = None
if (
resolved_scheme == "track"
and isinstance(source, Labels)
and has_tracks
and isinstance(render_overlay, list)
and render_overlay
and not _is_label_image(render_overlay[0])
):
ov_pal = get_palette(palette, max(len(source.tracks), 1))
overlay_colors = []
for el in render_overlay:
t = getattr(el, "track", None)
tidx = img_track_idx_map.get(id(t)) if t is not None else None
overlay_colors.append(
ov_pal[tidx % len(ov_pal)] if tidx is not None else ov_pal[0]
)
_apply_overlay(
render_image_data,
render_overlay,
alpha=overlay_alpha,
palette=overlay_palette,
outline=overlay_outline,
outline_width=overlay_outline_width,
outline_color=overlay_outline_color,
colors=overlay_colors,
)
# Draw motion trails behind the poses and centroids. Trails need temporal
# context, so they are only drawn when the source is a Labels object. They
# are drawn even when the current frame has no instances, since past frames
# may still contribute (matching render_video).
if (
show_trails
and isinstance(source, Labels)
and trail_length > 0
and skeleton is not None
):
from sleap_io.rendering.overlays import draw_trails as _draw_trails
trail_targets = _resolve_trail_node(trail_node, skeleton)
frame_idx_to_lf = {lframe.frame_idx: lframe for lframe in source.find(lf.video)}
n_trail_colors = _n_trail_palette_colors(
has_tracks, n_tracks, frame_idx_to_lf.values()
)
trail_palette = get_palette(palette, n_trail_colors)
trails, trail_colors = _compute_trails(
fidx=fidx_for_callback,
frame_idx_to_lf=frame_idx_to_lf,
trail_length=trail_length,
trail_targets=trail_targets,
track_idx_map=img_track_idx_map,
palette_colors=trail_palette,
has_tracks=has_tracks,
)
if trails:
# A uniform trail_color overrides the per-track palette colors.
trail_draw_kwargs: dict = {}
if trail_color is not None:
trail_draw_kwargs["color"] = resolve_color(trail_color)
else:
trail_draw_kwargs["colors"] = trail_colors
# trail_width is NOT pre-scaled: the trail is drawn here, then the
# whole image is upscaled once by `scale` inside render_frame, so
# the final width matches pose edges (line_width * scale).
render_image_data = _draw_trails(
render_image_data,
trails,
line_width=trail_width,
alpha_fade=trail_alpha_fade,
alpha=trail_alpha,
offset=crop_offset,
**trail_draw_kwargs,
)
# Draw centroids on the image.
if show_centroids and isinstance(source, Labels) and source.centroids:
from sleap_io.rendering.overlays import draw_centroids as _draw_centroids
render_fidx = fidx_for_callback
if lf is not None:
# Scope to the rendered frame's own video. A centroid on a
# *different* video that happens to share this frame index must not
# bleed in, so read this frame's centroids directly (mirroring
# render_video's per-frame `lf.centroids`).
frame_centroids = list(lf.centroids)
else:
# No labeled frame resolved — only reachable via an explicit
# video+frame_idx that matched no frame, so `video` is a concrete
# spec. Scope centroids to it so a *different* video's centroid that
# shares this frame index can't bleed in (mirrors render_video's
# get_centroids(video=target_video)).
target_video = source.videos[video] if isinstance(video, int) else video
frame_centroids = source.get_centroids(
video=target_video, frame_idx=render_fidx
)
if frame_centroids:
if render_image_data.ndim == 2:
render_image_data = np.stack([render_image_data] * 3, axis=-1)
centroid_pal = get_palette(palette, max(len(source.tracks), 1))
c_colors = []
for c in frame_centroids:
tidx = img_track_idx_map.get(id(c.track)) if c.track else None
if tidx is not None:
c_colors.append(centroid_pal[tidx % len(centroid_pal)])
else:
c_colors.append(centroid_pal[0] if centroid_pal else (0, 255, 0))
# centroid_marker_size is NOT pre-scaled: the centroids are drawn
# here, then the whole image is upscaled once by `scale` inside
# render_frame, so the final radius matches pose nodes
# (marker_size * scale).
render_image_data = _draw_centroids(
render_image_data,
frame_centroids,
colors=c_colors,
marker_size=centroid_marker_size,
offset=crop_offset,
)
# Short-circuit: overlay-only mode (no poses to render)
if source is None:
# Scale if needed
if scale != 1.0:
render_image_data = _scale_frame(
_prepare_frame_rgba(render_image_data), scale
)[:, :, :3]
if save_path is not None:
_save_image(render_image_data, save_path)
return render_image_data
# Build instance metadata for callbacks
instance_metadata = []
for inst in instances:
meta = {}
if hasattr(inst, "track") and inst.track is not None:
meta["track_name"] = inst.track.name
if hasattr(inst, "score"):
meta["confidence"] = inst.score
instance_metadata.append(meta)
# Compute per-instance identity indices when the resolved scheme is
# "identity", mirroring the track-index plumbing.
identity_indices = None
n_identities = 0
if resolved_scheme == "identity":
catalog = source.identities if isinstance(source, Labels) else None
identity_indices, n_identities = _compute_identity_coloring(instances, catalog)
# Compute per-instance category indices when the resolved scheme is
# "category", mirroring the identity plumbing.
category_indices = None
n_categories = 0
if resolved_scheme == "category":
catalog = source.categories if isinstance(source, Labels) else None
category_indices, n_categories = _compute_category_coloring(instances, catalog)
# Render
rendered = render_frame(
frame=render_image_data,
instances_points=render_points,
edge_inds=edge_inds,
node_names=node_names,
color_by=resolved_scheme,
palette=palette,
marker_shape=marker_shape,
marker_size=marker_size,
line_width=line_width,
alpha=alpha,
show_nodes=show_nodes,
show_edges=show_edges,
scale=scale,
track_indices=track_indices,
n_tracks=n_tracks,
identity_indices=identity_indices,
n_identities=n_identities,
category_indices=category_indices,
n_categories=n_categories,
pre_render_callback=pre_render_callback,
post_render_callback=post_render_callback,
per_instance_callback=per_instance_callback,
frame_idx=fidx_for_callback,
instance_metadata=instance_metadata,
crop_offset=crop_offset,
)
# Save if save_path provided
if save_path is not None:
_save_image(rendered, save_path)
return rendered
sleap_io.rendering.get_palette(name, n_colors)
¶
Get n colors from a named palette as RGB tuples.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
Literal | str
|
Palette name. Built-in options: 'standard', 'distinct', 'rainbow', 'warm', 'cool', 'pastel', 'seaborn', 'tableau10', 'viridis'. With colorcet installed: 'glasbey', 'glasbey_hv', 'glasbey_cool', 'glasbey_warm'. |
required |
n_colors
|
int
|
Number of colors needed. |
required |
Returns:
| Type | Description |
|---|---|
list[tuple[int, int, int]]
|
List of (R, G, B) tuples. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If palette name is not recognized. |
Source code in sleap_io/rendering/colors.py
def get_palette(name: PaletteName | str, n_colors: int) -> list[tuple[int, int, int]]:
"""Get n colors from a named palette as RGB tuples.
Args:
name: Palette name. Built-in options: 'standard', 'distinct', 'rainbow',
'warm', 'cool', 'pastel', 'seaborn', 'tableau10', 'viridis'.
With colorcet installed: 'glasbey', 'glasbey_hv', 'glasbey_cool',
'glasbey_warm'.
n_colors: Number of colors needed.
Returns:
List of (R, G, B) tuples.
Raises:
ValueError: If palette name is not recognized.
"""
# Try built-in palettes first
if name in PALETTES:
palette = PALETTES[name]
return _extend_palette(palette, n_colors)
# Try colorcet palettes
import colorcet as cc
if name in cc.palette:
hex_colors = cc.palette[name]
rgb_colors = [_hex_to_rgb(c) for c in hex_colors]
return _extend_palette(rgb_colors, n_colors)
# Unknown palette - raise error with available options
raise ValueError(
f"Unknown palette: {name}. "
f"Available: {list(PALETTES.keys())} (built-in), "
"or any colorcet palette (e.g., glasbey, glasbey_hv, fire, rainbow4)"
)
sleap_io.rendering.resolve_color(color)
¶
Resolve a flexible color specification to an RGB tuple.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
color
|
ColorSpec
|
Color specification in various formats:
- RGB int tuple: |
required |
Returns:
| Type | Description |
|---|---|
tuple[int, int, int]
|
RGB tuple of integers in 0-255 range. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If color specification is invalid. |
Examples:
Source code in sleap_io/rendering/colors.py
def resolve_color(color: ColorSpec) -> tuple[int, int, int]:
"""Resolve a flexible color specification to an RGB tuple.
Args:
color: Color specification in various formats:
- RGB int tuple: ``(255, 128, 0)``
- RGB float tuple: ``(1.0, 0.5, 0.0)`` - values in 0.0-1.0 range
- Grayscale int: ``128`` → ``(128, 128, 128)``
- Grayscale float: ``0.5`` → ``(127, 127, 127)``
- Named color: ``"black"``, ``"white"``, ``"red"``, etc.
- Hex color: ``"#ff8000"`` or ``"#f80"``
- Palette index: ``"tableau10[2]"``, ``"glasbey[5]"``
Returns:
RGB tuple of integers in 0-255 range.
Raises:
ValueError: If color specification is invalid.
Examples:
>>> resolve_color((255, 128, 0))
(255, 128, 0)
>>> resolve_color((1.0, 0.5, 0.0))
(255, 127, 0)
>>> resolve_color("red")
(255, 0, 0)
>>> resolve_color("#ff8000")
(255, 128, 0)
>>> resolve_color("#f80")
(255, 136, 0)
>>> resolve_color("tableau10[2]")
(44, 160, 44)
>>> resolve_color(128)
(128, 128, 128)
>>> resolve_color(0.5)
(127, 127, 127)
"""
# Grayscale int
if isinstance(color, int):
value = max(0, min(255, color))
return (value, value, value)
# Grayscale float (0.0-1.0 range)
if isinstance(color, float):
value = int(max(0.0, min(1.0, color)) * 255)
return (value, value, value)
# Tuple (RGB)
if isinstance(color, tuple):
if len(color) != 3:
raise ValueError(f"RGB tuple must have 3 elements, got {len(color)}")
r, g, b = color
# Detect float vs int by Python type
if isinstance(r, float) or isinstance(g, float) or isinstance(b, float):
# Float tuple: 0.0-1.0 range
r_int = int(max(0.0, min(1.0, float(r))) * 255)
g_int = int(max(0.0, min(1.0, float(g))) * 255)
b_int = int(max(0.0, min(1.0, float(b))) * 255)
return (r_int, g_int, b_int)
else:
# Int tuple: 0-255 range
r_int = max(0, min(255, int(r)))
g_int = max(0, min(255, int(g)))
b_int = max(0, min(255, int(b)))
return (r_int, g_int, b_int)
# String
if isinstance(color, str):
color_lower = color.lower().strip()
# Named color
if color_lower in NAMED_COLORS:
return NAMED_COLORS[color_lower]
# Hex color
if color.startswith("#"):
hex_part = color[1:]
if len(hex_part) == 3 or len(hex_part) == 6:
return _hex_to_rgb(color)
raise ValueError(f"Invalid hex color: {color}")
# Palette index: "palette_name[index]"
palette_match = re.match(r"^(\w+)\[(\d+)\]$", color)
if palette_match:
palette_name = palette_match.group(1)
index = int(palette_match.group(2))
# Get palette colors
try:
palette_colors = get_palette(palette_name, index + 1)
if index < len(palette_colors):
return palette_colors[index]
except ValueError:
pass
raise ValueError(f"Invalid palette index: {color}")
raise ValueError(
f"Unknown color: {color!r}. Valid named colors: {list(NAMED_COLORS.keys())}"
)
raise TypeError(f"Invalid color type: {type(color).__name__}")
sleap_io.rendering.RenderContext
¶
Context passed to pre/post render callbacks.
This context provides access to the Skia canvas and frame-level metadata for drawing custom overlays before or after pose rendering.
Attributes:
| Name | Type | Description |
|---|---|---|
canvas |
Skia canvas for drawing. |
|
frame_idx |
Current frame index. |
|
frame_size |
(width, height) tuple of original frame dimensions. |
|
instances |
List of instances in this frame. |
|
skeleton_edges |
Edge connectivity as list of (src, dst) tuples. |
|
node_names |
List of node name strings. |
|
scale |
Current scale factor for rendering. |
|
offset |
Current offset (x, y) for cropped/zoomed views. |
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class RenderContext. |
__init__ |
Method generated by attrs for class RenderContext. |
__repr__ |
Method generated by attrs for class RenderContext. |
world_to_canvas |
Transform world coordinates to canvas coordinates. |
Source code in sleap_io/rendering/callbacks.py
@define
class RenderContext:
"""Context passed to pre/post render callbacks.
This context provides access to the Skia canvas and frame-level metadata
for drawing custom overlays before or after pose rendering.
Attributes:
canvas: Skia canvas for drawing.
frame_idx: Current frame index.
frame_size: (width, height) tuple of original frame dimensions.
instances: List of instances in this frame.
skeleton_edges: Edge connectivity as list of (src, dst) tuples.
node_names: List of node name strings.
scale: Current scale factor for rendering.
offset: Current offset (x, y) for cropped/zoomed views.
"""
canvas: "skia.Canvas"
frame_idx: int
frame_size: tuple[int, int]
instances: list
skeleton_edges: list[tuple[int, int]]
node_names: list[str]
scale: float = 1.0
offset: tuple[float, float] = (0.0, 0.0)
def world_to_canvas(self, x: float, y: float) -> tuple[float, float]:
"""Transform world coordinates to canvas coordinates.
Args:
x: X coordinate in world/frame space.
y: Y coordinate in world/frame space.
Returns:
(x, y) coordinates in canvas space.
"""
return (
(x - self.offset[0]) * self.scale,
(y - self.offset[1]) * self.scale,
)
__annotations__ = {'canvas': "'skia.Canvas'", 'frame_idx': 'int', 'frame_size': 'tuple[int, int]', 'instances': 'list', 'skeleton_edges': 'list[tuple[int, int]]', 'node_names': 'list[str]', 'scale': 'float', 'offset': 'tuple[float, float]'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = False
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Context passed to pre/post render callbacks.\n\nThis context provides access to the Skia canvas and frame-level metadata\nfor drawing custom overlays before or after pose rendering.\n\nAttributes:\n canvas: Skia canvas for drawing.\n frame_idx: Current frame index.\n frame_size: (width, height) tuple of original frame dimensions.\n instances: List of instances in this frame.\n skeleton_edges: Edge connectivity as list of (src, dst) tuples.\n node_names: List of node name strings.\n scale: Current scale factor for rendering.\n offset: Current offset (x, y) for cropped/zoomed views.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 18
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('canvas', 'frame_idx', 'frame_size', 'instances', 'skeleton_edges', 'node_names', 'scale', 'offset')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.rendering.callbacks'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('canvas', 'frame_idx', 'frame_size', 'instances', 'skeleton_edges', 'node_names', 'scale', 'offset', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
Method generated by attrs for class RenderContext.
Source code in sleap_io/rendering/callbacks.py
@define
class RenderContext:
"""Context passed to pre/post render callbacks.
This context provides access to the Skia canvas and frame-level metadata
for drawing custom overlays before or after pose rendering.
Attributes:
canvas: Skia canvas for drawing.
frame_idx: Current frame index.
frame_size: (width, height) tuple of original frame dimensions.
__init__(canvas, frame_idx, frame_size, instances, skeleton_edges, node_names, scale=1.0, offset=(0.0, 0.0))
¶
Method generated by attrs for class RenderContext.
Source code in sleap_io/rendering/callbacks.py
__repr__()
¶
Method generated by attrs for class RenderContext.
Source code in sleap_io/rendering/callbacks.py
"""Callback context classes for custom rendering.
This module provides context objects that are passed to user-defined callbacks
during rendering, giving access to the Skia canvas and rendering metadata.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import numpy as np
from attrs import define
if TYPE_CHECKING:
import skia
world_to_canvas(x, y)
¶
Transform world coordinates to canvas coordinates.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
float
|
X coordinate in world/frame space. |
required |
y
|
float
|
Y coordinate in world/frame space. |
required |
Returns:
| Type | Description |
|---|---|
tuple[float, float]
|
(x, y) coordinates in canvas space. |
Source code in sleap_io/rendering/callbacks.py
def world_to_canvas(self, x: float, y: float) -> tuple[float, float]:
"""Transform world coordinates to canvas coordinates.
Args:
x: X coordinate in world/frame space.
y: Y coordinate in world/frame space.
Returns:
(x, y) coordinates in canvas space.
"""
return (
(x - self.offset[0]) * self.scale,
(y - self.offset[1]) * self.scale,
)
sleap_io.rendering.InstanceContext
¶
Context passed to per-instance callbacks.
This context provides access to the Skia canvas and instance-level metadata for drawing custom overlays after each instance is rendered.
Attributes:
| Name | Type | Description |
|---|---|---|
canvas |
Skia canvas for drawing. |
|
instance_idx |
Index of this instance within the frame. |
|
points |
(n_nodes, 2) array of keypoint coordinates. |
|
track_id |
Track ID if assigned, else None. |
|
track_name |
Track name string if available. |
|
confidence |
Instance confidence score if available. |
|
skeleton_edges |
Edge connectivity as list of (src, dst) tuples. |
|
node_names |
List of node name strings. |
|
scale |
Current scale factor for rendering. |
|
offset |
Current offset (x, y) for cropped/zoomed views. |
Methods:
| Name | Description |
|---|---|
__eq__ |
Method generated by attrs for class InstanceContext. |
__init__ |
Method generated by attrs for class InstanceContext. |
__repr__ |
Method generated by attrs for class InstanceContext. |
get_bbox |
Get bounding box of valid points. |
get_centroid |
Get centroid of valid points. |
world_to_canvas |
Transform world coordinates to canvas coordinates. |
Source code in sleap_io/rendering/callbacks.py
@define
class InstanceContext:
"""Context passed to per-instance callbacks.
This context provides access to the Skia canvas and instance-level metadata
for drawing custom overlays after each instance is rendered.
Attributes:
canvas: Skia canvas for drawing.
instance_idx: Index of this instance within the frame.
points: (n_nodes, 2) array of keypoint coordinates.
track_id: Track ID if assigned, else None.
track_name: Track name string if available.
confidence: Instance confidence score if available.
skeleton_edges: Edge connectivity as list of (src, dst) tuples.
node_names: List of node name strings.
scale: Current scale factor for rendering.
offset: Current offset (x, y) for cropped/zoomed views.
"""
canvas: "skia.Canvas"
instance_idx: int
points: np.ndarray
skeleton_edges: list[tuple[int, int]]
node_names: list[str]
track_id: int | None = None
track_name: str | None = None
confidence: float | None = None
scale: float = 1.0
offset: tuple[float, float] = (0.0, 0.0)
def world_to_canvas(self, x: float, y: float) -> tuple[float, float]:
"""Transform world coordinates to canvas coordinates.
Args:
x: X coordinate in world/frame space.
y: Y coordinate in world/frame space.
Returns:
(x, y) coordinates in canvas space.
"""
return (
(x - self.offset[0]) * self.scale,
(y - self.offset[1]) * self.scale,
)
def get_centroid(self) -> tuple[float, float] | None:
"""Get centroid of valid points.
Returns:
(x, y) mean of valid (non-NaN) points, or None if all invalid.
"""
valid_mask = np.isfinite(self.points).all(axis=1)
valid_points = self.points[valid_mask]
if len(valid_points) == 0:
return None
mean_pt = valid_points.mean(axis=0)
return (float(mean_pt[0]), float(mean_pt[1]))
def get_bbox(self) -> tuple[float, float, float, float] | None:
"""Get bounding box of valid points.
Returns:
(x1, y1, x2, y2) bounding box, or None if no valid points.
"""
valid_mask = np.isfinite(self.points).all(axis=1)
valid_points = self.points[valid_mask]
if len(valid_points) == 0:
return None
return (
float(valid_points[:, 0].min()),
float(valid_points[:, 1].min()),
float(valid_points[:, 0].max()),
float(valid_points[:, 1].max()),
)
__annotations__ = {'canvas': "'skia.Canvas'", 'instance_idx': 'int', 'points': 'np.ndarray', 'skeleton_edges': 'list[tuple[int, int]]', 'node_names': 'list[str]', 'track_id': 'int | None', 'track_name': 'str | None', 'confidence': 'float | None', 'scale': 'float', 'offset': 'tuple[float, float]'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = False
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Context passed to per-instance callbacks.\n\nThis context provides access to the Skia canvas and instance-level metadata\nfor drawing custom overlays after each instance is rendered.\n\nAttributes:\n canvas: Skia canvas for drawing.\n instance_idx: Index of this instance within the frame.\n points: (n_nodes, 2) array of keypoint coordinates.\n track_id: Track ID if assigned, else None.\n track_name: Track name string if available.\n confidence: Instance confidence score if available.\n skeleton_edges: Edge connectivity as list of (src, dst) tuples.\n node_names: List of node name strings.\n scale: Current scale factor for rendering.\n offset: Current offset (x, y) for cropped/zoomed views.\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 61
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('canvas', 'instance_idx', 'points', 'skeleton_edges', 'node_names', 'track_id', 'track_name', 'confidence', 'scale', 'offset')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.rendering.callbacks'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('canvas', 'instance_idx', 'points', 'skeleton_edges', 'node_names', 'track_id', 'track_name', 'confidence', 'scale', 'offset', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__eq__(other)
¶
Method generated by attrs for class InstanceContext.
Source code in sleap_io/rendering/callbacks.py
@define
class RenderContext:
"""Context passed to pre/post render callbacks.
This context provides access to the Skia canvas and frame-level metadata
for drawing custom overlays before or after pose rendering.
Attributes:
canvas: Skia canvas for drawing.
frame_idx: Current frame index.
frame_size: (width, height) tuple of original frame dimensions.
instances: List of instances in this frame.
skeleton_edges: Edge connectivity as list of (src, dst) tuples.
__init__(canvas, instance_idx, points, skeleton_edges, node_names, track_id=None, track_name=None, confidence=None, scale=1.0, offset=(0.0, 0.0))
¶
Method generated by attrs for class InstanceContext.
Source code in sleap_io/rendering/callbacks.py
__repr__()
¶
Method generated by attrs for class InstanceContext.
Source code in sleap_io/rendering/callbacks.py
"""Callback context classes for custom rendering.
This module provides context objects that are passed to user-defined callbacks
during rendering, giving access to the Skia canvas and rendering metadata.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import numpy as np
from attrs import define
if TYPE_CHECKING:
import skia
get_bbox()
¶
Get bounding box of valid points.
Returns:
| Type | Description |
|---|---|
tuple[float, float, float, float] | None
|
(x1, y1, x2, y2) bounding box, or None if no valid points. |
Source code in sleap_io/rendering/callbacks.py
def get_bbox(self) -> tuple[float, float, float, float] | None:
"""Get bounding box of valid points.
Returns:
(x1, y1, x2, y2) bounding box, or None if no valid points.
"""
valid_mask = np.isfinite(self.points).all(axis=1)
valid_points = self.points[valid_mask]
if len(valid_points) == 0:
return None
return (
float(valid_points[:, 0].min()),
float(valid_points[:, 1].min()),
float(valid_points[:, 0].max()),
float(valid_points[:, 1].max()),
)
get_centroid()
¶
Get centroid of valid points.
Returns:
| Type | Description |
|---|---|
tuple[float, float] | None
|
(x, y) mean of valid (non-NaN) points, or None if all invalid. |
Source code in sleap_io/rendering/callbacks.py
def get_centroid(self) -> tuple[float, float] | None:
"""Get centroid of valid points.
Returns:
(x, y) mean of valid (non-NaN) points, or None if all invalid.
"""
valid_mask = np.isfinite(self.points).all(axis=1)
valid_points = self.points[valid_mask]
if len(valid_points) == 0:
return None
mean_pt = valid_points.mean(axis=0)
return (float(mean_pt[0]), float(mean_pt[1]))
world_to_canvas(x, y)
¶
Transform world coordinates to canvas coordinates.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
float
|
X coordinate in world/frame space. |
required |
y
|
float
|
Y coordinate in world/frame space. |
required |
Returns:
| Type | Description |
|---|---|
tuple[float, float]
|
(x, y) coordinates in canvas space. |
Source code in sleap_io/rendering/callbacks.py
def world_to_canvas(self, x: float, y: float) -> tuple[float, float]:
"""Transform world coordinates to canvas coordinates.
Args:
x: X coordinate in world/frame space.
y: Y coordinate in world/frame space.
Returns:
(x, y) coordinates in canvas space.
"""
return (
(x - self.offset[0]) * self.scale,
(y - self.offset[1]) * self.scale,
)
sleap_io.draw_label_image(image, labels, alpha=0.3, palette='distinct', outline=False, outline_width=1, outline_color=None, scale=(1.0, 1.0), offset=(0.0, 0.0))
¶
Draw an integer label image as a colored overlay on an image.
This is an efficient rendering path for segmentation masks stored as integer label images (e.g., from instance or panoptic segmentation) where each pixel value represents a different object ID (0 = background).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image
|
ndarray
|
Image array of shape |
required |
labels
|
ndarray
|
Integer label array of shape |
required |
alpha
|
float
|
Opacity of the mask overlay (0.0 to 1.0). |
0.3
|
palette
|
str
|
Color palette name for assigning colors to label IDs. See
:func: |
'distinct'
|
outline
|
bool
|
If |
False
|
outline_width
|
int
|
Width of the outline in pixels (only used if
|
1
|
outline_color
|
tuple[int, int, int] | None
|
RGB color for outlines. If |
None
|
scale
|
tuple[float, float]
|
Resolution ratio |
(1.0, 1.0)
|
offset
|
tuple[float, float]
|
Origin |
(0.0, 0.0)
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The modified image array. |
Source code in sleap_io/rendering/overlays.py
def draw_label_image(
image: np.ndarray,
labels: np.ndarray,
alpha: float = 0.3,
palette: str = "distinct",
outline: bool = False,
outline_width: int = 1,
outline_color: tuple[int, int, int] | None = None,
scale: tuple[float, float] = (1.0, 1.0),
offset: tuple[float, float] = (0.0, 0.0),
) -> np.ndarray:
"""Draw an integer label image as a colored overlay on an image.
This is an efficient rendering path for segmentation masks stored as
integer label images (e.g., from instance or panoptic segmentation) where
each pixel value represents a different object ID (0 = background).
Args:
image: Image array of shape ``(H, W, 3)`` uint8. A 2-D grayscale
``(H, W)`` (or ``(H, W, 1)``) image is accepted and promoted to
RGB. Modified in-place and returned.
labels: Integer label array of shape ``(H, W)`` where 0 is background
and positive values are object IDs.
alpha: Opacity of the mask overlay (0.0 to 1.0).
palette: Color palette name for assigning colors to label IDs. See
:func:`~sleap_io.rendering.colors.get_palette` for options.
outline: If ``True``, draw outlines around each labeled region using
numpy edge detection.
outline_width: Width of the outline in pixels (only used if
``outline=True``).
outline_color: RGB color for outlines. If ``None``, uses a darkened
version of each region's fill color.
scale: Resolution ratio ``(sx, sy)`` for coordinate mapping.
offset: Origin ``(x, y)`` in image pixel coordinates.
Returns:
The modified image array.
"""
from sleap_io.rendering.colors import get_palette
# Ensure RGB so grayscale images can be blended with colored overlays.
image = _ensure_rgb(image)
# Get unique non-background labels
unique_ids = np.unique(labels)
unique_ids = unique_ids[unique_ids > 0]
if len(unique_ids) == 0:
return image
# Build color lookup table (LUT): label_id -> RGB
max_id = int(unique_ids.max())
palette_colors = get_palette(palette, max_id + 1)
# Create a LUT array: shape (max_id + 1, 3)
lut = np.zeros((max_id + 1, 3), dtype=np.float32)
for label_id in unique_ids:
lut[label_id] = palette_colors[int(label_id) % len(palette_colors)]
img_h, img_w = image.shape[:2]
has_transform = scale != (1.0, 1.0) or offset != (0.0, 0.0)
if has_transform:
from sleap_io.model.mask import _resize_nearest
sx, sy = scale
lab_h, lab_w = labels.shape[:2]
target_h = int(lab_h / sy)
target_w = int(lab_w / sx)
resized_labels = _resize_nearest(labels, target_h, target_w)
ox, oy = offset
y0 = max(0, int(oy))
x0 = max(0, int(ox))
y1 = min(img_h, int(oy) + target_h)
x1 = min(img_w, int(ox) + target_w)
if y1 <= y0 or x1 <= x0:
return image
my0 = y0 - int(oy)
mx0 = x0 - int(ox)
region = image[y0:y1, x0:x1]
label_region = resized_labels[my0 : my0 + (y1 - y0), mx0 : mx0 + (x1 - x0)]
draw_h = y1 - y0
draw_w = x1 - x0
else:
lab_h, lab_w = labels.shape[:2]
draw_h = min(lab_h, img_h)
draw_w = min(lab_w, img_w)
region = image[:draw_h, :draw_w]
label_region = labels[:draw_h, :draw_w]
# Vectorized blending: apply colored overlay where labels > 0
fg_mask = label_region > 0
if np.any(fg_mask):
# Clamp label values for LUT indexing
safe_labels = np.clip(label_region, 0, max_id)
overlay_colors = lut[safe_labels] # (H, W, 3)
region_float = region.astype(np.float32)
region_float[fg_mask] = (
region_float[fg_mask] * (1 - alpha) + overlay_colors[fg_mask] * alpha
)
region[:] = region_float.astype(np.uint8)
# Draw outlines if requested
if outline:
if has_transform:
# For transformed labels, draw outlines on the placed region
_draw_label_outlines(
image, labels, draw_h, draw_w, outline_width, outline_color, lut
)
else:
_draw_label_outlines(
image, labels, draw_h, draw_w, outline_width, outline_color, lut
)
return image
sleap_io.draw_masks(image, masks, color=(255, 0, 0), colors=None, alpha=0.3)
¶
Draw segmentation masks as colored overlays on an image.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image
|
ndarray
|
Image array of shape (H, W, 3) uint8. A 2-D grayscale
|
required |
masks
|
list[SegmentationMask]
|
List of SegmentationMask objects to draw. |
required |
color
|
tuple[int, int, int]
|
RGB color tuple for the mask overlay. Used when |
(255, 0, 0)
|
colors
|
list[tuple[int, int, int]] | None
|
Per-mask RGB color tuples. If provided, must have the same
length as |
None
|
alpha
|
float
|
Opacity of the mask overlay (0.0 to 1.0). |
0.3
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The modified image array. |
Source code in sleap_io/rendering/overlays.py
def draw_masks(
image: np.ndarray,
masks: list["SegmentationMask"],
color: tuple[int, int, int] = (255, 0, 0),
colors: list[tuple[int, int, int]] | None = None,
alpha: float = 0.3,
) -> np.ndarray:
"""Draw segmentation masks as colored overlays on an image.
Args:
image: Image array of shape (H, W, 3) uint8. A 2-D grayscale
``(H, W)`` (or ``(H, W, 1)``) image is accepted and promoted to
RGB. Modified in-place and returned.
masks: List of SegmentationMask objects to draw.
color: RGB color tuple for the mask overlay. Used when ``colors`` is
``None``.
colors: Per-mask RGB color tuples. If provided, must have the same
length as ``masks``. Overrides ``color``.
alpha: Opacity of the mask overlay (0.0 to 1.0).
Returns:
The modified image array.
"""
# Ensure RGB so grayscale images can be blended with colored overlays.
image = _ensure_rgb(image)
for i, mask in enumerate(masks):
mask_color = colors[i] if colors is not None else color
mask_data = mask.data
img_h, img_w = image.shape[:2]
if mask.has_spatial_transform:
from sleap_io.model.mask import _resize_nearest
# Compute image-space placement
target_h, target_w = mask.image_extent
resized = _resize_nearest(mask_data, target_h, target_w)
ox, oy = mask.offset
y0 = max(0, int(oy))
x0 = max(0, int(ox))
y1 = min(img_h, int(oy) + target_h)
x1 = min(img_w, int(ox) + target_w)
if y1 <= y0 or x1 <= x0:
continue
# Slice within the resized mask (handles negative offset)
my0 = y0 - int(oy)
mx0 = x0 - int(ox)
region = image[y0:y1, x0:x1]
mask_region = resized[my0 : my0 + (y1 - y0), mx0 : mx0 + (x1 - x0)]
else:
h, w = mask_data.shape
draw_h = min(h, img_h)
draw_w = min(w, img_w)
region = image[:draw_h, :draw_w]
mask_region = mask_data[:draw_h, :draw_w]
# Blend color into masked pixels
overlay = np.array(mask_color, dtype=np.float32)
region[mask_region] = (
region[mask_region] * (1 - alpha) + overlay * alpha
).astype(np.uint8)
return image
sleap_io.draw_bboxes(image, bboxes, color=(0, 255, 0), colors=None, line_width=2, fill_alpha=0.0, font=None)
¶
Draw bounding boxes on an image.
Draws bounding boxes as closed paths using skia-python. Both axis-aligned
and rotated bounding boxes are handled uniformly via corner points. For
PredictedBoundingBox instances, the confidence score is drawn as text
near the top-left corner.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image
|
ndarray
|
Image array of shape (H, W, 3) uint8. A 2-D grayscale
|
required |
bboxes
|
list[BoundingBox]
|
List of BoundingBox objects to draw. |
required |
color
|
tuple[int, int, int]
|
RGB color tuple for the bounding box outlines. Used when
|
(0, 255, 0)
|
colors
|
list[tuple[int, int, int]] | None
|
Per-bbox RGB color tuples. If provided, must have the same
length as |
None
|
line_width
|
int
|
Width of the outline in pixels. |
2
|
fill_alpha
|
float
|
If > 0, fill the bounding box interior with this opacity (0.0 to 1.0). |
0.0
|
font
|
str | None
|
Font family name for score text (e.g., |
None
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The modified image array. |
Source code in sleap_io/rendering/overlays.py
def draw_bboxes(
image: np.ndarray,
bboxes: list["BoundingBox"],
color: tuple[int, int, int] = (0, 255, 0),
colors: list[tuple[int, int, int]] | None = None,
line_width: int = 2,
fill_alpha: float = 0.0,
font: str | None = None,
) -> np.ndarray:
"""Draw bounding boxes on an image.
Draws bounding boxes as closed paths using skia-python. Both axis-aligned
and rotated bounding boxes are handled uniformly via corner points. For
``PredictedBoundingBox`` instances, the confidence score is drawn as text
near the top-left corner.
Args:
image: Image array of shape (H, W, 3) uint8. A 2-D grayscale
``(H, W)`` (or ``(H, W, 1)``) image is accepted and promoted to
RGB. Modified in-place and returned.
bboxes: List of BoundingBox objects to draw.
color: RGB color tuple for the bounding box outlines. Used when
``colors`` is ``None``.
colors: Per-bbox RGB color tuples. If provided, must have the same
length as ``bboxes``. Overrides ``color``.
line_width: Width of the outline in pixels.
fill_alpha: If > 0, fill the bounding box interior with this opacity
(0.0 to 1.0).
font: Font family name for score text (e.g., ``"Arial"``). If
``None``, uses the system default typeface.
Returns:
The modified image array.
"""
if not bboxes:
return image
import skia
from sleap_io.model.bbox import PredictedBoundingBox
# Ensure RGB before padding to RGBA.
image = _ensure_rgb(image)
# Pad to RGBA for skia surface
frame_rgba = np.dstack([image, np.full(image.shape[:2], 255, dtype=np.uint8)])
surface = skia.Surface(frame_rgba, colorType=skia.kRGBA_8888_ColorType)
canvas = surface.getCanvas()
if colors is None:
# Single color for all bboxes
stroke_paint = skia.Paint(
Color=skia.Color(*color),
AntiAlias=False,
Style=skia.Paint.kStroke_Style,
StrokeWidth=float(line_width),
StrokeCap=skia.Paint.kSquare_Cap,
)
fill_paint = None
if fill_alpha > 0:
fill_paint = skia.Paint(
Color=skia.Color4f(
color[0] / 255.0,
color[1] / 255.0,
color[2] / 255.0,
fill_alpha,
).toColor(),
AntiAlias=False,
Style=skia.Paint.kFill_Style,
)
for bbox in bboxes:
corners = bbox.corners
# Build a closed path from the 4 corners
path = skia.Path()
path.moveTo(float(corners[0][0]), float(corners[0][1]))
for j in range(1, len(corners)):
path.lineTo(float(corners[j][0]), float(corners[j][1]))
path.close()
# Draw fill if requested
if fill_paint is not None:
canvas.drawPath(path, fill_paint)
# Draw stroke
canvas.drawPath(path, stroke_paint)
# Score text for predicted bboxes
if isinstance(bbox, PredictedBoundingBox):
text_x = float(corners[0][0])
text_y = float(corners[0][1]) - 5
typeface = skia.Typeface(font if font else "sans-serif")
skia_font = skia.Font(typeface, 12)
text_paint = skia.Paint(Color=skia.Color(*color), AntiAlias=True)
canvas.drawString(
f"{bbox.score:.2f}", text_x, text_y, skia_font, text_paint
)
else:
# Per-bbox colors
for i, bbox in enumerate(bboxes):
c = colors[i]
stroke_paint = skia.Paint(
Color=skia.Color(*c),
AntiAlias=False,
Style=skia.Paint.kStroke_Style,
StrokeWidth=float(line_width),
StrokeCap=skia.Paint.kSquare_Cap,
)
fill_paint = None
if fill_alpha > 0:
fill_paint = skia.Paint(
Color=skia.Color4f(
c[0] / 255.0,
c[1] / 255.0,
c[2] / 255.0,
fill_alpha,
).toColor(),
AntiAlias=False,
Style=skia.Paint.kFill_Style,
)
corners = bbox.corners
# Build a closed path from the 4 corners
path = skia.Path()
path.moveTo(float(corners[0][0]), float(corners[0][1]))
for j in range(1, len(corners)):
path.lineTo(float(corners[j][0]), float(corners[j][1]))
path.close()
# Draw fill if requested
if fill_paint is not None:
canvas.drawPath(path, fill_paint)
# Draw stroke
canvas.drawPath(path, stroke_paint)
# Score text for predicted bboxes
if isinstance(bbox, PredictedBoundingBox):
text_x = float(corners[0][0])
text_y = float(corners[0][1]) - 5
typeface = skia.Typeface(font if font else "sans-serif")
skia_font = skia.Font(typeface, 12)
text_paint = skia.Paint(Color=skia.Color(*c), AntiAlias=True)
canvas.drawString(
f"{bbox.score:.2f}", text_x, text_y, skia_font, text_paint
)
# Copy RGB channels back to the input image
image[:] = frame_rgba[:, :, :3]
return image
sleap_io.draw_centroids(image, centroids, color=(0, 255, 0), colors=None, marker_size=5.0, alpha=1.0, offset=(0.0, 0.0))
¶
Draw centroid markers on an image.
Draws filled circles at each centroid position using skia-python.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image
|
ndarray
|
Image array of shape |
required |
centroids
|
list[Centroid]
|
List of |
required |
color
|
tuple[int, int, int]
|
RGB color tuple used when |
(0, 255, 0)
|
colors
|
list[tuple[int, int, int]] | None
|
Per-centroid RGB color tuples. If provided, must have the
same length as |
None
|
marker_size
|
float
|
Radius of the marker circle in pixels. |
5.0
|
alpha
|
float
|
Opacity for the markers (0.0 to 1.0). |
1.0
|
offset
|
tuple[float, float]
|
|
(0.0, 0.0)
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The modified image array. |
Source code in sleap_io/rendering/overlays.py
def draw_centroids(
image: np.ndarray,
centroids: list["Centroid"],
color: tuple[int, int, int] = (0, 255, 0),
colors: list[tuple[int, int, int]] | None = None,
marker_size: float = 5.0,
alpha: float = 1.0,
offset: tuple[float, float] = (0.0, 0.0),
) -> np.ndarray:
"""Draw centroid markers on an image.
Draws filled circles at each centroid position using skia-python.
Args:
image: Image array of shape ``(H, W, 3)`` uint8. Modified in-place
and returned.
centroids: List of ``Centroid`` objects to draw.
color: RGB color tuple used when ``colors`` is ``None``.
colors: Per-centroid RGB color tuples. If provided, must have the
same length as ``centroids``. Overrides ``color``.
marker_size: Radius of the marker circle in pixels.
alpha: Opacity for the markers (0.0 to 1.0).
offset: ``(ox, oy)`` offset to subtract from centroid coordinates
(used for cropped images).
Returns:
The modified image array.
"""
if not centroids:
return image
import skia
alpha_int = max(0, min(255, int(alpha * 255)))
ox, oy = offset
# Ensure RGB before padding to RGBA.
image = _ensure_rgb(image)
# Pad to RGBA for skia surface.
frame_rgba = np.dstack([image, np.full(image.shape[:2], 255, dtype=np.uint8)])
surface = skia.Surface(frame_rgba, colorType=skia.kRGBA_8888_ColorType)
canvas = surface.getCanvas()
for i, centroid in enumerate(centroids):
c = colors[i] if colors is not None else color
paint = skia.Paint(
Color=skia.Color(c[0], c[1], c[2], alpha_int),
AntiAlias=True,
Style=skia.Paint.kFill_Style,
)
cx = float(centroid.x) - ox
cy = float(centroid.y) - oy
canvas.drawCircle(cx, cy, float(marker_size), paint)
surface.flushAndSubmit()
result = frame_rgba[:, :, :3]
if image.shape == result.shape:
image[:] = result
return image
return result.copy()
sleap_io.draw_rois(image, rois, color=(0, 255, 0), colors=None, line_width=2, fill_alpha=0.0)
¶
Draw ROI geometries on an image.
Draws the boundary of each ROI's geometry using skia-python. Supports
Polygon, MultiPolygon, Point, MultiPoint, LineString,
MultiLineString, and GeometryCollection geometries.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image
|
ndarray
|
Image array of shape (H, W, 3) uint8. A 2-D grayscale
|
required |
rois
|
list[ROI]
|
List of ROI objects to draw. |
required |
color
|
tuple[int, int, int]
|
RGB color tuple for the ROI outlines. Used when |
(0, 255, 0)
|
colors
|
list[tuple[int, int, int]] | None
|
Per-ROI RGB color tuples. If provided, must have the same
length as |
None
|
line_width
|
int
|
Width of the outline in pixels. |
2
|
fill_alpha
|
float
|
If > 0, fill the ROI interior with this opacity (0.0 to 1.0). |
0.0
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The modified image array. |
Source code in sleap_io/rendering/overlays.py
def draw_rois(
image: np.ndarray,
rois: list["ROI"],
color: tuple[int, int, int] = (0, 255, 0),
colors: list[tuple[int, int, int]] | None = None,
line_width: int = 2,
fill_alpha: float = 0.0,
) -> np.ndarray:
"""Draw ROI geometries on an image.
Draws the boundary of each ROI's geometry using skia-python. Supports
``Polygon``, ``MultiPolygon``, ``Point``, ``MultiPoint``, ``LineString``,
``MultiLineString``, and ``GeometryCollection`` geometries.
Args:
image: Image array of shape (H, W, 3) uint8. A 2-D grayscale
``(H, W)`` (or ``(H, W, 1)``) image is accepted and promoted to
RGB. Modified in-place and returned.
rois: List of ROI objects to draw.
color: RGB color tuple for the ROI outlines. Used when ``colors`` is
``None``.
colors: Per-ROI RGB color tuples. If provided, must have the same
length as ``rois``. Overrides ``color``.
line_width: Width of the outline in pixels.
fill_alpha: If > 0, fill the ROI interior with this opacity (0.0 to
1.0).
Returns:
The modified image array.
"""
if not rois:
return image
import skia
# Ensure RGB before padding to RGBA.
image = _ensure_rgb(image)
# Pad to RGBA for skia surface
frame_rgba = np.dstack([image, np.full(image.shape[:2], 255, dtype=np.uint8)])
surface = skia.Surface(frame_rgba, colorType=skia.kRGBA_8888_ColorType)
canvas = surface.getCanvas()
if colors is None:
# Single color for all ROIs
stroke_paint = skia.Paint(
Color=skia.Color(*color),
AntiAlias=False,
Style=skia.Paint.kStroke_Style,
StrokeWidth=float(line_width),
StrokeCap=skia.Paint.kSquare_Cap,
)
fill_paint = None
if fill_alpha > 0:
fill_paint = skia.Paint(
Color=skia.Color4f(
color[0] / 255.0,
color[1] / 255.0,
color[2] / 255.0,
fill_alpha,
).toColor(),
AntiAlias=False,
Style=skia.Paint.kFill_Style,
)
for roi in rois:
_draw_geometry(canvas, roi.geometry, stroke_paint, fill_paint)
else:
# Per-ROI colors
for i, roi in enumerate(rois):
c = colors[i]
stroke_paint = skia.Paint(
Color=skia.Color(*c),
AntiAlias=False,
Style=skia.Paint.kStroke_Style,
StrokeWidth=float(line_width),
StrokeCap=skia.Paint.kSquare_Cap,
)
fill_paint = None
if fill_alpha > 0:
fill_paint = skia.Paint(
Color=skia.Color4f(
c[0] / 255.0,
c[1] / 255.0,
c[2] / 255.0,
fill_alpha,
).toColor(),
AntiAlias=False,
Style=skia.Paint.kFill_Style,
)
_draw_geometry(canvas, roi.geometry, stroke_paint, fill_paint)
# Copy RGB channels back to the input image
image[:] = frame_rgba[:, :, :3]
return image
sleap_io.draw_trails(image, trails, color=(0, 255, 0), colors=None, line_width=2.0, alpha_fade=True, alpha=1.0, offset=(0.0, 0.0))
¶
Draw motion trails as fading polylines on an image.
Each trail is a polyline tracing a node or centroid position across past frames. Segments are drawn individually so opacity can fade from faint (oldest) to opaque (newest).
All segments are rasterized into a separate transparent buffer with the
kSrc blend mode, so overlapping joints (and crossing trails) take the
newest segment's alpha instead of accumulating it. That buffer is then
composited onto the image in a single pass, touching only the pixels the
trails actually cover.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
image
|
ndarray
|
Image array of shape |
required |
trails
|
list[ndarray]
|
List of trails, where each trail is an |
required |
color
|
tuple[int, int, int]
|
RGB color tuple used when |
(0, 255, 0)
|
colors
|
list[tuple[int, int, int]] | None
|
Per-trail RGB color tuples. If provided, must have the same
length as |
None
|
line_width
|
float
|
Width of the trail line in pixels. |
2.0
|
alpha_fade
|
bool
|
If |
True
|
alpha
|
float
|
Global opacity multiplier (0.0 to 1.0). |
1.0
|
offset
|
tuple[float, float]
|
|
(0.0, 0.0)
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The image array with trails drawn. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in sleap_io/rendering/overlays.py
def draw_trails(
image: np.ndarray,
trails: list[np.ndarray],
color: tuple[int, int, int] = (0, 255, 0),
colors: list[tuple[int, int, int]] | None = None,
line_width: float = 2.0,
alpha_fade: bool = True,
alpha: float = 1.0,
offset: tuple[float, float] = (0.0, 0.0),
) -> np.ndarray:
"""Draw motion trails as fading polylines on an image.
Each trail is a polyline tracing a node or centroid position across past
frames. Segments are drawn individually so opacity can fade from faint
(oldest) to opaque (newest).
All segments are rasterized into a separate transparent buffer with the
``kSrc`` blend mode, so overlapping joints (and crossing trails) take the
newest segment's alpha instead of accumulating it. That buffer is then
composited onto the image in a single pass, touching only the pixels the
trails actually cover.
Args:
image: Image array of shape ``(H, W, 3)`` uint8. Modified in-place when
possible and returned.
trails: List of trails, where each trail is an ``(N, 2)`` float array of
``(x, y)`` coordinates ordered oldest to newest. Non-finite rows
(NaN) break the polyline so missing detections leave gaps.
color: RGB color tuple used when ``colors`` is ``None``.
colors: Per-trail RGB color tuples. If provided, must have the same
length as ``trails``. Overrides ``color``.
line_width: Width of the trail line in pixels.
alpha_fade: If ``True``, fade opacity from faint at the oldest segment to
fully opaque at the newest. If ``False``, all segments use ``alpha``.
alpha: Global opacity multiplier (0.0 to 1.0).
offset: ``(ox, oy)`` offset subtracted from coordinates (used for
cropped images).
Returns:
The image array with trails drawn.
Raises:
ValueError: If ``colors`` is provided and its length does not match
``trails``.
"""
if not trails:
return image
if colors is not None and len(colors) != len(trails):
raise ValueError(
f"colors has length {len(colors)} but there are {len(trails)} "
"trails; they must be the same length."
)
import skia
ox, oy = offset
# Ensure RGB so trail pixels can be composited back.
image = _ensure_rgb(image)
# Rasterize the trails into a separate transparent RGBA buffer. Drawing into
# a dedicated buffer (rather than the frame) lets the kSrc blend mode below
# replace overlapping joints instead of accumulating their alpha.
h, w = image.shape[:2]
trail_rgba = np.zeros((h, w, 4), dtype=np.uint8)
surface = skia.Surface(trail_rgba, colorType=skia.kRGBA_8888_ColorType)
canvas = surface.getCanvas()
for i, trail in enumerate(trails):
c = colors[i] if colors is not None else color
n_points = len(trail)
if n_points < 2:
# A single point has no segment to draw.
continue
# Each segment gets its own paint so opacity can fade per segment
# (a single skia.Path supports only one Paint).
n_segments = n_points - 1
for k in range(n_segments):
x0, y0 = trail[k]
x1, y1 = trail[k + 1]
if not (
np.isfinite(x0)
and np.isfinite(y0)
and np.isfinite(x1)
and np.isfinite(y1)
):
# Skip segments touching a missing (NaN) position.
continue
if alpha_fade:
# Newest segment (k = n_segments - 1) is fully opaque; oldest
# stays faintly visible rather than fully transparent.
seg_frac = max((k + 1) / n_segments, 0.05)
else:
seg_frac = 1.0
seg_alpha = max(0, min(255, int(seg_frac * alpha * 255)))
paint = skia.Paint(
Color=skia.Color(c[0], c[1], c[2], seg_alpha),
AntiAlias=True,
Style=skia.Paint.kStroke_Style,
StrokeWidth=float(line_width),
StrokeCap=skia.Paint.kRound_Cap,
BlendMode=skia.BlendMode.kSrc,
)
canvas.drawLine(
float(x0) - ox,
float(y0) - oy,
float(x1) - ox,
float(y1) - oy,
paint,
)
surface.flushAndSubmit()
# Composite only the pixels the trails actually covered (typically a small
# fraction of the frame). The buffer is unpremultiplied, so each covered
# pixel blends once as ``out = dst * (1 - a) + src * a``.
ys, xs = np.nonzero(trail_rgba[:, :, 3])
if len(ys):
a = trail_rgba[ys, xs, 3, None].astype(np.float32) / 255.0
src = trail_rgba[ys, xs, :3].astype(np.float32)
dst = image[ys, xs].astype(np.float32)
image[ys, xs] = (dst * (1.0 - a) + src * a).astype(np.uint8)
return image