# Rendering

sleap-io provides high-performance pose visualization using [skia-python](https://kyamagu.github.io/skia-python/), a production-quality 2D graphics library.

---

## Quick Start

Render a single frame to a numpy array:

```python
import sleap_io as sio

labels = sio.load_slp("predictions.slp")
img = sio.render_image(labels.labeled_frames[0])
```

![Basic rendering](assets/rendering/render_basic.png)

Render a full video:

```python
sio.render_video(labels, "output.mp4")
```

This produces an MP4 video file with skeleton overlays on all labeled frames.

---

## Color Schemes

Color scheme determines how poses are colored across instances and frames. The
`color_by` parameter defaults to `"auto"`, which resolves to `"track"` when tracks
are present, otherwise `"instance"` for a single image and `"node"` for a video or
multi-image render. Pass an explicit value to override:

### Color by track

Each tracked animal gets a consistent color across all frames:

```python
img = sio.render_image(lf, color_by="track")
```

![Color by track](assets/rendering/color_by_track.png)

### Color by instance

Each animal within a frame gets a unique color (colors may change between frames):

```python
img = sio.render_image(lf, color_by="instance")
```

![Color by instance](assets/rendering/color_by_instance.png)

### Color by node

Each body part gets a unique color (same across all animals):

```python
img = sio.render_image(lf, color_by="node")
```

![Color by node](assets/rendering/color_by_node.png)

---

## Color Palettes

### Built-in palettes

9 palettes are included with no additional dependencies.

#### standard

MATLAB default colors (default palette):

```python
img = sio.render_image(lf, color_by="node", palette="standard")
```

![palette standard](assets/rendering/palette_standard.png)

#### distinct

High-contrast colors for instances/tracks:

```python
img = sio.render_image(lf, color_by="node", palette="distinct")
```

![palette distinct](assets/rendering/palette_distinct.png)

#### rainbow

Spectrum colors for node types:

```python
img = sio.render_image(lf, color_by="node", palette="rainbow")
```

![palette rainbow](assets/rendering/palette_rainbow.png)

#### warm

Orange/red tones:

```python
img = sio.render_image(lf, color_by="node", palette="warm")
```

![palette warm](assets/rendering/palette_warm.png)

#### cool

Blue/purple tones:

```python
img = sio.render_image(lf, color_by="node", palette="cool")
```

![palette cool](assets/rendering/palette_cool.png)

#### pastel

Subtle colors for overlays:

```python
img = sio.render_image(lf, color_by="node", palette="pastel")
```

![palette pastel](assets/rendering/palette_pastel.png)

#### seaborn

Professional look for publications:

```python
img = sio.render_image(lf, color_by="node", palette="seaborn")
```

![palette seaborn](assets/rendering/palette_seaborn.png)

#### tableau10

Data visualization standard:

```python
img = sio.render_image(lf, color_by="node", palette="tableau10")
```

![palette tableau10](assets/rendering/palette_tableau10.png)

#### viridis

Perceptually uniform scientific:

```python
img = sio.render_image(lf, color_by="node", palette="viridis")
```

![palette viridis](assets/rendering/palette_viridis.png)

### Colorcet palettes

sleap-io includes [colorcet](https://colorcet.holoviz.org/) palettes:

#### glasbey

256 maximally distinct colors:

```python
img = sio.render_image(lf, color_by="node", palette="glasbey")
```

![palette glasbey](assets/rendering/palette_glasbey.png)

#### glasbey_hv

High visibility variant:

```python
img = sio.render_image(lf, color_by="node", palette="glasbey_hv")
```

![palette glasbey_hv](assets/rendering/palette_glasbey_hv.png)

#### glasbey_cool

Cool-toned variant:

```python
img = sio.render_image(lf, color_by="node", palette="glasbey_cool")
```

![palette glasbey_cool](assets/rendering/palette_glasbey_cool.png)

#### glasbey_warm

Warm-toned variant:

```python
img = sio.render_image(lf, color_by="node", palette="glasbey_warm")
```

![palette glasbey_warm](assets/rendering/palette_glasbey_warm.png)

### Getting palette colors programmatically

```python
colors = sio.rendering.get_palette("tableau10", 10)
# Returns: [(31, 119, 180), (255, 127, 14), ...]
```

---

## Marker Shapes

Five marker shapes are available for node visualization.

#### circle

Filled circle (default):

```python
img = sio.render_image(lf, marker_shape="circle", marker_size=6.0)
```

![shape circle](assets/rendering/shape_circle.png)

#### square

Filled square:

```python
img = sio.render_image(lf, marker_shape="square", marker_size=6.0)
```

![shape square](assets/rendering/shape_square.png)

#### diamond

Rotated square:

```python
img = sio.render_image(lf, marker_shape="diamond", marker_size=6.0)
```

![shape diamond](assets/rendering/shape_diamond.png)

#### triangle

Upward-pointing triangle:

```python
img = sio.render_image(lf, marker_shape="triangle", marker_size=6.0)
```

![shape triangle](assets/rendering/shape_triangle.png)

#### cross

Plus sign:

```python
img = sio.render_image(lf, marker_shape="cross", marker_size=6.0)
```

![shape cross](assets/rendering/shape_cross.png)

---

## Styling Options

### Marker and line sizes

Small markers and thin lines for detailed work:

```python
img = sio.render_image(lf, marker_size=3.0, line_width=1.5)
```

![size small](assets/rendering/size_small.png)

Medium markers and lines:

```python
img = sio.render_image(lf, marker_size=6.0, line_width=3.0)
```

![size medium](assets/rendering/size_medium.png)

Large markers and thick lines for visibility:

```python
img = sio.render_image(lf, marker_size=10.0, line_width=5.0)
```

![size large](assets/rendering/size_large.png)

### Transparency

Full opacity (default):

```python
img = sio.render_image(lf, alpha=1.0)
```

![alpha 100](assets/rendering/alpha_100.png)

Semi-transparent overlay:

```python
img = sio.render_image(lf, alpha=0.5)
```

![alpha 50](assets/rendering/alpha_50.png)

Subtle overlay:

```python
img = sio.render_image(lf, alpha=0.25)
```

![alpha 25](assets/rendering/alpha_25.png)

### Toggle elements

Both nodes and edges (default):

```python
img = sio.render_image(lf, show_nodes=True, show_edges=True)
```

![toggle both](assets/rendering/toggle_both.png)

Edges only:

```python
img = sio.render_image(lf, show_nodes=False, show_edges=True)
```

![toggle edges only](assets/rendering/toggle_edges_only.png)

Nodes only:

```python
img = sio.render_image(lf, show_nodes=True, show_edges=False)
```

![toggle nodes only](assets/rendering/toggle_nodes_only.png)

### Motion trails

Motion trails draw the trajectory of a node or centroid over the last
`trail_length` frames, so you can see how each animal moved through the
rendered output. Trails are drawn behind the poses and colored to match them
(by track when tracks are present, otherwise by instance).

```python
sio.render_video(
    labels,
    save_path="output.mp4",
    show_trails=True,        # enable trail drawing
    trail_length=10,         # number of past frames to trace
    trail_node="centroid",   # node name, list of node names, or "centroid"
    trail_width=2.0,         # trail line width in pixels
    trail_alpha_fade=True,   # fade from faint (oldest) to opaque (newest)
    trail_alpha=1.0,         # global trail opacity
    trail_color=None,        # uniform color, or None to match pose colors
)
```

`trail_node` accepts:

- `"centroid"` (default): trail the mean of each instance's visible nodes.
- A node name (e.g. `"head"`): trail that single node.
- A list of node names (e.g. `["head", "thorax"]`): draw one trail per node.

By default trails are colored to match the poses (by track when tracks are
present, otherwise by instance). Pass `trail_color` to override this with a
single uniform color for all trails — it accepts any color spec (RGB tuple,
named color, hex, or palette index). `trail_alpha` scales the overall opacity
and combines with `trail_alpha_fade`.

```python
# Faint white trails, uniform color, no fade.
sio.render_video(
    labels,
    save_path="output.mp4",
    show_trails=True,
    trail_color="white",
    trail_alpha=0.5,
    trail_alpha_fade=False,
)
```

Trails need temporal context, so they are only drawn for videos or for
`render_image` when the source is a `Labels` object (so past frames are
available). They are silently skipped for a single `LabeledFrame` or a list of
instances.

```python
# Trails also work on a single rendered frame from a Labels object.
img = sio.render_image(labels, lf_ind=100, show_trails=True, trail_length=20)
```

!!! note
    For untracked data, trails are matched by instance index across frames,
    which can jump between animals if the per-frame ordering changes. Trails
    are most reliable on tracked data.

---

## Scaling and Cropping

Control output resolution and focus on regions of interest.

### Output scaling

The `scale` parameter resizes the output. Graphics (markers, lines) scale proportionally:

```python
# Full resolution (default)
img = sio.render_image(lf, scale=1.0)

# Half resolution - faster, smaller files
img = sio.render_image(lf, scale=0.5)

# Quarter resolution - quick preview
img = sio.render_image(lf, scale=0.25)
```

![scale comparison](assets/rendering/scale_comparison.png)

### Cropping to a region

Use the `crop` parameter to render a specific region. Bounds are `(x1, y1, x2, y2)` where `(x1, y1)` is the top-left corner and `(x2, y2)` is the bottom-right (exclusive). Origin `(0, 0)` is at the image top-left.

Cropping works for both single images and videos. For videos, the same crop region is applied uniformly to all frames.

**Pixel coordinates** (integer tuple):

```python
import sleap_io as sio

labels = sio.load_slp("predictions.slp")
lf = labels.labeled_frames[0]

# Crop to region (x1, y1, x2, y2) in pixels
img = sio.render_image(lf, crop=(100, 100, 300, 300))

# Same for video rendering
sio.render_video(labels, "cropped.mp4", crop=(100, 100, 300, 300))
```

![crop region](assets/rendering/crop_region.png)

**Normalized coordinates** (float tuple in [0.0, 1.0]):

```python
# Crop center 50% of the frame
img = sio.render_image(lf, crop=(0.25, 0.25, 0.75, 0.75))

# Crop right half of the frame
img = sio.render_image(lf, crop=(0.5, 0.0, 1.0, 1.0))
```

Detection is type-based: all values must be Python `float` type and in [0.0, 1.0] range. Values outside this range (e.g., `(100.0, 100.0, 300.0, 300.0)`) are treated as pixel coordinates.

### Zoomed crop

Combine cropping with scaling for zoomed-in views:

```python
# Crop a small region and scale up for zoom effect
img = sio.render_image(lf, crop=(140, 120, 240, 220), scale=2.0)
```

![zoomed crop](assets/rendering/crop_zoomed.png)

---

## Background Control

Control the background when rendering poses. Use `background="video"` (default) to load video frames, or specify a color to render with a solid background.

!!! note "Remote video backgrounds"
    With `background="video"`, frames are read from the underlying [`Video`](model/video.md) backend, which can be a remote/cloud/Google-Drive source (see [Remote loading](remote.md)). Rendering then incurs the same network fetch, caching, and [`RemoteIOError`](remote.md) behavior as any other frame access.

### Named color

```python
img = sio.render_image(lf, background="black")
```

![background black](assets/rendering/background_black.png)

### RGB tuple

```python
img = sio.render_image(lf, background=(40, 40, 40))
```

![background rgb](assets/rendering/background_rgb.png)

### Hex color

```python
img = sio.render_image(lf, background="#1a1a2e")
```

![background hex](assets/rendering/background_hex.png)

### Palette color

```python
img = sio.render_image(lf, background="tableau10[0]")
```

![background palette](assets/rendering/background_palette.png)

### Color specification formats

The `background` parameter accepts many formats:

| Format | Example | Description |
|--------|---------|-------------|
| Named color | `"black"`, `"white"`, `"gray"` | Predefined color names |
| Hex (6-digit) | `"#ff8000"` | Standard hex color |
| Hex (3-digit) | `"#f80"` | Shorthand hex |
| RGB int tuple | `(255, 128, 0)` | Values 0-255 |
| RGB float tuple | `(1.0, 0.5, 0.0)` | Values 0.0-1.0 |
| Grayscale int | `40` | Single value 0-255 |
| Grayscale float | `0.15` | Single value 0.0-1.0 |
| Palette index | `"tableau10[0]"` | Color from palette |

Available named colors: `black`, `white`, `red`, `green`, `blue`, `yellow`, `cyan`, `magenta`, `gray`/`grey`, `orange`, `purple`, `pink`, `brown`.

---

## Creating Montages

Render multiple frames and combine them:

```python
import numpy as np
import sleap_io as sio

labels = sio.load_slp("predictions.slp")

frame_indices = [0, 100, 200, 300, 400]
frames = []
for i in frame_indices:
    img = sio.render_image(labels.labeled_frames[i], color_by="track")
    frames.append(img)

montage = np.concatenate(frames, axis=1)
```

![montage](assets/rendering/montage.png)

---

## Custom Rendering with Callbacks

Callbacks let you add custom graphics. You get direct access to the Skia canvas.

There are three callback types:

| Callback | Context Type | When Called |
|----------|--------------|-------------|
| `pre_render_callback` | [`RenderContext`](#sleap_io.rendering.RenderContext) | Before poses are drawn |
| `post_render_callback` | [`RenderContext`](#sleap_io.rendering.RenderContext) | After all poses are drawn |
| `per_instance_callback` | [`InstanceContext`](#sleap_io.rendering.InstanceContext) | After each instance is drawn |

### Instance labels

Draw track names above each instance:

```python
import skia
from sleap_io.rendering import InstanceContext

def draw_labels(ctx: InstanceContext):
    centroid = ctx.get_centroid()
    if centroid is None:
        return
    cx, cy = ctx.world_to_canvas(*centroid)
    font = skia.Font(skia.Typeface("Arial"), 14)
    label = ctx.track_name or f"Instance {ctx.instance_idx}"
    blob = skia.TextBlob(label, font)
    # Background
    bounds = font.measureText(label)
    bg = skia.Paint(Color=skia.Color4f(0, 0, 0, 0.6))
    ctx.canvas.drawRect(skia.Rect(cx - 2, cy - 18, cx + bounds + 2, cy - 4), bg)
    # Text
    paint = skia.Paint(Color=skia.ColorWHITE, AntiAlias=True)
    ctx.canvas.drawTextBlob(blob, cx, cy - 6, paint)

img = sio.render_image(lf, per_instance_callback=draw_labels)
```

![callback labels](assets/rendering/callback_labels.png)

### Bounding boxes

Draw dashed bounding boxes around instances:

```python
import skia
from sleap_io.rendering import InstanceContext

def draw_bbox(ctx: InstanceContext):
    bbox = ctx.get_bbox()
    if bbox is None:
        return
    x1, y1, x2, y2 = bbox
    x1, y1 = ctx.world_to_canvas(x1, y1)
    x2, y2 = ctx.world_to_canvas(x2, y2)
    pad = 8
    rect = skia.Rect(x1 - pad, y1 - pad, x2 + pad, y2 + pad)
    dash = skia.DashPathEffect.Make([6, 3], 0)
    paint = skia.Paint(
        Color=skia.ColorWHITE,
        Style=skia.Paint.kStroke_Style,
        StrokeWidth=2,
        PathEffect=dash,
    )
    ctx.canvas.drawRect(rect, paint)

img = sio.render_image(lf, per_instance_callback=draw_bbox)
```

![callback bbox](assets/rendering/callback_bbox.png)

### Frame info overlay

Add frame number and instance count:

```python
import skia
from sleap_io.rendering import RenderContext

def draw_frame_info(ctx: RenderContext):
    font = skia.Font(skia.Typeface("Arial"), 14)
    text = f"Frame: {ctx.frame_idx}  Instances: {len(ctx.instances)}"
    blob = skia.TextBlob(text, font)
    bg = skia.Paint(Color=skia.Color4f(0, 0, 0, 0.7))
    ctx.canvas.drawRect(skia.Rect(4, 4, 200, 24), bg)
    paint = skia.Paint(Color=skia.ColorWHITE, AntiAlias=True)
    ctx.canvas.drawTextBlob(blob, 8, 18, paint)

img = sio.render_image(lf, post_render_callback=draw_frame_info)
```

![callback frame info](assets/rendering/callback_frame_info.png)

### Combining callbacks

```python
from sleap_io.rendering import InstanceContext

def combined_per_instance(ctx: InstanceContext):
    draw_bbox(ctx)
    draw_labels(ctx)

img = sio.render_image(
    lf,
    per_instance_callback=combined_per_instance,
    post_render_callback=draw_frame_info,
)
```

![callback combined](assets/rendering/callback_combined.png)

---

## Segmentation Overlays

Overlay segmentation masks on images via `render_image`/`render_video` or standalone functions.

!!! tip "Masks and label images are drawn automatically"
    When no explicit `overlay` is passed and the labels carry annotations,
    `render_video` (and `sio render`) auto-draw them: **label images take
    precedence**, falling back to `SegmentationMask` annotations
    (`labels.get_masks(video=...)`, routed to the correct frame). `render_image`
    auto-draws the frame's `masks` when no overlay is given. So
    `sio render preds.slp -o out.mp4` draws segmentation predictions out of the
    box. Pass an explicit `overlay` to override. Auto-drawing works on both RGB
    and single-channel grayscale `(H, W, 1)` video.

!!! note "Coloring overlays by track"
    With `color_by="track"` (the default `"auto"` when the labels carry tracks),
    `SegmentationMask`, `ROI`, and `BoundingBox` overlays are colored by their
    `.track` identity using the pose `palette`, so a tracked object keeps a
    stable color across frames and matches its poses, centroids, and trails.
    Untracked elements use the first palette color. When `color_by` is not
    `"track"` (or the overlay carries no tracks), elements fall back to
    positional coloring from `overlay_palette` (default `"distinct"`).

### Using render_image / render_video

The `overlay` parameter on `render_image` and `render_video` accepts label images, `SegmentationMask`, `ROI`, or `BoundingBox` objects:

```python
import numpy as np
import sleap_io as sio

# Label image overlay (no poses)
img = sio.render_image(image=frame, overlay=label_mask, overlay_alpha=0.4)

# With outlines
img = sio.render_image(
    image=frame,
    overlay=label_mask,
    overlay_alpha=0.4,
    overlay_outline=True,
)

# SegmentationMask, ROI, or BoundingBox objects (a list, or a single object)
img = sio.render_image(image=frame, overlay=masks, overlay_alpha=0.3)
img = sio.render_image(image=frame, overlay=rois, overlay_alpha=0.3)
img = sio.render_image(image=frame, overlay=bboxes, overlay_alpha=0.3)

# A single annotation object also works (treated like a one-element list)
img = sio.render_image(image=frame, overlay=masks[0], overlay_alpha=0.3)

# Overlay on a labeled frame (poses render on top)
img = sio.render_image(lf, overlay=label_mask, overlay_alpha=0.4)
```

For video rendering, `overlay` can be a 3-D array `(T, H, W)`, a callable, or a list of objects with `frame_idx`:

```python
# 3-D label image stack
sio.render_video(labels, "output.mp4", overlay=label_stack)

# Callable for lazy per-frame loading
sio.render_video(labels, "output.mp4", overlay=lambda idx: load_mask(idx))

# SegmentationMask objects (filtered per-frame by frame_idx)
sio.render_video(labels, "output.mp4", overlay=masks)
```

### Standalone functions

For direct control, use `sio.draw_label_image`, `sio.draw_masks`, `sio.draw_rois`, `sio.draw_bboxes`, or `sio.draw_centroids`:

#### Label images (instance/panoptic segmentation)

```python
# label_image: (H, W) int array, 0=background, 1..N=object IDs
sio.draw_label_image(image, label_image, alpha=0.4, palette="distinct")
```

Add outlines around each segment:

```python
sio.draw_label_image(
    image,
    label_image,
    alpha=0.4,
    palette="distinct",
    outline=True,
    outline_width=2,
)
```

Use a uniform outline color:

```python
sio.draw_label_image(
    image,
    label_image,
    alpha=0.3,
    outline=True,
    outline_color=(255, 255, 255),
)
```

#### Binary segmentation masks

For `SegmentationMask` objects, use `sio.draw_masks` with per-mask coloring:

```python
# Single color for all masks
sio.draw_masks(image, masks, color=(255, 0, 0), alpha=0.3)

# Per-mask colors
colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255)]
sio.draw_masks(image, masks, colors=colors, alpha=0.3)
```

#### ROI geometries

For `ROI` objects (polygons, points, lines), use `sio.draw_rois`:

```python
sio.draw_rois(image, rois, color=(0, 255, 0), line_width=2, fill_alpha=0.2)
```

#### Bounding boxes

For `BoundingBox` objects, use `sio.draw_bboxes`:

```python
sio.draw_bboxes(image, bboxes, color=(0, 255, 0), line_width=2, fill_alpha=0.1)
```

#### Centroids

For `Centroid` objects, use `sio.draw_centroids` to draw a filled marker at each
position:

```python
sio.draw_centroids(image, centroids, color=(0, 255, 0), marker_size=5.0)
```

---

## Video Rendering

### Basic video rendering

```python
sio.render_video(labels, "output.mp4")
```

### Render a clip

```python
sio.render_video(labels, "clip.mp4", start=100, end=200)
```

### Quality presets

```python
# Fast preview (0.25x resolution)
sio.render_video(labels, "preview.mp4", preset="preview")

# Draft quality (0.5x resolution)
sio.render_video(labels, "draft.mp4", preset="draft")

# Full quality (default)
sio.render_video(labels, "final.mp4", preset="final")
```

### Encoding options

```python
sio.render_video(
    labels,
    "output.mp4",
    fps=30.0,           # Output frame rate
    crf=18,             # Quality (2-32, lower=better)
    x264_preset="slow", # Encoding speed
)
```

---

## CLI Reference

For CLI usage, see the [CLI Guide](cli.md#sio-render).

```bash
# Basic rendering
sio render -i predictions.slp -o output.mp4

# Fast preview
sio render -i predictions.slp --preset preview

# Single frame to PNG
sio render -i predictions.slp --lf 0 -o frame.png

# Custom styling
sio render -i predictions.slp -o styled.mp4 \
    --color-by track --palette tableau10 --marker-shape diamond

# Motion trails
sio render -i predictions.slp -o output.mp4 --trails --trail-length 10

# Trails for specific nodes
sio render -i predictions.slp -o output.mp4 --trails --trail-node head,thorax

# Styled trails (uniform color, faint, no fade)
sio render -i predictions.slp -o output.mp4 --trails \
    --trail-color white --trail-alpha 0.5 --no-trail-fade

# Disable the progress bar
sio render -i predictions.slp -o output.mp4 --no-progress

# Segmentation overlay from TIFF stack
sio render -i predictions.slp --overlay masks.tif --overlay-alpha 0.4

# Overlay from directory of per-frame TIFFs
sio render -i predictions.slp --overlay masks/

# Single frame with overlay and outlines
sio render -i predictions.slp --lf 0 --overlay masks.tif \
    --overlay-outline --overlay-outline-color white

# Overlay-only mode (no labels file needed)
sio render --images frames/ --overlay masks.tif -o output.mp4

# Overlay-only with TIFF stack images
sio render --images frames.tif --overlay masks/ --overlay-outline -o output.mp4

# Color overlay with a specific palette
sio render -i predictions.slp --overlay masks.tif --overlay-palette tableau10

# Discover palettes and named colors
sio render --list-palettes
sio render --list-colors
```

---

## API Reference

::: sleap_io.render_video
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.render_image
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.rendering.get_palette
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.rendering.resolve_color
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.rendering.RenderContext
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.rendering.InstanceContext
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.draw_label_image
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.draw_masks
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.draw_bboxes
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.draw_centroids
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.draw_rois
    options:
      show_root_heading: true
      heading_level: 3

::: sleap_io.draw_trails
    options:
      show_root_heading: true
      heading_level: 3