video
sleap_io.transform.video
¶
Video transformation pipeline.
This module provides functions for transforming videos and their associated label coordinates. It handles loading videos, applying frame transformations, updating landmark coordinates, and saving the results.
Classes:
| Name | Description |
|---|---|
Transform |
Composable geometric transformation for video frames and coordinates. |
Functions:
| Name | Description |
|---|---|
compute_transform_summary |
Compute a summary of transforms to be applied. |
get_out_of_bounds_mask |
Get a boolean mask indicating which points are outside bounds. |
transform_embedded_video |
Transform an embedded video and write to an HDF5 output file. |
transform_labels |
Transform all videos in a Labels object and update coordinates. |
transform_video |
Transform a video file and save to a new path. |
Attributes:
| Name | Type | Description |
|---|---|---|
TYPE_CHECKING |
Returns True when the argument is true, False otherwise. |
|
__cached__ |
str(object='') -> str |
|
__doc__ |
str(object='') -> str |
|
__file__ |
str(object='') -> str |
|
__name__ |
str(object='') -> str |
|
__package__ |
str(object='') -> str |
TYPE_CHECKING = False
module-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__cached__ = '/home/runner/work/sleap-io/sleap-io/sleap_io/transform/__pycache__/video.cpython-313.pyc'
module-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__doc__ = 'Video transformation pipeline.\n\nThis module provides functions for transforming videos and their associated\nlabel coordinates. It handles loading videos, applying frame transformations,\nupdating landmark coordinates, and saving the results.\n'
module-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__file__ = '/home/runner/work/sleap-io/sleap-io/sleap_io/transform/video.py'
module-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__name__ = 'sleap_io.transform.video'
module-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__package__ = 'sleap_io.transform'
module-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
Transform
¶
Composable geometric transformation for video frames and coordinates.
Transforms are applied in a fixed pipeline order: crop -> scale -> rotate -> pad -> flip. This ensures consistent and predictable behavior when combining transforms.
Attributes:
| Name | Type | Description |
|---|---|---|
crop |
Crop region as (x1, y1, x2, y2) pixel coordinates. The region from (x1, y1) to (x2, y2) is extracted, where (x2, y2) is exclusive. |
|
scale |
Scale factors as (scale_x, scale_y). Use (0.5, 0.5) for 50% size. Can also be specified as a single float for uniform scaling. |
|
rotate |
Rotation angle in degrees. Positive values rotate clockwise. |
|
pad |
Padding as (top, right, bottom, left) in pixels. |
|
quality |
Interpolation quality for frame transforms. One of "nearest", "bilinear", or "bicubic". |
|
fill |
Fill value for out-of-bounds regions. Can be a single int for grayscale or (R, G, B) tuple for color. |
|
clip_rotation |
If True, rotation clips to original dimensions. If False (default), canvas expands to fit the entire rotated image. |
|
flip_h |
If True, flip horizontally (mirror left-right). |
|
flip_v |
If True, flip vertically (mirror top-bottom). |
Methods:
| Name | Description |
|---|---|
__bool__ |
Return True if any transformation is defined. |
__eq__ |
Method generated by attrs for class Transform. |
__init__ |
Method generated by attrs for class Transform. |
__repr__ |
Method generated by attrs for class Transform. |
apply_to_frame |
Transform a video frame. |
apply_to_points |
Transform landmark coordinates. |
output_size |
Compute output dimensions after applying the transformation. |
to_matrix |
Compute combined 3x3 affine transformation matrix. |
Source code in sleap_io/transform/core.py
@attrs.define
class Transform:
"""Composable geometric transformation for video frames and coordinates.
Transforms are applied in a fixed pipeline order:
crop -> scale -> rotate -> pad -> flip.
This ensures consistent and predictable behavior when combining transforms.
Attributes:
crop: Crop region as (x1, y1, x2, y2) pixel coordinates. The region from
(x1, y1) to (x2, y2) is extracted, where (x2, y2) is exclusive.
scale: Scale factors as (scale_x, scale_y). Use (0.5, 0.5) for 50% size.
Can also be specified as a single float for uniform scaling.
rotate: Rotation angle in degrees. Positive values rotate clockwise.
pad: Padding as (top, right, bottom, left) in pixels.
quality: Interpolation quality for frame transforms. One of "nearest",
"bilinear", or "bicubic".
fill: Fill value for out-of-bounds regions. Can be a single int for
grayscale or (R, G, B) tuple for color.
clip_rotation: If True, rotation clips to original dimensions. If False
(default), canvas expands to fit the entire rotated image.
flip_h: If True, flip horizontally (mirror left-right).
flip_v: If True, flip vertically (mirror top-bottom).
"""
crop: tuple[int, int, int, int] | None = None
scale: tuple[float, float] | None = None
rotate: float | None = None
pad: tuple[int, int, int, int] | None = None
quality: str = "bilinear"
fill: tuple[int, ...] | int = 0
clip_rotation: bool = False
flip_h: bool = False
flip_v: bool = False
def _rotation_output_size(
self, width: int, height: int
) -> tuple[int, int, float, float]:
"""Compute output size and center offset for rotation.
Args:
width: Pre-rotation width.
height: Pre-rotation height.
Returns:
Tuple of (new_width, new_height, offset_x, offset_y) where offsets
are the translation needed to center the rotated content.
"""
if self.rotate is None or self.rotate == 0 or self.clip_rotation:
return (width, height, 0.0, 0.0)
angle_rad = np.radians(abs(self.rotate))
cos_a = abs(np.cos(angle_rad))
sin_a = abs(np.sin(angle_rad))
# New bounding box dimensions
new_width = int(np.ceil(width * cos_a + height * sin_a))
new_height = int(np.ceil(width * sin_a + height * cos_a))
# Offset to center the rotated image in the new canvas
offset_x = (new_width - width) / 2
offset_y = (new_height - height) / 2
return (new_width, new_height, offset_x, offset_y)
def output_size(self, input_size: tuple[int, int]) -> tuple[int, int]:
"""Compute output dimensions after applying the transformation.
Args:
input_size: Input (width, height) in pixels.
Returns:
Output (width, height) in pixels.
"""
width, height = input_size
# Apply crop
if self.crop is not None:
x1, y1, x2, y2 = self.crop
width = x2 - x1
height = y2 - y1
# Apply scale
if self.scale is not None:
scale_x, scale_y = self.scale
width = int(round(width * scale_x))
height = int(round(height * scale_y))
# Apply rotation (may expand canvas if not clipping)
if self.rotate is not None and self.rotate != 0:
width, height, _, _ = self._rotation_output_size(width, height)
# Apply pad
if self.pad is not None:
top, right, bottom, left = self.pad
width = width + left + right
height = height + top + bottom
return (width, height)
def to_matrix(self, input_size: tuple[int, int]) -> np.ndarray:
"""Compute combined 3x3 affine transformation matrix.
The transformation matrix can be used to transform homogeneous coordinates:
[new_x] [a b tx] [old_x]
[new_y] = [c d ty] [old_y]
[ 1 ] [0 0 1] [ 1 ]
Args:
input_size: Input (width, height) in pixels.
Returns:
3x3 affine transformation matrix as numpy array.
"""
# Start with identity matrix
matrix = np.eye(3, dtype=np.float64)
width, height = input_size
# Apply crop (translate by negative crop origin)
if self.crop is not None:
x1, y1, x2, y2 = self.crop
crop_matrix = np.array(
[[1, 0, -x1], [0, 1, -y1], [0, 0, 1]], dtype=np.float64
)
matrix = crop_matrix @ matrix
width = x2 - x1
height = y2 - y1
# Apply scale
if self.scale is not None:
scale_x, scale_y = self.scale
scale_matrix = np.array(
[[scale_x, 0, 0], [0, scale_y, 0], [0, 0, 1]], dtype=np.float64
)
matrix = scale_matrix @ matrix
width = int(round(width * scale_x))
height = int(round(height * scale_y))
# Apply rotation (about center of current frame)
# Note: SLEAP uses center pixel indexing where (0, 0) is the center of the
# top-left pixel. The geometric center of an image is at ((w-1)/2, (h-1)/2).
if self.rotate is not None and self.rotate != 0:
angle_rad = np.radians(self.rotate)
cos_a = np.cos(angle_rad)
sin_a = np.sin(angle_rad)
cx, cy = (width - 1) / 2, (height - 1) / 2
# Get rotation output size (may expand if not clipping)
new_width, new_height, offset_x, offset_y = self._rotation_output_size(
width, height
)
# Rotation about center, then translate to new center if expanded
# Combined: T(new_cx, new_cy) @ R @ T(-cx, -cy)
# Note: Image coordinates use y-down, so clockwise rotation matrix is:
# [cos, -sin]
# [sin, cos]
new_cx = (new_width - 1) / 2
new_cy = (new_height - 1) / 2
rotate_matrix = np.array(
[
[cos_a, -sin_a, new_cx - cos_a * cx + sin_a * cy],
[sin_a, cos_a, new_cy - sin_a * cx - cos_a * cy],
[0, 0, 1],
],
dtype=np.float64,
)
matrix = rotate_matrix @ matrix
width, height = new_width, new_height
# Apply pad (translate by padding offset)
if self.pad is not None:
top, right, bottom, left = self.pad
pad_matrix = np.array(
[[1, 0, left], [0, 1, top], [0, 0, 1]], dtype=np.float64
)
matrix = pad_matrix @ matrix
width = width + left + right
height = height + top + bottom
# Apply horizontal flip (x -> (width - 1) - x)
# With center pixel indexing, pixel centers range from 0 to width-1,
# so we flip around (width-1)/2 by mapping x -> (width-1) - x.
if self.flip_h:
flip_h_matrix = np.array(
[[-1, 0, width - 1], [0, 1, 0], [0, 0, 1]], dtype=np.float64
)
matrix = flip_h_matrix @ matrix
# Apply vertical flip (y -> (height - 1) - y)
# With center pixel indexing, pixel centers range from 0 to height-1,
# so we flip around (height-1)/2 by mapping y -> (height-1) - y.
if self.flip_v:
flip_v_matrix = np.array(
[[1, 0, 0], [0, -1, height - 1], [0, 0, 1]], dtype=np.float64
)
matrix = flip_v_matrix @ matrix
return matrix
def apply_to_points(
self, points: np.ndarray, input_size: tuple[int, int]
) -> np.ndarray:
"""Transform landmark coordinates.
Args:
points: Coordinate array of shape (n_points, 2) or (n_points, D) where
the first two columns are (x, y) coordinates. NaN values are preserved.
input_size: Input (width, height) in pixels.
Returns:
Transformed coordinates with same shape as input.
"""
if points.size == 0:
return points.copy()
# Handle both (n, 2) and (n, D) arrays
xy = points[..., :2].copy()
result = points.copy()
# Get transformation matrix
matrix = self.to_matrix(input_size)
# Create mask for valid (non-NaN) points
valid_mask = ~np.isnan(xy).any(axis=-1)
if valid_mask.any():
# Convert to homogeneous coordinates
valid_xy = xy[valid_mask]
ones = np.ones((valid_xy.shape[0], 1), dtype=np.float64)
homogeneous = np.hstack([valid_xy, ones])
# Apply transformation
transformed = (matrix @ homogeneous.T).T
# Extract x, y from homogeneous coordinates
result[valid_mask, 0] = transformed[:, 0]
result[valid_mask, 1] = transformed[:, 1]
return result
def apply_to_frame(self, frame: np.ndarray) -> np.ndarray:
"""Transform a video frame.
Args:
frame: Input frame as numpy array with shape (H, W) or (H, W, C).
Returns:
Transformed frame as numpy array.
"""
from sleap_io.transform.frame import transform_frame
return transform_frame(
frame,
crop=self.crop,
scale=self.scale,
rotate=self.rotate,
pad=self.pad,
quality=self.quality,
fill=self.fill,
expand_rotation=not self.clip_rotation,
flip_h=self.flip_h,
flip_v=self.flip_v,
)
def __bool__(self) -> bool:
"""Return True if any transformation is defined."""
return any(
[
self.crop is not None,
self.scale is not None,
self.rotate is not None and self.rotate != 0,
self.pad is not None and any(p != 0 for p in self.pad),
self.flip_h,
self.flip_v,
]
)
__annotations__ = {'crop': 'tuple[int, int, int, int] | None', 'scale': 'tuple[float, float] | None', 'rotate': 'float | None', 'pad': 'tuple[int, int, int, int] | None', 'quality': 'str', 'fill': 'tuple[int, ...] | int', 'clip_rotation': 'bool', 'flip_h': 'bool', 'flip_v': 'bool'}
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__attrs_own_setattr__ = False
class-attribute
¶
Returns True when the argument is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.
__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe at 0x7f08471ce840>, field_transformer=None)
class-attribute
¶
Effective class properties as derived from parameters to attr.s() or
define() decorators.
This is the same data structure that attrs uses internally to decide how to construct the final class.
Warning:
This feature is currently **experimental** and is not covered by our
strict backwards-compatibility guarantees.
Attributes:
| Name | Type | Description |
|---|---|---|
is_exception |
bool
|
Whether the class is treated as an exception class. |
is_slotted |
bool
|
Whether the class is |
has_weakref_slot |
bool
|
Whether the class has a slot for weak references. |
is_frozen |
bool
|
Whether the class is frozen. |
kw_only |
KeywordOnly
|
Whether / how the class enforces keyword-only arguments on the
|
collected_fields_by_mro |
bool
|
Whether the class fields were collected by method resolution order.
That is, correctly but unlike |
added_init |
bool
|
Whether the class has an attrs-generated |
added_repr |
bool
|
Whether the class has an attrs-generated |
added_eq |
bool
|
Whether the class has attrs-generated equality methods. |
added_ordering |
bool
|
Whether the class has attrs-generated ordering methods. |
hashability |
Hashability
|
How |
added_match_args |
bool
|
Whether the class supports positional |
added_str |
bool
|
Whether the class has an attrs-generated |
added_pickling |
bool
|
Whether the class has attrs-generated |
on_setattr_hook |
Callable[[Any, Attribute[Any], Any], Any] | None
|
The class's |
field_transformer |
Callable[[Attribute[Any]], Attribute[Any]] | None
|
The class's |
.. versionadded:: 25.4.0
__doc__ = 'Composable geometric transformation for video frames and coordinates.\n\nTransforms are applied in a fixed pipeline order:\ncrop -> scale -> rotate -> pad -> flip.\nThis ensures consistent and predictable behavior when combining transforms.\n\nAttributes:\n crop: Crop region as (x1, y1, x2, y2) pixel coordinates. The region from\n (x1, y1) to (x2, y2) is extracted, where (x2, y2) is exclusive.\n scale: Scale factors as (scale_x, scale_y). Use (0.5, 0.5) for 50% size.\n Can also be specified as a single float for uniform scaling.\n rotate: Rotation angle in degrees. Positive values rotate clockwise.\n pad: Padding as (top, right, bottom, left) in pixels.\n quality: Interpolation quality for frame transforms. One of "nearest",\n "bilinear", or "bicubic".\n fill: Fill value for out-of-bounds regions. Can be a single int for\n grayscale or (R, G, B) tuple for color.\n clip_rotation: If True, rotation clips to original dimensions. If False\n (default), canvas expands to fit the entire rotated image.\n flip_h: If True, flip horizontally (mirror left-right).\n flip_v: If True, flip vertically (mirror top-bottom).\n'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__firstlineno__ = 14
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral.
int('0b100', base=0) 4
__match_args__ = ('crop', 'scale', 'rotate', 'pad', 'quality', 'fill', 'clip_rotation', 'flip_h', 'flip_v')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__ = 'sleap_io.transform.core'
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to 'utf-8'. errors defaults to 'strict'.
__slots__ = ('crop', 'scale', 'rotate', 'pad', 'quality', 'fill', 'clip_rotation', 'flip_h', 'flip_v', '__weakref__')
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__static_attributes__ = ()
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__weakref__
property
¶
list of weak references to the object
__bool__()
¶
Return True if any transformation is defined.
Source code in sleap_io/transform/core.py
__eq__(other)
¶
Method generated by attrs for class Transform.
Source code in sleap_io/transform/core.py
"""Composable geometric transformation for video frames and coordinates.
Transforms are applied in a fixed pipeline order:
crop -> scale -> rotate -> pad -> flip.
This ensures consistent and predictable behavior when combining transforms.
Attributes:
crop: Crop region as (x1, y1, x2, y2) pixel coordinates. The region from
(x1, y1) to (x2, y2) is extracted, where (x2, y2) is exclusive.
scale: Scale factors as (scale_x, scale_y). Use (0.5, 0.5) for 50% size.
Can also be specified as a single float for uniform scaling.
rotate: Rotation angle in degrees. Positive values rotate clockwise.
pad: Padding as (top, right, bottom, left) in pixels.
quality: Interpolation quality for frame transforms. One of "nearest",
__init__(crop=None, scale=None, rotate=None, pad=None, quality='bilinear', fill=0, clip_rotation=False, flip_h=False, flip_v=False)
¶
Method generated by attrs for class Transform.
Source code in sleap_io/transform/core.py
"bilinear", or "bicubic".
fill: Fill value for out-of-bounds regions. Can be a single int for
grayscale or (R, G, B) tuple for color.
clip_rotation: If True, rotation clips to original dimensions. If False
(default), canvas expands to fit the entire rotated image.
flip_h: If True, flip horizontally (mirror left-right).
flip_v: If True, flip vertically (mirror top-bottom).
"""
crop: tuple[int, int, int, int] | None = None
__repr__()
¶
Method generated by attrs for class Transform.
Source code in sleap_io/transform/core.py
"""Core Transform class for composable geometric transformations.
This module defines the Transform dataclass which represents a composable
geometric transformation that can be applied to both video frames and
landmark coordinates.
"""
from __future__ import annotations
import attrs
import numpy as np
@attrs.define
class Transform:
apply_to_frame(frame)
¶
Transform a video frame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
frame
|
ndarray
|
Input frame as numpy array with shape (H, W) or (H, W, C). |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
Transformed frame as numpy array. |
Source code in sleap_io/transform/core.py
def apply_to_frame(self, frame: np.ndarray) -> np.ndarray:
"""Transform a video frame.
Args:
frame: Input frame as numpy array with shape (H, W) or (H, W, C).
Returns:
Transformed frame as numpy array.
"""
from sleap_io.transform.frame import transform_frame
return transform_frame(
frame,
crop=self.crop,
scale=self.scale,
rotate=self.rotate,
pad=self.pad,
quality=self.quality,
fill=self.fill,
expand_rotation=not self.clip_rotation,
flip_h=self.flip_h,
flip_v=self.flip_v,
)
apply_to_points(points, input_size)
¶
Transform landmark coordinates.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
ndarray
|
Coordinate array of shape (n_points, 2) or (n_points, D) where the first two columns are (x, y) coordinates. NaN values are preserved. |
required |
input_size
|
tuple[int, int]
|
Input (width, height) in pixels. |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
Transformed coordinates with same shape as input. |
Source code in sleap_io/transform/core.py
def apply_to_points(
self, points: np.ndarray, input_size: tuple[int, int]
) -> np.ndarray:
"""Transform landmark coordinates.
Args:
points: Coordinate array of shape (n_points, 2) or (n_points, D) where
the first two columns are (x, y) coordinates. NaN values are preserved.
input_size: Input (width, height) in pixels.
Returns:
Transformed coordinates with same shape as input.
"""
if points.size == 0:
return points.copy()
# Handle both (n, 2) and (n, D) arrays
xy = points[..., :2].copy()
result = points.copy()
# Get transformation matrix
matrix = self.to_matrix(input_size)
# Create mask for valid (non-NaN) points
valid_mask = ~np.isnan(xy).any(axis=-1)
if valid_mask.any():
# Convert to homogeneous coordinates
valid_xy = xy[valid_mask]
ones = np.ones((valid_xy.shape[0], 1), dtype=np.float64)
homogeneous = np.hstack([valid_xy, ones])
# Apply transformation
transformed = (matrix @ homogeneous.T).T
# Extract x, y from homogeneous coordinates
result[valid_mask, 0] = transformed[:, 0]
result[valid_mask, 1] = transformed[:, 1]
return result
output_size(input_size)
¶
Compute output dimensions after applying the transformation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_size
|
tuple[int, int]
|
Input (width, height) in pixels. |
required |
Returns:
| Type | Description |
|---|---|
tuple[int, int]
|
Output (width, height) in pixels. |
Source code in sleap_io/transform/core.py
def output_size(self, input_size: tuple[int, int]) -> tuple[int, int]:
"""Compute output dimensions after applying the transformation.
Args:
input_size: Input (width, height) in pixels.
Returns:
Output (width, height) in pixels.
"""
width, height = input_size
# Apply crop
if self.crop is not None:
x1, y1, x2, y2 = self.crop
width = x2 - x1
height = y2 - y1
# Apply scale
if self.scale is not None:
scale_x, scale_y = self.scale
width = int(round(width * scale_x))
height = int(round(height * scale_y))
# Apply rotation (may expand canvas if not clipping)
if self.rotate is not None and self.rotate != 0:
width, height, _, _ = self._rotation_output_size(width, height)
# Apply pad
if self.pad is not None:
top, right, bottom, left = self.pad
width = width + left + right
height = height + top + bottom
return (width, height)
to_matrix(input_size)
¶
Compute combined 3x3 affine transformation matrix.
The transformation matrix can be used to transform homogeneous coordinates
[new_x] [a b tx][old_x] [new_y] = [c d ty][old_y] [ 1 ] [0 0 1][ 1 ]
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_size
|
tuple[int, int]
|
Input (width, height) in pixels. |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
3x3 affine transformation matrix as numpy array. |
Source code in sleap_io/transform/core.py
def to_matrix(self, input_size: tuple[int, int]) -> np.ndarray:
"""Compute combined 3x3 affine transformation matrix.
The transformation matrix can be used to transform homogeneous coordinates:
[new_x] [a b tx] [old_x]
[new_y] = [c d ty] [old_y]
[ 1 ] [0 0 1] [ 1 ]
Args:
input_size: Input (width, height) in pixels.
Returns:
3x3 affine transformation matrix as numpy array.
"""
# Start with identity matrix
matrix = np.eye(3, dtype=np.float64)
width, height = input_size
# Apply crop (translate by negative crop origin)
if self.crop is not None:
x1, y1, x2, y2 = self.crop
crop_matrix = np.array(
[[1, 0, -x1], [0, 1, -y1], [0, 0, 1]], dtype=np.float64
)
matrix = crop_matrix @ matrix
width = x2 - x1
height = y2 - y1
# Apply scale
if self.scale is not None:
scale_x, scale_y = self.scale
scale_matrix = np.array(
[[scale_x, 0, 0], [0, scale_y, 0], [0, 0, 1]], dtype=np.float64
)
matrix = scale_matrix @ matrix
width = int(round(width * scale_x))
height = int(round(height * scale_y))
# Apply rotation (about center of current frame)
# Note: SLEAP uses center pixel indexing where (0, 0) is the center of the
# top-left pixel. The geometric center of an image is at ((w-1)/2, (h-1)/2).
if self.rotate is not None and self.rotate != 0:
angle_rad = np.radians(self.rotate)
cos_a = np.cos(angle_rad)
sin_a = np.sin(angle_rad)
cx, cy = (width - 1) / 2, (height - 1) / 2
# Get rotation output size (may expand if not clipping)
new_width, new_height, offset_x, offset_y = self._rotation_output_size(
width, height
)
# Rotation about center, then translate to new center if expanded
# Combined: T(new_cx, new_cy) @ R @ T(-cx, -cy)
# Note: Image coordinates use y-down, so clockwise rotation matrix is:
# [cos, -sin]
# [sin, cos]
new_cx = (new_width - 1) / 2
new_cy = (new_height - 1) / 2
rotate_matrix = np.array(
[
[cos_a, -sin_a, new_cx - cos_a * cx + sin_a * cy],
[sin_a, cos_a, new_cy - sin_a * cx - cos_a * cy],
[0, 0, 1],
],
dtype=np.float64,
)
matrix = rotate_matrix @ matrix
width, height = new_width, new_height
# Apply pad (translate by padding offset)
if self.pad is not None:
top, right, bottom, left = self.pad
pad_matrix = np.array(
[[1, 0, left], [0, 1, top], [0, 0, 1]], dtype=np.float64
)
matrix = pad_matrix @ matrix
width = width + left + right
height = height + top + bottom
# Apply horizontal flip (x -> (width - 1) - x)
# With center pixel indexing, pixel centers range from 0 to width-1,
# so we flip around (width-1)/2 by mapping x -> (width-1) - x.
if self.flip_h:
flip_h_matrix = np.array(
[[-1, 0, width - 1], [0, 1, 0], [0, 0, 1]], dtype=np.float64
)
matrix = flip_h_matrix @ matrix
# Apply vertical flip (y -> (height - 1) - y)
# With center pixel indexing, pixel centers range from 0 to height-1,
# so we flip around (height-1)/2 by mapping y -> (height-1) - y.
if self.flip_v:
flip_v_matrix = np.array(
[[1, 0, 0], [0, -1, height - 1], [0, 0, 1]], dtype=np.float64
)
matrix = flip_v_matrix @ matrix
return matrix
compute_transform_summary(labels, transforms)
¶
Compute a summary of transforms to be applied.
This is useful for dry-run mode to preview what will happen.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
Source Labels object. |
required |
transforms
|
dict[int, Transform] | Transform
|
Either a single Transform or dict mapping video indices. |
required |
Returns:
| Type | Description |
|---|---|
dict
|
Dictionary with transform summary information including warnings for: - Rotation clips >20% of frame area - Landmarks go out of bounds - Output size <32px in any dimension - Non-integer dimensions (rounding applied) - Crop extends outside frame (padding applied) |
Source code in sleap_io/transform/video.py
def compute_transform_summary(
labels: "Labels",
transforms: dict[int, Transform] | Transform,
) -> dict:
"""Compute a summary of transforms to be applied.
This is useful for dry-run mode to preview what will happen.
Args:
labels: Source Labels object.
transforms: Either a single Transform or dict mapping video indices.
Returns:
Dictionary with transform summary information including warnings for:
- Rotation clips >20% of frame area
- Landmarks go out of bounds
- Output size <32px in any dimension
- Non-integer dimensions (rounding applied)
- Crop extends outside frame (padding applied)
"""
import numpy as np
# Normalize transforms to dict
if isinstance(transforms, Transform):
transforms_dict = {i: transforms for i in range(len(labels.videos))}
else:
transforms_dict = transforms
summary = {
"videos": [],
"total_frames": 0,
"total_instances": 0,
"warnings": [],
}
for video_idx, video in enumerate(labels.videos):
transform = transforms_dict.get(video_idx, Transform())
# Get video info
video_info = {
"index": video_idx,
"filename": video.filename,
"has_transform": bool(transform),
}
# Get dimensions
if hasattr(video, "shape") and video.shape is not None:
n_frames, h, w = video.shape[:3]
input_size = (w, h)
video_info["input_size"] = input_size
video_info["n_frames"] = n_frames
summary["total_frames"] += n_frames
else:
video_info["input_size"] = None
video_info["n_frames"] = None
input_size = None
# Compute output size and generate warnings
if transform and input_size:
output_size = transform.output_size(input_size)
video_info["output_size"] = output_size
out_w, out_h = output_size
in_w, in_h = input_size
# Warning: Output size very small (<32px)
if out_w < 32 or out_h < 32:
summary["warnings"].append(
f"Video {video_idx}: Output size very small ({out_w}x{out_h})"
)
# Warning: Crop extends outside frame (padding applied)
if transform.crop:
x1, y1, x2, y2 = transform.crop
if x1 < 0 or y1 < 0 or x2 > in_w or y2 > in_h:
summary["warnings"].append(
f"Video {video_idx}: Crop extends outside frame, "
f"padding will be applied"
)
# Warning: Rotation clips >20% of frame
if transform.rotate and transform.rotate != 0 and transform.clip_rotation:
# Calculate how much area is lost when clipping rotation
angle_rad = np.radians(abs(transform.rotate))
cos_a = abs(np.cos(angle_rad))
sin_a = abs(np.sin(angle_rad))
# Get the size after crop/scale (before rotation)
pre_rotate_w, pre_rotate_h = in_w, in_h
if transform.crop:
x1, y1, x2, y2 = transform.crop
pre_rotate_w = x2 - x1
pre_rotate_h = y2 - y1
if transform.scale:
scale_x, scale_y = transform.scale
pre_rotate_w = int(round(pre_rotate_w * scale_x))
pre_rotate_h = int(round(pre_rotate_h * scale_y))
# Expanded dimensions without clipping
expanded_w = int(np.ceil(pre_rotate_w * cos_a + pre_rotate_h * sin_a))
expanded_h = int(np.ceil(pre_rotate_w * sin_a + pre_rotate_h * cos_a))
# Calculate area loss percentage
original_area = pre_rotate_w * pre_rotate_h
expanded_area = expanded_w * expanded_h
# The clipped content is the original minus what fits in the rotated box
# For a simpler estimate, use the ratio of areas
if expanded_area > original_area:
clip_fraction = 1 - (original_area / expanded_area)
clip_pct = clip_fraction * 100
if clip_pct > 20:
summary["warnings"].append(
f"Video {video_idx}: Rotation will clip ~{clip_pct:.0f}% "
f"of frame area"
)
# Warning: Non-integer dimensions (rounding)
if transform.scale:
scale_x, scale_y = transform.scale
# Get pre-scale dimensions
pre_scale_w, pre_scale_h = in_w, in_h
if transform.crop:
x1, y1, x2, y2 = transform.crop
pre_scale_w = x2 - x1
pre_scale_h = y2 - y1
# Calculate exact dimensions
exact_w = pre_scale_w * scale_x
exact_h = pre_scale_h * scale_y
rounded_w = int(round(exact_w))
rounded_h = int(round(exact_h))
# Check if rounding was needed
if abs(exact_w - rounded_w) > 0.01 or abs(exact_h - rounded_h) > 0.01:
summary["warnings"].append(
f"Video {video_idx}: Dimensions rounded: "
f"{exact_w:.1f}x{exact_h:.1f} -> {rounded_w}x{rounded_h}"
)
# Count instances and check for OOB landmarks
n_instances = 0
n_oob_landmarks = 0
if transform and input_size:
output_size = transform.output_size(input_size)
out_w, out_h = output_size
for lf in labels.labeled_frames:
if lf.video is video:
n_instances += len(lf.instances)
for instance in lf.instances:
points = instance.numpy(invisible_as_nan=False)
if points.size == 0:
continue
# Transform points to output space
transformed = transform.apply_to_points(points, input_size)
# Check for OOB (excluding NaN points)
valid_mask = ~np.isnan(transformed).any(axis=-1)
valid_points = transformed[valid_mask]
if valid_points.size > 0:
oob = (
(valid_points[:, 0] < 0)
| (valid_points[:, 0] >= out_w)
| (valid_points[:, 1] < 0)
| (valid_points[:, 1] >= out_h)
)
n_oob_landmarks += np.sum(oob)
else:
for lf in labels.labeled_frames:
if lf.video is video:
n_instances += len(lf.instances)
video_info["n_instances"] = n_instances
summary["total_instances"] += n_instances
# Warning: Landmarks go out of bounds
if n_oob_landmarks > 0:
summary["warnings"].append(
f"Video {video_idx}: {n_oob_landmarks} landmark(s) will be "
f"outside frame bounds"
)
# Add transform details
if transform:
video_info["transform"] = {
"crop": transform.crop,
"scale": transform.scale,
"rotate": transform.rotate,
"pad": transform.pad,
}
summary["videos"].append(video_info)
return summary
get_out_of_bounds_mask(points, bounds)
¶
Get a boolean mask indicating which points are outside bounds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
points
|
ndarray
|
Coordinate array of shape (n_points, 2) where each row is (x, y). |
required |
bounds
|
tuple[int, int, int, int]
|
Bounds as (x_min, y_min, x_max, y_max). |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
Boolean array of shape (n_points,) where True indicates the point is out of bounds. NaN points are considered in bounds (not marked as OOB). |
Source code in sleap_io/transform/points.py
def get_out_of_bounds_mask(
points: np.ndarray,
bounds: tuple[int, int, int, int],
) -> np.ndarray:
"""Get a boolean mask indicating which points are outside bounds.
Args:
points: Coordinate array of shape (n_points, 2) where each row is (x, y).
bounds: Bounds as (x_min, y_min, x_max, y_max).
Returns:
Boolean array of shape (n_points,) where True indicates the point is
out of bounds. NaN points are considered in bounds (not marked as OOB).
"""
x_min, y_min, x_max, y_max = bounds
# Mask for valid (non-NaN) points
valid_mask = ~np.isnan(points).any(axis=-1)
# Initialize result - NaN points are not considered OOB
oob_mask = np.zeros(len(points), dtype=bool)
if valid_mask.any():
valid_points = points[valid_mask]
oob = (
(valid_points[:, 0] < x_min)
| (valid_points[:, 0] >= x_max)
| (valid_points[:, 1] < y_min)
| (valid_points[:, 1] >= y_max)
)
oob_mask[valid_mask] = oob
return oob_mask
transform_embedded_video(video, output_path, video_idx, transform, image_format='png', plugin=None, progress_callback=None)
¶
Transform an embedded video and write to an HDF5 output file.
This function reads embedded frames, transforms them, and writes them to a new embedded video dataset in the output HDF5 file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
video
|
Video
|
Source video object with embedded images. |
required |
output_path
|
Path
|
Path to the output SLP/HDF5 file (must already exist). |
required |
video_idx
|
int
|
Index of this video in the labels (used for group naming). |
required |
transform
|
Transform
|
Transform to apply to each frame. |
required |
image_format
|
str
|
Image format for encoding ("png" or "jpg"). |
'png'
|
plugin
|
str | None
|
Image plugin to use for encoding ("opencv" or "imageio"). |
None
|
progress_callback
|
Callable[[int, int], None] | None
|
Optional callback called with (current_frame, total_frames). |
None
|
Returns:
| Type | Description |
|---|---|
Video
|
New Video object pointing to the embedded video in the output file. |
Source code in sleap_io/transform/video.py
def transform_embedded_video(
video: "Video",
output_path: Path,
video_idx: int,
transform: Transform,
image_format: str = "png",
plugin: str | None = None,
progress_callback: Callable[[int, int], None] | None = None,
) -> "Video":
"""Transform an embedded video and write to an HDF5 output file.
This function reads embedded frames, transforms them, and writes them to a new
embedded video dataset in the output HDF5 file.
Args:
video: Source video object with embedded images.
output_path: Path to the output SLP/HDF5 file (must already exist).
video_idx: Index of this video in the labels (used for group naming).
transform: Transform to apply to each frame.
image_format: Image format for encoding ("png" or "jpg").
plugin: Image plugin to use for encoding ("opencv" or "imageio").
progress_callback: Optional callback called with (current_frame, total_frames).
Returns:
New Video object pointing to the embedded video in the output file.
"""
import json
import sys
import h5py
import numpy as np
from sleap_io.io.video_reading import VideoBackend, get_default_image_plugin
from sleap_io.model.video import Video
# Determine plugin
if plugin is None:
plugin = get_default_image_plugin()
if plugin is None:
plugin = "opencv" if "cv2" in sys.modules else "imageio"
# Get embedded frame indices
frame_inds = _get_frame_indices(video)
n_frames = len(frame_inds)
# Handle empty videos (no embedded frames)
if n_frames == 0:
# Return a video object that references the same empty structure
from sleap_io.io.video_reading import VideoBackend
source_video = video.source_video if video.source_video is not None else video
return Video(
filename=str(output_path),
backend=video.backend, # Keep same backend reference
source_video=source_video,
)
# Get input dimensions for computing output size
if video.shape is not None:
h, w = video.shape[1:3]
input_size = (w, h)
else:
raise ValueError("Cannot determine dimensions for embedded video")
# Compute output dimensions
out_w, out_h = transform.output_size(input_size)
# Determine number of channels from first frame
first_frame = video[frame_inds[0]]
channels = first_frame.shape[2] if first_frame.ndim == 3 else 1
# Transform and encode all frames
imgs_data = []
for i, frame_idx in enumerate(frame_inds):
frame = video[frame_idx]
# Apply transform
transformed = transform.apply_to_frame(frame)
# Encode frame
if plugin == "opencv":
import cv2
img_data = np.squeeze(
cv2.imencode("." + image_format, transformed)[1]
).astype("int8")
channel_order = "BGR"
else: # imageio
import imageio.v3 as iio
if transformed.shape[-1] == 1:
transformed = transformed.squeeze(axis=-1)
img_data = np.frombuffer(
iio.imwrite("<bytes>", transformed, extension="." + image_format),
dtype="int8",
)
channel_order = "RGB"
imgs_data.append(img_data)
if progress_callback is not None:
progress_callback(i + 1, n_frames)
# Write to HDF5 file
group = f"video{video_idx}"
with h5py.File(output_path, "a") as f:
# Create dataset with fixed-length encoding
img_bytes_len = max(len(img) for img in imgs_data)
ds = f.create_dataset(
f"{group}/video",
shape=(len(imgs_data), img_bytes_len),
dtype="int8",
compression="gzip",
)
for i, img in enumerate(imgs_data):
ds[i, : len(img)] = img
# Store metadata with TRANSFORMED dimensions
ds.attrs["format"] = image_format
ds.attrs["channel_order"] = channel_order
ds.attrs["frames"] = n_frames
ds.attrs["height"] = out_h
ds.attrs["width"] = out_w
ds.attrs["channels"] = channels
# Store FPS if available
if video.fps is not None:
ds.attrs["fps"] = video.fps
# Store frame indices (same as source)
f.create_dataset(f"{group}/frame_numbers", data=frame_inds)
# Store source video reference
source_video = video.source_video if video.source_video is not None else video
grp = f.require_group(f"{group}/source_video")
# Build source video dict
source_dict = {
"backend": {
"filename": source_video.filename,
"grayscale": source_video.grayscale,
}
}
grp.attrs["json"] = json.dumps(source_dict, separators=(",", ":"))
# Create and return the new embedded Video object
embedded_video = Video(
filename=str(output_path),
backend=VideoBackend.from_filename(
str(output_path),
dataset=f"{group}/video",
grayscale=video.grayscale,
keep_open=False,
),
source_video=source_video,
)
return embedded_video
transform_labels(labels, transforms, output_path, video_output_dir=None, fps=None, crf=25, preset='superfast', keyframe_interval=None, no_audio=False, progress_callback=None, dry_run=False)
¶
Transform all videos in a Labels object and update coordinates.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
labels
|
Labels
|
Source Labels object. |
required |
transforms
|
dict[int, Transform] | Transform
|
Either a single Transform to apply to all videos, or a dict mapping video indices to their respective Transforms. |
required |
output_path
|
str | Path
|
Path (or str) to save the transformed SLP file. |
required |
video_output_dir
|
str | Path | None
|
Directory (str or Path) for transformed videos. If None, uses "{output_path.stem}.videos/". Ignored for embedded videos. |
None
|
fps
|
float | None
|
Output frame rate. If None, preserves source FPS. Ignored for embedded. |
None
|
crf
|
int
|
Constant rate factor for video quality (0-51, lower is better). Ignored for embedded videos. |
25
|
preset
|
str
|
x264 encoding preset. Ignored for embedded videos. |
'superfast'
|
keyframe_interval
|
float | None
|
Interval between keyframes in seconds. If None, uses encoder default. Ignored for embedded videos. |
None
|
no_audio
|
bool
|
If True, strips audio from output. Ignored for embedded videos. |
False
|
progress_callback
|
Callable[[str, int, int], None] | None
|
Optional callback called with (video_name, current, total). |
None
|
dry_run
|
bool
|
If True, compute transforms but don't process videos. |
False
|
Returns:
| Type | Description |
|---|---|
Labels
|
New Labels object with transformed videos and adjusted coordinates. |
Notes
For embedded videos (.pkg.slp), the output will also be an embedded file
with transformed frame images. The video_output_dir and video encoding
parameters are ignored for embedded videos.
Source code in sleap_io/transform/video.py
def transform_labels(
labels: "Labels",
transforms: dict[int, Transform] | Transform,
output_path: str | Path,
video_output_dir: str | Path | None = None,
fps: float | None = None,
crf: int = 25,
preset: str = "superfast",
keyframe_interval: float | None = None,
no_audio: bool = False,
progress_callback: Callable[[str, int, int], None] | None = None,
dry_run: bool = False,
) -> "Labels":
"""Transform all videos in a Labels object and update coordinates.
Args:
labels: Source Labels object.
transforms: Either a single Transform to apply to all videos, or a dict
mapping video indices to their respective Transforms.
output_path: Path (or str) to save the transformed SLP file.
video_output_dir: Directory (str or Path) for transformed videos. If None, uses
"{output_path.stem}.videos/". Ignored for embedded videos.
fps: Output frame rate. If None, preserves source FPS. Ignored for embedded.
crf: Constant rate factor for video quality (0-51, lower is better).
Ignored for embedded videos.
preset: x264 encoding preset. Ignored for embedded videos.
keyframe_interval: Interval between keyframes in seconds. If None, uses
encoder default. Ignored for embedded videos.
no_audio: If True, strips audio from output. Ignored for embedded videos.
progress_callback: Optional callback called with (video_name, current, total).
dry_run: If True, compute transforms but don't process videos.
Returns:
New Labels object with transformed videos and adjusted coordinates.
Notes:
For embedded videos (`.pkg.slp`), the output will also be an embedded file
with transformed frame images. The `video_output_dir` and video encoding
parameters are ignored for embedded videos.
"""
from sleap_io.model.video import Video
output_path = Path(output_path)
if video_output_dir is not None:
video_output_dir = Path(video_output_dir)
# Normalize transforms to dict
if isinstance(transforms, Transform):
transforms_dict = {i: transforms for i in range(len(labels.videos))}
else:
transforms_dict = transforms
# Check if any videos are embedded
has_embedded = any(_is_embedded_video(v) for v in labels.videos)
# Setup video output directory (for non-embedded videos)
if video_output_dir is None:
video_output_dir = output_path.with_name(output_path.stem + ".videos")
if not dry_run and not has_embedded:
video_output_dir.mkdir(parents=True, exist_ok=True)
# Create a copy of labels to modify
new_labels = labels.copy()
# Track video replacements
video_map: dict[Video, Video] = {}
# For embedded videos, we need to save labels first to create the HDF5 file
# Then embed transformed frames into it
embedded_videos_to_process: list[tuple[int, "Video", Transform]] = []
# Process each video
for video_idx, video in enumerate(labels.videos):
transform = transforms_dict.get(video_idx, Transform())
# Skip if no transform
if not transform:
continue
# Get video dimensions for coordinate transformation
if hasattr(video, "shape") and video.shape is not None:
h, w = video.shape[1:3]
input_size = (w, h)
else:
# Try to get from first frame
frame_inds = _get_frame_indices(video)
if frame_inds:
try:
frame = video[frame_inds[0]]
h, w = frame.shape[:2]
input_size = (w, h)
except Exception:
raise ValueError(
f"Cannot determine dimensions for video {video_idx}: "
f"{video.filename}"
)
else:
raise ValueError(
f"Cannot determine dimensions for video {video_idx}: "
f"{video.filename}"
)
# Handle embedded vs regular videos differently
is_embedded = _is_embedded_video(video)
if not dry_run:
if is_embedded:
# Queue embedded video for processing after labels are saved
embedded_videos_to_process.append((video_idx, video, transform))
else:
# Regular video - transform to .mp4 file
video_name = Path(video.filename).stem
output_video_path = video_output_dir / f"{video_name}.transformed.mp4"
def _progress(current: int, total: int) -> None:
if progress_callback is not None:
progress_callback(video_name, current, total)
transform_video(
video=video,
output_path=output_video_path,
transform=transform,
fps=fps,
crf=crf,
preset=preset,
keyframe_interval=keyframe_interval,
no_audio=no_audio,
progress_callback=_progress,
)
# Create new video object
new_video = Video.from_filename(
output_video_path.as_posix(), grayscale=video.grayscale
)
new_video.source_video = video
video_map[new_labels.videos[video_idx]] = new_video
# Update coordinates for this video's labeled frames
copied_video = new_labels.videos[video_idx]
# Compute output bounds for OOB visibility check
output_w, output_h = transform.output_size(input_size)
output_bounds = (0, 0, output_w, output_h)
for lf in new_labels.labeled_frames:
if lf.video is not copied_video:
continue
for instance in lf.instances:
points = instance.numpy(invisible_as_nan=False)
transformed_points = transform.apply_to_points(points, input_size)
instance.points["xy"] = transformed_points
# Mark out-of-bounds points as not visible
oob_mask = get_out_of_bounds_mask(transformed_points, output_bounds)
instance.points["visible"][oob_mask] = False
# Replace video references for regular videos
if video_map:
new_labels.replace_videos(video_map=video_map)
# Process embedded videos after labels structure is ready
# For embedded videos, we need to:
# 1. Save the labels first (creates HDF5 structure)
# 2. Transform and embed frames into the saved file
# 3. Update video references in the returned labels
if embedded_videos_to_process and not dry_run:
# Save labels first to create the HDF5 file
new_labels.save(str(output_path))
# Now transform and embed each video
embedded_video_map: dict[Video, Video] = {}
for video_idx, video, transform in embedded_videos_to_process:
video_name = Path(video.filename).stem
def _progress(current: int, total: int) -> None:
if progress_callback is not None:
progress_callback(video_name, current, total)
new_embedded_video = transform_embedded_video(
video=video,
output_path=output_path,
video_idx=video_idx,
transform=transform,
progress_callback=_progress,
)
embedded_video_map[new_labels.videos[video_idx]] = new_embedded_video
# Replace video references for embedded videos
if embedded_video_map:
new_labels.replace_videos(video_map=embedded_video_map)
# Update videos_json in the HDF5 file to point to the embedded data
# We can't re-save because that would overwrite the embedded video data
_update_videos_json(output_path, new_labels.videos)
return new_labels
transform_video(video, output_path, transform, fps=None, crf=25, preset='superfast', keyframe_interval=None, no_audio=False, progress_callback=None)
¶
Transform a video file and save to a new path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
video
|
Video
|
Source video object. |
required |
output_path
|
str | Path
|
Path (or str) to save the transformed video. |
required |
transform
|
Transform
|
Transform to apply to each frame. |
required |
fps
|
float | None
|
Output frame rate. If None, uses source FPS. |
None
|
crf
|
int
|
Constant rate factor for video quality (0-51, lower is better). |
25
|
preset
|
str
|
x264 encoding preset. |
'superfast'
|
keyframe_interval
|
float | None
|
Interval between keyframes in seconds. If None, uses encoder default. |
None
|
no_audio
|
bool
|
If True, strips audio from output. |
False
|
progress_callback
|
Callable[[int, int], None] | None
|
Optional callback called with (current_frame, total_frames). |
None
|
Returns:
| Type | Description |
|---|---|
Path
|
Path to the output video file. |
Notes
For embedded HDF5 videos, use transform_embedded_video() instead.
Source code in sleap_io/transform/video.py
def transform_video(
video: "Video",
output_path: str | Path,
transform: Transform,
fps: float | None = None,
crf: int = 25,
preset: str = "superfast",
keyframe_interval: float | None = None,
no_audio: bool = False,
progress_callback: Callable[[int, int], None] | None = None,
) -> Path:
"""Transform a video file and save to a new path.
Args:
video: Source video object.
output_path: Path (or str) to save the transformed video.
transform: Transform to apply to each frame.
fps: Output frame rate. If None, uses source FPS.
crf: Constant rate factor for video quality (0-51, lower is better).
preset: x264 encoding preset.
keyframe_interval: Interval between keyframes in seconds. If None, uses
encoder default.
no_audio: If True, strips audio from output.
progress_callback: Optional callback called with (current_frame, total_frames).
Returns:
Path to the output video file.
Notes:
For embedded HDF5 videos, use `transform_embedded_video()` instead.
"""
from sleap_io.io.video_writing import VideoWriter
output_path = Path(output_path)
# Get frame indices to iterate over
frame_inds = _get_frame_indices(video)
n_frames = len(frame_inds)
if fps is None:
if hasattr(video, "backend") and video.backend is not None:
try:
fps = video.backend.fps
except Exception:
fps = 30.0
else:
fps = 30.0
# Create output directory
output_path.parent.mkdir(parents=True, exist_ok=True)
# Write transformed video
with VideoWriter(
filename=output_path,
fps=fps,
crf=crf,
preset=preset,
keyframe_interval=keyframe_interval,
no_audio=no_audio,
) as writer:
for i, frame_idx in enumerate(frame_inds):
# Read frame
frame = video[frame_idx]
if frame is None:
continue
# Apply transform
transformed_frame = transform.apply_to_frame(frame)
# Write frame
writer(transformed_frame)
# Progress callback
if progress_callback is not None:
progress_callback(i + 1, n_frames)
return output_path