API¶

Important

Python API’s are designed to minimize differences in C++ and Python; i.e. all classes and methods should have the same name; function signatures should also be same as possible. For the slight differences, see below in details.

All functionality in pylibfreenect2.libfreenect2 is directly accesible from the top-level pylibfreenect2.* namespace.

The sections below are organized by following the offical docs.

Frame Listeners¶

FrameType¶

class pylibfreenect2.FrameType[source]¶

Python-side enum for libfreenect::Frame::Type in C++.

The value can be Color, Ir or Depth.

Warning

The name is slightly different between Python and C++ ( Frame::Type -> FrameType).

Examples

Suppose the following C++ code:

libfreenect2::Frame *rgb = frames[libfreenect2::Frame::Color];
libfreenect2::Frame *ir = frames[libfreenect2::Frame::Ir];
libfreenect2::Frame *depth = frames[libfreenect2::Frame::Depth];

This can be translated in Python like:

rgb = frames[pylibfreenect2.FrameType.Color]
ir = frames[pylibfreenect2.FrameType.Ir]
depth = frames[pylibfreenect2.FrameType.Depth]

or you can use str key:

rgb = frames["color"]
ir = frames["ir"]
depth = frames["depth"]

Frame¶

class pylibfreenect2.libfreenect2.Frame¶

Python interface for libfreenect2::Frame.

The Frame is a container of the C++ pointer libfreenect2::Frame*.

Note

By default, Frame just keeps a pointer of libfreenect2::Frame that should be allocated and released by SyncMultiFrameListener (i.e. Frame itself doesn’t own the allocated memory) as in C++. However, if Frame is created by providing width, height and bytes_per_pixel, then it allocates necessary memory in __cinit__ and release it in __dealloc__ method.

Parameters:

width : int, optional: Width of Frame. Default is None.
height : int, optional: Height of Frame. Default is None.
bytes_per_pixel : int, optional: Bytes per pixels of Frame. Default is None.
frame_type : int, optional: Underlying frame type. Default is -1. Used by asarray method.
numpy_array : numpy.ndarray, optional: Numpy array of depth or ir data with ndim=2, that will be converted to a frame class. Default is None.

See also

pylibfreenect2.FrameType

Attributes:

ptr : libfreenect2::Frame*: Pointer of Frame.
take_ownership : bool: If True, the class instance allocates memory for Frame* and release it in __dealloc__. If width, height and bytes_per_pixel are given in __cinit__, which is necessary to allocate how much memory we need, take_ownership is set to True internally, otherwise False. Note that the value itself cannot be set by users.
frame_type : int: Underlying frame type.

asarray()¶

Frame to numpy.ndarray conversion

Internal data of Frame can be represented as:

3d array of numpy.uint8 for color
2d array of numpy.float32 for IR and depth

Parameters:	dtype : numpy dtype, optional Data type (`numpy.uint8` or `numpy.float32`). If None, data type is automatically selected if possible. Default is None.
Returns:	array : `numpy.ndarray`, shape: `(height, width)` for IR and depth, ``(4, height, width)`` for Color. Array of internal frame.
Raises:	ValueError If dtype is None and underlying frame type cannot be determined. If dtype neither `numpy.uint8` nor `numpy.float32` is specified

Examples

rgb_array = frames["color"].asarray()
ir_array = frames["ir"].asarray()
depth_array = frames["depth"].asarray()

undistorted = Frame(512, 424, 4)
registered = Frame(512, 424, 4)
undistorted_arrray = undistorted.asarray(dtype=np.float32)
registered_array = registered.asarray(dtype=np.uint8)

bytes_per_pixel¶: Same as libfreenect2::Frame::bytes_per_pixel

exposure¶: Same as libfreenect2::Frame::exposure

gain¶: Same as libfreenect2::Frame::gain

gamma¶: Same as libfreenect2::Frame::gamma

height¶: Same as libfreenect2::Frame::height

sequence¶: Same as libfreenect2::Frame::sequence

timestamp¶: Same as libfreenect2::Frame::timestamp

width¶: Same as libfreenect2::Frame::width

FrameMap¶

class pylibfreenect2.libfreenect2.FrameMap¶

Python interface for libfreenect2::FrameMap.

The FrameMap is a container of C++ value libfreenect2::FrameMap (aliased to std::map<libfreenect2::Frame::Type,libfreenect2::Frame*> in C++).

Note

By default, FrameMap just keeps a reference of libfreenect2::FrameMap that should be allcoated and released by SyncMultiFrameListener (i.e. FrameMap itself doesn’t own the allocated memory) as in C++.

Attributes:	internal_frame_map : std::map<libfreenect2::Frame::Type, libfreenect2::Frame*> Internal FrameMap.

__getitem__¶

Get access to the internal FrameMap.

This allows the following dict-like syntax:

color = frames[pylibfreenect2.FrameType.Color]

color = frames['color']

color = frames[1] # with IntEnum value

The key can be of FrameType (a subclass of IntEnum), str or int type as shown above.

Parameters:	key : `FrameType`, str or int Key for the internal FrameMap. available str keys are `color`, `ir` and `depth`.
Returns:	frame : Frame Frame for the specified key.
Raises:	KeyError if unknown key is specified

See also

pylibfreenect2.FrameType

SyncMultiFrameListener¶

class pylibfreenect2.libfreenect2.SyncMultiFrameListener¶

Python interface for libfreenect2::SyncMultiFrameListener.

The SyncMultiFrameListener is a container of C++ pointer libfreenect2::SyncMultiFrameListener*. The pointer of SyncMultiFrameListener is allocated in __cinit__ and released in __dealloc__ method.

Parameters:	frame_types : unsigned int, optional Frame types that we want to listen. It can be logical OR of: `FrameType.Color` `FrameType.Ir` `FrameType.Depth` Default is `FrameType.Color \| FrameType.Ir \| FrameType.Depth`

See also

pylibfreenect2.FrameType

Attributes:	ptr : libfreenect2.SyncMultiFrameListener* Pointer of `libfreenect2::SyncMultiFrameListener` listener_ptr_alias : libfreenect2.FrameListener* Pointer of `libfreenect2::FrameListener`. This is necessary to call methods that operate on `libfreenect2::FrameListener*`, not `libfreenect2::SyncMultiFrameListener`.

hasNewFrame()¶

Same as libfreenect2::SyncMultiFrameListener::hasNewFrame().

Returns:	r : Bool True if SyncMultiFrameListener has a new frame, False otherwise.

release()¶

Same as libfreenect2::SyncMultiFrameListener::release(Frame&).

Parameters:	frame_map : FrameMap FrameMap.

waitForNewFrame()¶

Same as libfreenect2::SyncMultiFrameListener::waitForNewFrame(Frame&).

Warning

Function signature can be different between Python and C++.

Parameters:	frame_map : FrameMap, optional If not None, SyncMultiFrameListener write to it inplace, otherwise a new FrameMap is allocated within the function and then returned. milliseconds : int, optional If >= 0, a timeout is set for getting the new frame. In case the new frame did not arrive beore the timeout, None will be returned.
Returns:	frame_map : FrameMap FrameMap, or None if timeout exceeded. Note FrameMap must be releaseed by call-side by calling `release` function.

Examples

Suppose the following C++ code:

libfreenect2::FrameMap frames;
listener->waitForNewFrame(frames);

This can be translated in Python as follows:

frames = listener.waitForNewFrame()

or you can write it more similar to C++:

frames = pylibfreenect2.FrameMap()
listener.waitForNewFrame(frames)

Example with millisecond timeout:

frames = listener.waitForNewFrame(milliseconds=50)
if frames:
    # frames should be set to a value.
else:
    # Timeout happened.

Initialization and Device Control¶

Freenect2Device¶

class pylibfreenect2.libfreenect2.Freenect2Device¶

Python interface for libfreenect2::Freenect2Device.

The Freenect2Device is a container of C++ pointer libfreenect2::Freenect2Device*.

Note

Freenect2Device just keeps a pointer of libfreenect2::Freenect2Device that should be allocated and released by Freenect2. Freenect2Device itself doesn’t own the memory.

A valid device can be created by openDefaultDevice:

fn = Freenect2()
assert fn.enumerateDevices() > 0
device = fn.openDefaultDevice()

or by openDevice:

fn = Freenect2()
assert fn.enumerateDevices() > 0
serial = fn.getDeviceSerialNumber(0)
device = fn.openDevice(serial)

Attributes:	ptr : `libfreenect2::Freenect2Device*`

close()¶: Same as libfreenect2::Freenect2Device::close()

getColorCameraParams()¶: Same as libfreenect2::Freenect2Device::getColorCameraParams()

getFirmwareVersion()¶: Same as libfreenect2::Freenect2Device::getFirmwareVersion()

getIrCameraParams()¶: Same as libfreenect2::Freenect2Device::getIrCameraParams()

getSerialNumber()¶: Same as libfreenect2::Freenect2Device::getSerialNumber()

setColorFrameListener()¶: Same as libfreenect2::Freenect2Device::setColorFrameListener(FrameListener*)

setIrAndDepthFrameListener()¶: Same as libfreenect2::Freenect2Device::setIrAndDepthFrameListener(FrameListener*)

start()¶: Same as libfreenect2::Freenect2Device::start()

startStreams()¶: Same as libfreenect2::Freenect2Device::startStreams(bool, bool)

stop()¶: Same as libfreenect2::Freenect2Device::stop()

Freenect2¶

class pylibfreenect2.libfreenect2.Freenect2¶

Python interface for libfreenect2::Freenect2.

The Freenect2 is a container of C++ pointer libfreenect2::Freenect2*. The pointer of Freenect2 is allocated in __cinit__ and released in __dealloc__ method.

Attributes:	ptr : `libfreenect2::Freenect2*`

enumerateDevices()¶: Same as libfreenect2::Freenect2::enumerateDevices()

getDefaultDeviceSerialNumber()¶: Same as libfreenect2::Freenect2::getDefaultDeviceSerialNumber()

getDeviceSerialNumber()¶: Same as libfreenect2::Freenect2::getDeviceSerialNumber(int)

openDefaultDevice()¶

Open the first device

Parameters:	pipeline : PacketPipeline, optional Pipeline. Default is None.
Returns:	device : Freenect2Device

openDevice()¶

Open device by serial number or index

Parameters:	name : int or str Serial number (str) or device index (int) pipeline : PacketPipeline, optional Pipeline. Default is None.
Raises:	ValueError If invalid name is specified.

ColorCameraParams¶

class pylibfreenect2.libfreenect2.ColorCameraParams¶

Python interface for libfreenect2::Freenect2Device::ColorCameraParams.

Attributes:	params : `libfreenect2::Freenect2Device::ColorCameraParams`

cx¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::cx

cy¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::cy

fx¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::fx

fy¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::fy

mx_x0y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x0y0

mx_x0y1¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x0y1

mx_x0y2¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x0y2

mx_x0y3¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x0y3

mx_x1y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x1y0

mx_x1y1¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x1y1

mx_x1y2¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x1y2

mx_x2y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x2y0

mx_x2y1¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x2y1

mx_x3y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::mx_x3y0

my_x0y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x0y0

my_x0y1¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x0y1

my_x0y2¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x0y2

my_x0y3¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x0y3

my_x1y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x1y0

my_x1y1¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x1y1

my_x1y2¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x1y2

my_x2y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x2y0

my_x2y1¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x2y1

my_x3y0¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::my_x3y0

shift_d¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::shift_d

shift_m¶: Same as libfreenect2::Freenect2Device::ColorCameraParams::shift_m

IrCameraParams¶

class pylibfreenect2.libfreenect2.IrCameraParams¶

Python interface for libfreenect2::IrCameraParams.

Attributes:	params : `libfreenect2::Freenect2Device::IrCameraParams`

cx¶: Same as libfreenect2::Freenect2Device::IrCameraParams::cx

cy¶: Same as libfreenect2::Freenect2Device::IrCameraParams::cy

fx¶: Same as libfreenect2::Freenect2Device::IrCameraParams::fx

fy¶: Same as libfreenect2::Freenect2Device::IrCameraParams::fy

k1¶: Same as libfreenect2::Freenect2Device::IrCameraParams::k1

k2¶: Same as libfreenect2::Freenect2Device::IrCameraParams::k2

k3¶: Same as libfreenect2::Freenect2Device::IrCameraParams::k3

p1¶: Same as libfreenect2::Freenect2Device::IrCameraParams::p1

p2¶: Same as libfreenect2::Freenect2Device::IrCameraParams::p2

Logging utilities¶

LoggerLevel¶

class pylibfreenect2.LoggerLevel[source]¶

Python-side enum for libfreenect::Logger::Level in C++.

Warning

The name is slightly different between Python and C++ ( Logger::Level -> LoggerLevel).

Examples

Suppose the following C++ code:

libfreenect2::Logger* logger = libfreenect2::createConsoleLogger(
    libfreenect2::Logger::Level::Debug);

This can be translated in Python like:

logger = pylibfreenect2.createConsoleLogger(
    pylibfreenect2.LoggerLevel.Debug)

Logger¶

class pylibfreenect2.libfreenect2.Logger¶

Python interface for libfreenect2::Logger

The Logger is a container of C++ pointer libfreenect2::Logger*.

Attributes:	ptr : libfreenect2::Logger* C++ pointer of Logger

level()¶: Same as Level level()

log()¶: Same as void log(Level level, const std::string &messagge)

Functions¶

`createConsoleLogger`	Same as `Logger* libfreenect2::createConsoleLogger(Level level)`
`createConsoleLoggerWithDefaultLevel`	Same as `Logger* libfreenect2::createConsoleLoggerWithDefaultLevel()`
`getGlobalLogger`	Same as `Logger* libfreenect2::getGlobalLogger()`
`setGlobalLogger`	Same as `void libfreenect2::getGlobalLogger(Logger*)`

Packet Pipelines¶

PacketPipeline¶

class pylibfreenect2.libfreenect2.PacketPipeline¶

Base class for other pipeline classes.

Attributes:	pipeline_ptr_alias : `libfreenect2::PacketPipeline` owened_by_device* : bool

CpuPacketPipeline¶

class pylibfreenect2.libfreenect2.CpuPacketPipeline¶

Pipeline with CPU depth processing.

Attributes:	pipeline : libfreenect2::CpuPacketPipeline*

OpenCLPacketPipeline¶

class pylibfreenect2.libfreenect2.OpenCLPacketPipeline¶

Pipeline with OpenCL depth processing.

Parameters:	device_id : int, optional Device id. Default is -1.
Attributes:	pipeline : libfreenect2::OpenCLPacketPipeline*

OpenCLKdePacketPipeline¶

class pylibfreenect2.libfreenect2.OpenCLKdePacketPipeline¶

Depth packet processing pipeline using KDE depth unwrapping algorithms

Parameters:	device_id : int, optional Device id. Default is -1.

References

https://github.com/OpenKinect/libfreenect2/pull/742

Attributes:	pipeline : libfreenect2::OpenCLKdePacketPipeline*

OpenGLPacketPipeline¶

class pylibfreenect2.libfreenect2.OpenGLPacketPipeline¶

Pipeline with OpenGL depth processing.

Parameters:	debug : bool, optional Debugging mode. Default is False.
Attributes:	pipeline : libfreenect2::OpenGLPacketPipeline*

CudaPacketPipeline¶

class pylibfreenect2.libfreenect2.CudaPacketPipeline¶

Pipeline with Cuda depth processing.

Parameters:	device_id : int, optional Device id. Default is -1.
Attributes:	pipeline : libfreenect2::CudaPacketPipeline*

Registration and Geometry¶

Registration¶

class pylibfreenect2.libfreenect2.Registration¶

Python interface for libfreenect2::Registration.

The Registration is a container of C++ pointer libfreenect2::Registration*. The pointer of Registration is allocated in __cinit__ and released in __dealloc__ method.

Parameters:	irparams : IrCameraParams IR camera parameters. cparams : ColorCameraParams Color camera parameters.

Attributes:	ptr : `libfreenect2::Registration*`

apply()¶

Same as libfreenect2::Registration::apply.

Parameters:

rgb : Frame: (1920, 1080) color frame
depth : Frame: (512, 424) depth frame
undistorted : Frame: (512, 424) registered depth frame
registered : Frame: (512, 424) registered color frame
enable_filter : Bool, optional
bigdepth : Frame, optional: (1920, 1082) bigdepth frame
color_depth_map : numpy.ndarray, optional: Array of shape: (424*512,), dtype np.int32

Raises:

ValueError: If invalid shape of frame/array is provided

getPointXYZ()¶

Same as libfreenect2::Registration::getPointXYZ.

Parameters:	undistorted : Frame `(512, 424)` Undistorted depth frame r : int Row (y) index in depth image c : int Column (x) index in depth image.
Returns:	tuple : (X coordinate of the 3-D point (meter), Y coordinate of the 3-D point (meter), Z coordinate of the 3-D point (meter))

getPointXYZRGB()¶

Same as libfreenect2::Registration::getPointXYZRGB.

Parameters:	undistorted : Frame `(512, 424)` Undistorted depth frame registered : Frame `(512, 424)` Registered color frame r : int Row (y) index in depth image c : int Column (x) index in depth image.
Returns:	tuple : (X coordinate of the 3-D point (meter), Y coordinate of the 3-D point (meter), Z coordinate of the 3-D point (meter), B, G, R)

undistortDepth()¶

Same as libfreenect2::Registration::undistortDepth(bool, bool).

Parameters:	depth : Frame `(512, 424)` depth frame undistorted : Frame `(512, 424)` registered depth frame
Raises:	ValueError If invalid shape of frame/array is provided