For a more complete documentation, go visit the official website.
[[TOC]]
A library to share streams of framed data between processes via shared memory. It supports any kind of data stream: it has been used with multichannel audio, video frames, 3D models, OSC messages, and various others types of data. Shmdata is server-less, but requires applications to link data streams using socket path (e.g. "/tmp/my-shmdata-stream"). Shmdata is very fast and allows processes to access data streams without the need for extra copies.
The communication paradigm is 1 to many, i.e., one writer is making available data frames to several followers. Followers and writers can hot connect & disconnect. Shmdata transmission supports buffer resizing. Each shmdata has a type specified with a string description, itself published by the shmdata writer at each reconnection. The type is specified as a user-defined string. Although left to the user, we encourage type specification to follow GStreamer 1.0 caps specification format, for instance "audio/x-raw,format=S16LE,channels=2,layout=interleaved".
Note the existence of NDI2shmdata that converts shmdata to NewTek's NDI, and vice versa.
Shmdata runs on Linux and should works on Unix-like operating systems that supports System V shared memory, System V semaphore and Unix sockets, such as OSX.
Shmdata is intended to be used as an extension to applications in order to enable shmdata reading and/or writing, thanks to one of the shdmata APIs. Currently, the following languages are available for implementation specific tools or to add shmdata support in your software: C/C++, Python and GStreamer.
On ubuntu 20.04 (amd64), shmdata can be installed as follows:
sudo apt install -y coreutils wget && \
wget -qO - https://sat-mtl.gitlab.io/distribution/mpa-focal-amd64-nvidia/sat-metalab-mpa-keyring.gpg \
| gpg --dearmor \
| sudo dd of=/usr/share/keyrings/sat-metalab-mpa-keyring.gpg && \
echo 'deb [ arch=amd64, signed-by=/usr/share/keyrings/sat-metalab-mpa-keyring.gpg ] https://sat-mtl.gitlab.io/distribution/mpa-focal-amd64-nvidia/debs/ sat-metalab main' \
| sudo tee /etc/apt/sources.list.d/sat-metalab-mpa.list && \
echo 'deb [ arch=amd64, signed-by=/usr/share/keyrings/sat-metalab-mpa-keyring.gpg ] https://sat-mtl.gitlab.io/distribution/mpa-datasets/debs/ sat-metalab main' \
| sudo tee /etc/apt/sources.list.d/sat-metalab-mpa-datasets.list && \
sudo apt-get update
sudo apt install libshmdata
Otherwise, you can install shmdata from sources
The following command sequence will let you check your shmdata installation. It describes how to i) generate a video test signal into a shmdata, ii) monitor it, and iii) read the video shmdata for display. All these steps are performed by separate commands running on the same computer.
Create a shmdata in which raw video will be sent. The path to the shmdata will be /tmp/video_shmdata.
# generate a video test signal into a shmdata
gst-launch-1.0 --gst-plugin-path=/usr/lib/gstreamer-1.0/ videotestsrc ! shmdatasink socket-path=/tmp/video_shmdata
Note: The command gst-launch allows for running GStreamer pipeline from command line. Here the pipeline is composed of two elements: videotestsrc that transmits its video stream to the shmdatasink element. The parameter socket-path indicates to the shmdatasink where the shmdata must be located. Finally, the --gst-plugin-path option tells gst-launch where it can find the shmdata GStreamer elements (by default, shmdata GStreamer plugins are installed in /usr/lib/gstreamer-1.0/. If you built shmdata from source, you will have to change this path to /usr/local/lib/gstreamer-1.0).
The sdflow utility is installed along with the shmdata library. It prints the shmdata metadata once connected with the shmdata writer, and then a line of information for each buffer pushed by the shmdata writer. Keep the video shmdata running, and then from a new terminal type:
# monitor a shmdata type and frame sizes
$ sdflow /tmp/video_shmdata
connected: type video/x-raw, format=(string)I420, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
0 size: 115200 data: EBEBEBEBEBEBEBEBEBEBEBEBEBEBEB...
1 size: 115200 data: EBEBEBEBEBEBEBEBEBEBEBEBEBEBEB...
etc
Note that you can monitor a shmadata framerate using pv and sdflow.
With the video transmission still running (and optionally, the sdflow monitoring), open a new terminal window and display the video using the following command:
# read the video shmdata and display its content into a window
gst-launch-1.0 --gst-plugin-path=/usr/lib/gstreamer-1.0/ shmdatasrc socket-path=/tmp/video_shmdata ! xvimagesink
Xvimagesink does not work in some remote server scenarios, however aasink provides a way to monitor a video feed using only ascii, even on an headless server instance via ssh.:
# read the video shmdata and display its content into a terminal
gst-launch-1.0 --gst-plugin-path=/usr/lib/gstreamer-1.0/ shmdatasrc socket-path=/tmp/video_shmdata ! aasink
In previous section, GStreamer is used in order to illustrate a transmission, but you can also include shmdata in you project with specific languages, using native shmdata API. Here follows some examples in various languages:
Note that in order to develop with shmdata C or C++ API, you may want to install development files:
sudo apt install libshmdata-dev
Then C and C++ using shmdata can be compiled with help from pkgconfig, as follows:
# C code:
gcc -o check-c-wrapper $(pkg-config --cflags shmdata-1.3) ./check-c-wrapper.cpp $(pkg-config --libs shmdata-1.3)
# C++ code
g++ -o check-writer-follower $(pkg-config --cflags shmdata-1.3) ./check-writer-follower.cpp $(pkg-config --libs shmdata-1.3)To contribute to shmdata, see the contribution guide
This project is made possible thanks to the Society for Arts and Technology. SAT and to the Ministère de l'Économie et de l'Innovation du Québec (MEI).
Shmdata is maintained by the Metalab, the Society for Arts and Technology [SAT] research laboratory. Our mission is to provide artists and creators with a powerful ecosystem of immersion and telepresence software for live arts and new media installations.
Shmdata is also used in the Scènes ouvertes project: a network of more than 20 venues from all across the province of Quebec collaborating through artistic telepresence installations .
See a list of shmdata authors here.