Introduction

For our end of degree project in computer engineering at the University of Sherbrooke, the RendezVoUS team is happy to present you the next generation video-conference system : Steno!

Description

No more video-conferences filled with technical problems and meetings with multiple people behind the same laptop!

Steno is an intelligent video-conferencing system that allows you to make worry-free calls by using 16 microphones and a 360° camera. The system use artificial intelligence as well as other technologies to offer an enhanced experience both visually and audibly.

Steno film and record each person individually, then sends everything to the video-conferencing system. It's like if everyone had an webcam! Steno films you dynamically, which means that if you move around the table, it will be able track your voice and position, and make sure all your words and mouvements are sent to other end of the video-conference.

Specifications

Hardware

• Platform : Jetson TX2

• Sound card : 16SoundsUSB

• Microphones (x16) : xSoundsMicrophones

• Audio amplifiers (x2) : Stereo 20W Class D Audio Amplifier - MAX9744

• Speakers (x4) : Yeeco 2.5 inch 4 ohm stero audio speakers

• Camera : See3CAM_CU135 – 4K Custom Lens USB 3.0 Camera Board (Color)

• Lens : Cvivid Lenses 1.2mm Wide Angle 220 Degree Fisheye

Software

• Operating system : JetPack 4.2.2

• Sound source localisation, tracking, separation and post-filtering : ODAS

• Neural network framework : Darknet

• Real-time person detection : YOLO

• Echo and noise cancelling : Pulse Audio

• Virtual devices : V4l2

• WebRTC video-conference web server : Jitsi Meet

• Graphical user interfaces : Qt

The Prototype

Build and Run

See the wiki for more.

Licensing

GNU General Public License v3.0; see LICENSE for details.