Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve frametiming for linux capture #2333

Merged
merged 5 commits into from
Apr 12, 2024
Merged

Improve frametiming for linux capture #2333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d5d6bf6
Fix frametiming for kmsgrab and x11grab
gschintgen Mar 30, 2024
e696a7b
Fix frametiming for NvFCB (Linux)
gschintgen Apr 1, 2024
d11cc14
kmsgrab: add sleep overshoot debugging
gschintgen Apr 4, 2024
9a1d365
Linux: simplify sleep before next frame snapshot
gschintgen Apr 4, 2024
5737f0d
Capture/Linux: add sleep overshoot logging
gschintgen Apr 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/platform/common.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@
#include <mutex>
#include <string>

#include "src/config.h"
#include "src/logging.h"
#include "src/stat_trackers.h"
#include "src/thread_safe.h"
#include "src/utility.h"
#include "src/video_colorspace.h"
Expand All @@ -19,6 +21,8 @@
#include <moonlight-common-c/src/Limelight.h>
}

using namespace std::literals;

struct sockaddr;
struct AVFrame;
struct AVBufferRef;
Expand Down Expand Up @@ -499,6 +503,22 @@
int env_width, env_height;

int width, height;

protected:
// collect capture timing data (at loglevel debug)
stat_trackers::min_max_avg_tracker<double> sleep_overshoot_tracker;
void
log_sleep_overshoot(std::chrono::nanoseconds overshoot_ns) {

Check warning on line 511 in src/platform/common.h

View check run for this annotation

Codecov / codecov/patch

src/platform/common.h#L511 

Added line #L511 was not covered by tests
if (config::sunshine.min_log_level <= 1) {
// Print sleep overshoot stats to debug log every 20 seconds
auto print_info = [&](double min_overshoot, double max_overshoot, double avg_overshoot) {
auto f = stat_trackers::one_digit_after_decimal();

Check warning on line 515 in src/platform/common.h

View check run for this annotation

Codecov / codecov/patch

src/platform/common.h#L514-L515 

Added lines #L514 - L515 were not covered by tests
BOOST_LOG(debug) << "Sleep overshoot (min/max/avg): " << f % min_overshoot << "ms/" << f % max_overshoot << "ms/" << f % avg_overshoot << "ms";
};

Check warning on line 517 in src/platform/common.h

View check run for this annotation

Codecov / codecov/patch

src/platform/common.h#L517 

Added line #L517 was not covered by tests
// std::chrono::nanoseconds overshoot_ns = std::chrono::steady_clock::now() - next_frame;
sleep_overshoot_tracker.collect_and_callback_on_interval(overshoot_ns.count() / 1000000., print_info, 20s);
}
}

Check warning on line 521 in src/platform/common.h

View check run for this annotation

Codecov / codecov/patch

src/platform/common.h#L521 

Added line #L521 was not covered by tests
};

class mic_t {
Expand Down
15 changes: 10 additions & 5 deletions src/platform/linux/cuda.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -800,16 +800,21 @@
handle.reset();
});

sleep_overshoot_tracker.reset();

Check warning on line 803 in src/platform/linux/cuda.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/cuda.cpp#L803 

Added line #L803 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();
if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);
}
while (next_frame > now) {
std::this_thread::sleep_for(1ns);
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 150ms, *cursor);
Expand Down
30 changes: 20 additions & 10 deletions src/platform/linux/kmsgrab.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1193,17 +1193,22 @@
capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) override {
auto next_frame = std::chrono::steady_clock::now();

sleep_overshoot_tracker.reset();

Check warning on line 1196 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1196 

Added line #L1196 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();

if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);

Check warning on line 1202 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1202 

Added line #L1202 was not covered by tests
}
while (next_frame > now) {
std::this_thread::sleep_for(1ns);
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();

Check warning on line 1204 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1204 

Added line #L1204 was not covered by tests
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

Check warning on line 1206 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1206 

Added line #L1206 was not covered by tests

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 1000ms, *cursor);
Expand Down Expand Up @@ -1412,17 +1417,22 @@
capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) {
auto next_frame = std::chrono::steady_clock::now();

sleep_overshoot_tracker.reset();

Check warning on line 1420 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1420 

Added line #L1420 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();

if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);

Check warning on line 1426 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1426 

Added line #L1426 was not covered by tests
}
while (next_frame > now) {
std::this_thread::sleep_for(1ns);
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();

Check warning on line 1428 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1428 

Added line #L1428 was not covered by tests
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

Check warning on line 1430 in src/platform/linux/kmsgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/kmsgrab.cpp#L1430 

Added line #L1430 was not covered by tests

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 1000ms, *cursor);
Expand Down
28 changes: 20 additions & 8 deletions src/platform/linux/wlgrab.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,16 +129,22 @@
capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) override {
auto next_frame = std::chrono::steady_clock::now();

sleep_overshoot_tracker.reset();

Check warning on line 132 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L132 

Added line #L132 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();

if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);

Check warning on line 138 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L138 

Added line #L138 was not covered by tests
}
while (next_frame > now) {
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();

Check warning on line 140 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L140 

Added line #L140 was not covered by tests
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

Check warning on line 142 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L142 

Added line #L142 was not covered by tests

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 1000ms, *cursor);
Expand Down Expand Up @@ -259,16 +265,22 @@
capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) override {
auto next_frame = std::chrono::steady_clock::now();

sleep_overshoot_tracker.reset();

Check warning on line 268 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L268 

Added line #L268 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();

if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);

Check warning on line 274 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L274 

Added line #L274 was not covered by tests
}
while (next_frame > now) {
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();

Check warning on line 276 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L276 

Added line #L276 was not covered by tests
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

Check warning on line 278 in src/platform/linux/wlgrab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/wlgrab.cpp#L278 

Added line #L278 was not covered by tests

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 1000ms, *cursor);
Expand Down
30 changes: 20 additions & 10 deletions src/platform/linux/x11grab.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -481,17 +481,22 @@
capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) override {
auto next_frame = std::chrono::steady_clock::now();

sleep_overshoot_tracker.reset();

Check warning on line 484 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L484 

Added line #L484 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();

if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);

Check warning on line 490 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L490 

Added line #L490 was not covered by tests
}
while (next_frame > now) {
std::this_thread::sleep_for(1ns);
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();

Check warning on line 492 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L492 

Added line #L492 was not covered by tests
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

Check warning on line 494 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L494 

Added line #L494 was not covered by tests

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 1000ms, *cursor);
Expand Down Expand Up @@ -622,17 +627,22 @@
capture(const push_captured_image_cb_t &push_captured_image_cb, const pull_free_image_cb_t &pull_free_image_cb, bool *cursor) override {
auto next_frame = std::chrono::steady_clock::now();

sleep_overshoot_tracker.reset();

Check warning on line 630 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L630 

Added line #L630 was not covered by tests

while (true) {
auto now = std::chrono::steady_clock::now();

if (next_frame > now) {
std::this_thread::sleep_for((next_frame - now) / 3 * 2);
std::this_thread::sleep_for(next_frame - now);

Check warning on line 636 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L636 

Added line #L636 was not covered by tests
}
while (next_frame > now) {
std::this_thread::sleep_for(1ns);
now = std::chrono::steady_clock::now();
now = std::chrono::steady_clock::now();

Check warning on line 638 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L638 

Added line #L638 was not covered by tests
std::chrono::nanoseconds overshoot_ns = now - next_frame;
log_sleep_overshoot(overshoot_ns);

Check warning on line 640 in src/platform/linux/x11grab.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/linux/x11grab.cpp#L640 

Added line #L640 was not covered by tests

next_frame += delay;
if (next_frame < now) { // some major slowdown happened; we couldn't keep up
next_frame = now + delay;
}
next_frame = now + delay;

std::shared_ptr<platf::img_t> img_out;
auto status = snapshot(pull_free_image_cb, img_out, 1000ms, *cursor);
Expand Down
14 changes: 3 additions & 11 deletions src/platform/windows/display_base.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@
SetThreadExecutionState(ES_CONTINUOUS);
});

stat_trackers::min_max_avg_tracker<double> sleep_overshoot_tracker;
sleep_overshoot_tracker.reset();

Check warning on line 157 in src/platform/windows/display_base.cpp

View check run for this annotation

Codecov / codecov/patch

src/platform/windows/display_base.cpp#L157 

Added line #L157 was not covered by tests

while (true) {
// This will return false if the HDR state changes or for any number of other
Expand Down Expand Up @@ -184,16 +184,8 @@
}
else {
high_precision_sleep(sleep_period);

if (config::sunshine.min_log_level <= 1) {
// Print sleep overshoot stats to debug log every 20 seconds
auto print_info = [&](double min_overshoot, double max_overshoot, double avg_overshoot) {
auto f = stat_trackers::one_digit_after_decimal();
BOOST_LOG(debug) << "Sleep overshoot (min/max/avg): " << f % min_overshoot << "ms/" << f % max_overshoot << "ms/" << f % avg_overshoot << "ms";
};
std::chrono::nanoseconds overshoot_ns = std::chrono::steady_clock::now() - sleep_target;
sleep_overshoot_tracker.collect_and_callback_on_interval(overshoot_ns.count() / 1000000., print_info, 20s);
}
std::chrono::nanoseconds overshoot_ns = std::chrono::steady_clock::now() - sleep_target;
log_sleep_overshoot(overshoot_ns);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to avoid too much code duplication, I moved out the overshoot logging to common platform code. Unfortunately I don't have a Windows build environment set up, so I couldn't test these changes.


status = snapshot(pull_free_image_cb, img_out, 0ms, *cursor);

Expand Down
Loading