Introduce kittest for automated UI testing and git-lfs for UI snapshot handling #8182

lucasmerlin · 2024-11-19T15:57:28Z

What

Integrates egui_kittest for automated UI testing, including snapshot testing
Update egui to latest main
Introduces git-lfs to the repo.
- Add instructions in BUILD.md
- Improve check_large_files.py to verify that all PNGs are indeed handled by git-lfs.
Run mac rust tests on main to cover for the UI snapshot tests
Related: Use a software renderer for kittest #8245

NOTE: I (@abey79) repeat: introduce git-lfs to the repo.

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)
I have tested the web demo (if applicable):
- Using examples from latest main build: rerun.io/viewer
- Using full set of examples from nightly build: rerun.io/viewer
The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
If applicable, add a new check to the release checklist!
If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide
Run tests in CI
Setup git lfs

To run all checks from main, comment on the PR with @rerun-bot full-check.

To deploy documentation changes immediately after merging this PR, add the deploy docs label.

# Conflicts: # Cargo.lock # Cargo.toml

github-actions · 2024-11-27T14:40:14Z

Web viewer built successfully. If applicable, you should also test it:

I have tested the web viewer

Result	Commit	Link
✅	`722fa53`	https://rerun.io/viewer/pr/8182

^{Note: This comment is updated whenever you push a commit.}

github-actions · 2024-11-27T18:30:01Z

Latest documentation preview deployed successfully.

Result	Commit	Link
✅	`722fa53`	https://landing-bg8co3i6u-rerun.vercel.app/docs

^{Note: This comment is updated whenever you push a commit.}

emilk

I'm very excited to get this in!

crates/viewer/re_time_panel/Cargo.toml

crates/viewer/re_time_panel/src/lib.rs

crates/viewer/re_time_panel/tests/time_panel_tests.rs

emilk · 2024-11-28T08:16:30Z

crates/viewer/re_viewer_context/src/test_context.rs

            });
        });
    }

+    pub fn run_simple(&self, egui_ctx: &egui::Context, func: impl FnOnce(&ViewerContext<'_>)) {


This should have a docstring. I also don't understand the naming.

It's more like the previous run should be called run_ui, and this run_test or just run?

Does this even need to be pub?

I'm reshuffling/improving things here. I'm expecting that this will further evolve when we better integrate this with kittest.

Does this even need to be pub?

Yeah, since we use it in re_time_panel

I think we have:

run (does just the basic non-ui thing, takes a FnOnce(&ViewerContex))

run_in_egui_central_panel (legacy thing that use __run_test_ctx and takes FnMut(&ViewerContext, Ui)). This should probably go away in the future.

some form of a run_with_kittest() (for a future PR)

All of these should handle_system_command(), so we can get rid of run_and_handle_system_commands().

I'm updating this PR according to this.

edit: for clarity, we use run in the existing kittest test, by manually rolling the kittest stuff in the closure.

emilk

I'm very excited to get this in!

crates/viewer/re_time_panel/tests/time_panel_tests.rs

abey79 · 2024-11-28T11:12:25Z

@rerun-bot full-check

github-actions · 2024-11-28T11:12:54Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12067771530

abey79 · 2024-11-28T12:22:19Z

@rerun-bot full-check

github-actions · 2024-11-28T12:22:50Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12068811066

abey79 · 2024-11-28T16:38:19Z

@rerun-bot full-check

github-actions · 2024-11-28T16:38:51Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12072739603

abey79 · 2024-11-28T17:37:47Z

@rerun-bot full-check

github-actions · 2024-11-28T17:38:16Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12073470333

abey79 · 2024-11-28T17:46:07Z

@rerun-bot full-check

github-actions · 2024-11-28T17:46:35Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12073556187

abey79 · 2024-11-28T17:56:48Z

@rerun-bot full-check

github-actions · 2024-11-28T17:57:14Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12073654214

abey79 · 2024-11-28T18:24:45Z

Last run attempt did work correctly CI-wise, but test failed. Looks like a git-lfs thing

abey79 · 2024-11-28T19:56:14Z

@rerun-bot full-check

github-actions · 2024-11-28T19:56:48Z

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12074809976

abey79 · 2024-11-29T07:56:41Z

finally...

Unclear about nightly as its been broken for a week.

### Related - Related to #8182 ### What Introduce a new helper to manage and visualise failed kittest snapshot tests using Rerun. ``` pixi run snapshots --help # view all failed snapshot pixi run snapshots # remove all failed snapshot temp files pixi run snapshots --clean # only view (or clean) snapshot for that crate pixi run snapshots -p re_time_panel ``` https://github.com/user-attachments/assets/c2ad63be-ff85-4235-a161-a359ff540d89

lucasmerlin and others added 5 commits November 19, 2024 16:54

Add kittest and time panel snapshot test

430844e

Add some test data

d6aac19

Merge branch 'main' into lucas/add-kittest

049a997

# Conflicts: # Cargo.lock # Cargo.toml

Post-merge fix, update egui to latest main, misc fixes

79071e8

Add snapshot

d6a9951

spurious

910bd00

abey79 added ui concerns graphical user interface 🔨 testing testing and benchmarks exclude from changelog PRs with this won't show up in CHANGELOG.md labels Nov 27, 2024

abey79 added 3 commits November 27, 2024 17:02

WIP readme update

8a0738a

Moved info to BUILD.md and added a png check to check_large_files.py

20ab64a

Merge branch 'main' into lucas/add-kittest

9dacd38

abey79 marked this pull request as ready for review November 27, 2024 18:30

abey79 changed the title ~~Add kittest and time panel snapshot test~~ Introduce kittest for automated UI testing and git-lfs for UI snapshot handling Nov 27, 2024

lints

6b24350

emilk requested review from emilk November 28, 2024 08:03

emilk reviewed Nov 28, 2024

View reviewed changes

emilk approved these changes Nov 28, 2024

View reviewed changes

lucasmerlin commented Nov 28, 2024

View reviewed changes

crates/viewer/re_time_panel/tests/time_panel_tests.rs Outdated Show resolved Hide resolved

abey79 added 3 commits November 28, 2024 12:04

Review comments

e180e1b

Skip UI tests when not running on macos

e6ab7af

Merge branch 'main' into lucas/add-kittest

f1b86c7

abey79 added 2 commits November 28, 2024 13:21

skip -> ignore

a548d8e

WIP: build_ui

594dfba

abey79 added 3 commits November 28, 2024 17:15

Fix tests not passing due to time panel store subscriber not registered

4fbfab8

Merge branch 'main' into lucas/add-kittest

1e331f7

Run mac tests on main

c7122ed

abey79 added 2 commits November 28, 2024 17:43

Added comment

03b750f

try again with the yml dsl

d49fe7e

claud to the rescue... maybe?

1a1adf4

who am I kidding?

9eac0a6

abey79 and others added 2 commits November 28, 2024 20:55

enable lfs

5e03ae7

Merge branch 'main' into lucas/add-kittest

722fa53

abey79 merged commit 97b9e44 into main Nov 29, 2024
70 checks passed

abey79 deleted the lucas/add-kittest branch November 29, 2024 07:56

abey79 mentioned this pull request Dec 6, 2024

Add a script to easily compare failed snapshot tests #8346

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce kittest for automated UI testing and git-lfs for UI snapshot handling #8182

Introduce kittest for automated UI testing and git-lfs for UI snapshot handling #8182

lucasmerlin commented Nov 19, 2024 •

edited by abey79

Loading

github-actions bot commented Nov 27, 2024 •

edited

Loading

github-actions bot commented Nov 27, 2024 •

edited

Loading

emilk left a comment

emilk Nov 28, 2024

emilk Nov 28, 2024

abey79 Nov 28, 2024

lucasmerlin Nov 28, 2024

abey79 Nov 28, 2024 •

edited

Loading

emilk left a comment

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 29, 2024

Introduce kittest for automated UI testing and git-lfs for UI snapshot handling #8182

Introduce kittest for automated UI testing and git-lfs for UI snapshot handling #8182

Conversation

lucasmerlin commented Nov 19, 2024 • edited by abey79 Loading

What

Checklist

github-actions bot commented Nov 27, 2024 • edited Loading

github-actions bot commented Nov 27, 2024 • edited Loading

emilk left a comment

Choose a reason for hiding this comment

emilk Nov 28, 2024

Choose a reason for hiding this comment

emilk Nov 28, 2024

Choose a reason for hiding this comment

abey79 Nov 28, 2024

Choose a reason for hiding this comment

lucasmerlin Nov 28, 2024

Choose a reason for hiding this comment

abey79 Nov 28, 2024 • edited Loading

Choose a reason for hiding this comment

emilk left a comment

Choose a reason for hiding this comment

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 28, 2024

abey79 commented Nov 28, 2024

github-actions bot commented Nov 28, 2024

abey79 commented Nov 29, 2024

lucasmerlin commented Nov 19, 2024 •

edited by abey79

Loading

github-actions bot commented Nov 27, 2024 •

edited

Loading

github-actions bot commented Nov 27, 2024 •

edited

Loading

abey79 Nov 28, 2024 •

edited

Loading