Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task]: improve loading time for execution information perspective with large number of executions #2845

Closed
bamaer opened this issue Apr 23, 2023 · 1 comment
Assignees
Labels
Hop Gui P2 Default Priority task
Milestone

Comments

@bamaer
Copy link
Contributor

bamaer commented Apr 23, 2023

What needs to happen?

the execution information perspective is slow to load large numbers of executions and/or executions with large file sizes (e.g. when executed with Detailed or Row Level logging).

Additionally: show a warning when a workflow or pipeline is executed with Detailed or Row Level logging, or provide an option to skip execution logging for Detailed or Row Level logging.

Issue Priority

Priority: 2

Issue Component

Component: Hop Gui

@mattcasters
Copy link
Contributor

I think that for large logging text blocks it's just not going to work to store this in the execution state JSON.
There's no skipping the loading of individual fields when inflating a JSON file.
As such I think that in order to get your hands on the logging text we need to provide a separate method in IExecutionInfoLocation and we need to store the logging text in a separate file.

@mattcasters mattcasters added this to the 2.5 milestone Apr 27, 2023
mattcasters added a commit to mattcasters/hop that referenced this issue Apr 27, 2023
…ation perspective with large number of executions
mattcasters added a commit to mattcasters/hop that referenced this issue Apr 27, 2023
…ation perspective with large number of executions
mattcasters added a commit to mattcasters/hop that referenced this issue Apr 27, 2023
…ation perspective with large number of executions
bamaer added a commit that referenced this issue May 2, 2023
bamaer pushed a commit to bamaer/hop that referenced this issue May 7, 2023
…ation perspective with large number of executions
bamaer pushed a commit to bamaer/hop that referenced this issue May 7, 2023
…ation perspective with large number of executions
@hansva hansva closed this as completed Jun 7, 2023
@bamaer bamaer reopened this Nov 18, 2023
bamaer added a commit to bamaer/hop that referenced this issue Nov 18, 2023
bamaer added a commit to bamaer/hop that referenced this issue Nov 20, 2023
hansva added a commit that referenced this issue Nov 22, 2023
scan subfolders to reduce exec info perspective loading time #2845
@hansva hansva modified the milestones: 2.5, 2.8 Nov 22, 2023
@hansva hansva closed this as completed Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Hop Gui P2 Default Priority task
Projects
None yet
Development

No branches or pull requests

3 participants