Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature][Zeta] Support delete logs regularly #7787

Merged
merged 27 commits into from
Nov 5, 2024
Merged

[Feature][Zeta] Support delete logs regularly #7787

merged 27 commits into from
Nov 5, 2024

Conversation

corgy-w
Copy link
Contributor

@corgy-w corgy-w commented Oct 7, 2024

Purpose of this pull request

Delete logs regularly

For details, see logging.md

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@github-actions github-actions bot added document Zeta e2e dependencies Pull requests that update a dependency file labels Oct 7, 2024
dailai
dailai reviewed Oct 9, 2024
View reviewed changes
@corgy-w corgy-w force-pushed the delete-logs-regularly branch from 1d85e73 to 7704b1b Compare October 9, 2024 03:33
@corgy-w corgy-w force-pushed the delete-logs-regularly branch from d888885 to 4516baf Compare October 11, 2024 02:06
@hailin0 hailin0 linked an issue Oct 11, 2024 that may be closed by this pull request
[Feature][Zeta] Support automatic delete of completed task log files #7716
Closed
3 tasks
@hailin0
Copy link
Member

hailin0 commented Oct 11, 2024

link #7712

This was referenced Oct 11, 2024
Merged
Closed
@dailai
Copy link
Contributor

dailai commented Oct 14, 2024

Thinks @corgy-w. Please fix the ci.

@corgy-w
Copy link
Contributor Author

corgy-w commented Oct 14, 2024

Thinks @corgy-w. Please fix the ci.

In progress, but I don't know what went wrong. I've been looking for it.

@corgy-w corgy-w force-pushed the delete-logs-regularly branch from a67d14e to f22fc2b Compare October 21, 2024 03:01
@corgy-w
Copy link
Contributor Author

corgy-w commented Oct 23, 2024

@dailai @hailin0 @Hisoka-X PTAL

@github-actions github-actions bot removed the dependencies Pull requests that update a dependency file label Oct 23, 2024
@corgy-w
Merge branch 'refs/heads/dev' into delete-logs-regularly
25a373b 
# Conflicts:
#	seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/RestHttpGetCommandProcessor.java
#	seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/servlet/BaseServlet.java
#	seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/servlet/LogBaseServlet.java
@corgy-w
[Improve][Zeta] Optimize code
f2860cb
@corgy-w corgy-w requested a review from Hisoka-X October 29, 2024 07:10
Hisoka-X
Hisoka-X reviewed Oct 29, 2024
View reviewed changes
...atunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java Outdated
Comment on lines 408 to 418
for (Map.Entry<TaskGroupLocation, CompletableFuture<SlotProfile>> entry :
preApplyResourceFutures.entrySet()) {
try {
Address worker = entry.getValue().get().getWorker();
historyExecutionPlan.add(
new ExecutionAddress(worker.getHost(), worker.getPort()));

} catch (Exception e) {
LOGGER.warning("history execution plan add worker failed", e);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @zhangshenghang for a look.

...ine-server/src/main/java/org/apache/seatunnel/engine/server/rest/service/BaseLogService.java
log.warn(String.format("Log file path is empty, get logRef : %s", logRef));
return null;
}
return LogUtil.getLogPath();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @zhangshenghang

Hisoka-X
Hisoka-X previously approved these changes Oct 29, 2024
View reviewed changes
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if ci passes. Thanks @corgy-w !

@corgy-w corgy-w requested a review from hailin0 October 30, 2024 10:45
@Hisoka-X Hisoka-X changed the title [Improve][Zeta] Delete logs regularly [Feature][Zeta] Support delete logs regularly Oct 30, 2024
@corgy-w corgy-w requested a review from Hisoka-X November 1, 2024 10:11
Hisoka-X
Hisoka-X previously approved these changes Nov 1, 2024
View reviewed changes
@Hisoka-X Hisoka-X requested a review from hailin0 November 1, 2024 22:08
@corgy-w
Merge branch 'refs/heads/dev' into delete-logs-regularly
ec552e2 
# Conflicts:
#	seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java
@corgy-w
Copy link
Contributor Author

corgy-w commented Nov 5, 2024

@hailin0 @Hisoka-X At present, only a timeout ci, please have time to review it

@hailin0 hailin0 merged commit 5089d8a into apache:dev Nov 5, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dailai dailai dailai left review comments

@hailin0 hailin0 hailin0 approved these changes

@Hisoka-X Hisoka-X Hisoka-X approved these changes

Assignees
No one assigned
Labels
approved document e2e reviewed Zeta Rest API Zeta
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Feature][Zeta] Support automatic delete of completed task log files
4 participants
@corgy-w @hailin0 @dailai @Hisoka-X