Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Subtask]: Do not cache DataFile in memory when evaluating tables. #1840

Closed
2 of 3 tasks
Tracked by #1837
, #1930
...
zhoujinsong opened this issue Aug 14, 2023 · 0 comments · Fixed by #1956
Closed
2 of 3 tasks
Tracked by #1837
, #1930
...

[Subtask]: Do not cache DataFile in memory when evaluating tables. #1840

zhoujinsong opened this issue Aug 14, 2023 · 0 comments · Fixed by #1956
Labels
type:subtask Subtask of another issue

Comments

@zhoujinsong
Copy link
Contributor

Search before asking

  • I have searched in the issues and found no similar issues.

Description

Amoro will evaluate every table to determine if it is needed to perform optimizing.

Currently, we need to scan all files of a table to complete the evaluating process, but during the scanning process, we only need to collect information about the files and do not need to cache these file objects to reduce memory consumption.

Parent issue

#1837

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type:subtask Subtask of another issue
Projects
None yet
Milestone
No milestone
1 participant
@zhoujinsong