Image Optimization - Create new ImagePool for each batch #470

Merged
a258wang merged 1 commits from amy-image-optimization-image-pool into main 2 months ago
Owner

We have been having issues with the image optimization script consuming egregious amounts of memory and failing CI as a result. Previously, we tried processing images in "batches" in order to reduce memory usage, however this fix did not work for long.

This PR changes the script so that we use a new ImagePool to process each batch of images. Preliminary results in my local environment and on CI seem promising, and this is a relatively low risk (and low effort) change, so let's see how it holds up.

We have been having issues with the image optimization script consuming egregious amounts of memory and failing CI as a result. Previously, we tried processing images in "batches" in order to reduce memory usage, however this fix did not work for long. This PR changes the script so that we use a new `ImagePool` to process each batch of images. Preliminary results in my local environment and on CI seem promising, and this is a relatively low risk (and low effort) change, so let's see how it holds up.
a258wang added 1 commit 2 months ago
f589ec9bfe Create new ImagePool for each batch
a258wang requested review from snedadah 2 months ago
snedadah approved these changes 2 months ago
a258wang merged commit 14ef810ad9 into main 2 months ago
a258wang deleted branch amy-image-optimization-image-pool 2 months ago

Reviewers

snedadah approved these changes 2 months ago
continuous-integration/drone/push Build is passing
The pull request has been merged as 14ef810ad9.
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This pull request currently doesn't have any dependencies.

Loading…
There is no content yet.