Image Optimization - Create new ImagePool for each batch #470

a258wang · 2022-06-28T00:44:57-04:00

a258wang commented

2022-06-28 00:44:57 -04:00

We have been having issues with the image optimization script consuming egregious amounts of memory and failing CI as a result. Previously, we tried processing images in "batches" in order to reduce memory usage, however this fix did not work for long.

This PR changes the script so that we use a new ImagePool to process each batch of images. Preliminary results in my local environment and on CI seem promising, and this is a relatively low risk (and low effort) change, so let's see how it holds up.

We have been having issues with the image optimization script consuming egregious amounts of memory and failing CI as a result. Previously, we tried processing images in "batches" in order to reduce memory usage, however this fix did not work for long. This PR changes the script so that we use a new `ImagePool` to process each batch of images. Preliminary results in my local environment and on CI seem promising, and this is a relatively low risk (and low effort) change, so let's see how it holds up.

a258wang added 1 commit 2022-06-28 00:44:59 -04:00

continuous-integration/drone/push Build is passing Details

f589ec9bfe Create new ImagePool for each batch

a258wang requested review from snedadah 2022-06-28 00:53:19 -04:00

a258wang referenced this pull request

2022-06-28 00:54:48 -04:00

Fix Andrew Wang photos #469

snedadah approved these changes 2022-06-28 14:51:16 -04:00

a258wang merged commit 14ef810ad9 into main

2022-06-28 22:54:21 -04:00

a258wang deleted branch amy-image-optimization-image-pool

2022-06-28 22:54:22 -04:00