next image optimization significantly slower in next after 10.0.7

[mjenzen]

What version of Next.js are you using?

10.1.3-canary-0

What version of Node.js are you using?

12.12

What browser are you using?

any

What operating system are you using?

linux server

How are you deploying your application?

next start

Describe the Bug

The latest canary releases seem to have fixed the image optimization memory consumption, however the initial image optimization is noticeably slower than the previous version (10.0.7) and can take seconds to populate images on the page. It seems to be exacerbated by loading more images in quick succession as well.

Expected Behavior

Images should load with minimal delay as seen in 10.0.7 to prevent degrading user experience

To Reproduce

Load a page with numerous next/image components and look for images popping in significantly after page load

[karlhorky]

Below is some more context in the tweet from @lovell, the sharp maintainer. Benchmarks may not be completely optimized, but apparently the squoosh approach now in Next.js is up to 25x slower:

Benchmark: lovell/sharp#2643

Tweet: https://twitter.com/lovell/status/1376647760309587979

[JimmyGray]

I'm having the same issue. Some of our pages have 20+ images and the difference is huge. I've rolled back to 10.0.6 for now.

[henryStelle]

I've experienced the same problem, @JimmyGray.
As explained in #22253, sharp was removed to decrease bundle size and increase supported platforms. However, some of these issues have been partially addressed with the v0.28 release of sharp lovell/sharp#2604.

[amuttsch]

We had to roll back as well. We have up to 40 images per page which takes up to a minute to process on my local machine. Due to business concerns we cannot cache images indefinitely and have to regenerate them each day. The current performance is not working on production for us if users have to wait dozens of seconds to see images. We also had to roll back to 10.0 for now.

Is it possible to revert back to sharp as they have addressed the compatibility issues or maybe using it with a feature flag? Thanks!

[kripod]

I would love to see bringing back sharp, potentially as an optional peer dependency. Although Squoosh is a fantastic project on its own, I think it should only be used as a fallback or an opt-in until the situation with ARM machines improves.

[addyosmani]

@amuttsch would it be possible to share a URL with examples of the kinds of images per page that are taking a long time to process locally? Speaking with the Squoosh team about the performance feedback on this thread and any repros would be helpful to look at :)

[NikhilVerma]

We are also noticing similar issues:

• Image sizes are larger for the same quality parameter, I believe Next.js should reduce the default image size from 75 to compensate. Otherwise the consumers will see their website sizes grow. Here is an example.
  • image1 this image comes from an older version of Next.js 56.4kb
  • image2 same image, same quality with the latest next.js version 63.4KB
• The response times for first requests have increased tremendously, you can use the same images in the previous example and tweak the quality param to something new. I see at least 3x slowdown in my testing.
• We use a lot of local images and Next.js caches them only for 60s with no way to override that value. So every 60s we have to face with a long load time for requested images along with a high CPU usage.

My recommendations

1. Re-add sharp as a additional config/override/loader. I am not sure if we can get faster than a bare-metal native implementation, even if WASM is making great strides.
2. Allow overriding the default seconds for cache

Our solution

For now to mitigate this we are adding an additional layer of caching on top of /_next/image in our NGINX instance. It's not ideal because we shouldn't be needing this layer but it helps with the 60s expiry of Next.js local assets.

Here is a sample config

http {
    proxy_cache_path /tmp/image_cache levels=1:2 keys_zone=image_cache:30m use_temp_path=off inactive=7d max_size=2G loader_files=100 loader_sleep=50 loader_threshold=200;
    proxy_cache_valid 604800s; # 7 days
    proxy_cache_lock on;
    proxy_cache_use_stale updating;
    proxy_buffer_size 16k;
    proxy_buffers 16 16k;
    proxy_buffering on;
    proxy_http_version 1.1;
    proxy_ignore_headers Vary;
    proxy_ignore_headers Expires;
    proxy_ignore_headers Set-Cookie;
    proxy_ignore_headers Cache-Control;

    server {
        # ... server params

        # NextJS Proxy for images
        # We need this because of https://github.com/vercel/next.js/issues/23637#issuecomment-822358192
        # Until it's improved we need to maintain our own copy of next/images
        location /_next/image {
            proxy_pass http://localhost:3001;

            # request_uri is important because images have GET params which define quality, url and size
            # $http_accept is important because it changes the response type of next/image
            proxy_cache_key "$request_uri$http_accept";

            proxy_cache image_cache;
        }
    }
}

[amuttsch]

@addyosmani Unfortunately, I'm not able to share images due to copyright and licensing concerns.

But to give you some metadata: our source images are around 500-1000kB in size and have a resolution around 1500x2300 pixel and they are sized down to a width of ~220px using q=75 and webp format. I hope this helps you :-)

[timcole]

Just jumping in here to add my info on this. I recently switched to next 10.1.4-canary.10 to see if that solved the image memory leak but unfortunately our web servers were still using 1GB of ram each (we only use next/image for 5 images). Switching back to 10.0.6 we average around 30Mi.

Example Images:
https://constellation.spaceflight.live/falcon9.webp
https://constellation.spaceflight.live/SN11.webp
https://constellation.spaceflight.live/b/b1fd1f3a97183e4a8d04794c71b7fdaf.webp
https://constellation.spaceflight.live/b/cd23264325291a089910a28d70139647.webp
https://constellation.spaceflight.live/b/431a992c3171c39331a337f99533c3cc.webp

Code is public at https://github.com/spaceflight-live/comet

[fabb]

I'm seeing a response time increase of 12% after updating from 10.0.7 to 10.2.0 (and updating from webpack 4 to webpack 5). I'm not using the image optimization feature at all though. Could the issue still be related to this issue?

[fabb]

I created a separate issue: #25032