High memory usage when loading & resizing a 6000x6000 jpg => out of memory

[mabead]

Prerequisites

• I have written a descriptive issue title
• I have verified that I am running the latest version of ImageSharp
• I have verified if the problem exist in both DEBUG and RELEASE mode
• I have searched open and closed issues to ensure it has not already been reported

Description

Usaging ImageSharp beta003, I resize an image with the following code:

                using (var image = Image.Load(sourceFileName))
                using (var outputFile = File.OpenWrite(target))
                {
                    image.Mutate(ctx => ctx.Resize(newWidth, newHeight));
                    image.SaveAsJpeg(outputFile);
                }

I can execute this code a few times but it eventually causes my process to die because of an out-of-memory.

Steps to Reproduce

This code works perfectly fine on my machine (hehe) but when I run in production where memory is more limited, I quickly get out of memory errors. In production, my code runs on linux, on dotnet core 2.0.5 inside of a docker container that is limited to 1 GB of RAM.

The problem occurs with this 6000x6000 jpg.

Here's what I did (each step invokes the code mentionned above, so the image is loaded and resized starting from the file on disk):

1. resize to 401x401 => memory increases from 143 MiB to 572 MiB (jump of ~430 MiB)
2. resize to 402x402 => memory increases from 572 MiB to 971 MiB which is very close to the memory allocated to my docker container. At this point, I suppose that the garbage collector kicks in because I see my memory dropping back to 369 MB
3. resize to 403x403 => memory increases to 548 MiB
4. resize to 404x404 => memory increases to 794 MiB
5. resize to 405x405 => my process dies with out-of-memory

From step 2 to 3 and 3 to 4, we can see that the minimum memory required to resize an image is a little more than 200 MiB. So when step 5 starts, we can expect a quick demand for more than 200 MiB and the garbage collector does not have the time to free memory before the process dies because it is running out of memory.

So first question: does ImageSharp allocates unmanaged memory that can prevent the garbage collector to properly handle the request for memory involved in step 5? To be more general, do you see something else that could cause this failure?

When we look at these numbers, there is something else interesting: even after the garbage collector run after step 2, the memory did not go all the way down to 143 MiB (the initial level). It only went back to 369 MiB. To investigate that, I tried something on my dev machine: I loaded the same file and checked the memory usage at a few steps:

1. Before doing the Image.Load
2. After doing the Image.Load
3. After disposing the loaded image (thus after saving the resized image as a jpg)

Here's what the process memory usage looked like (as shown by the Visual Studio diagnostic tool... which I don't know if it is totally reliable):

1. 68 MB
2. 421 MB
3. 391 MB

So once again, it looks like there is still something being consumed by ImageSharp after the image is resized.

So second question: Does ImageSharp reserves memory, buffer pools or something else that could cause this high memory consumption even after I am done resizing the image? If yes, is there a way to free this memory? Can I limit this memory?

System Configuration

• ImageSharp version: beta003
• Environment (Operating system, version and so on): docker image based on microsoft/aspnetcore:2.0.5
• .NET Framework version: dotnet core 2.0.5

[antonfirsov]

@mabead does this help:
https://sixlabors.github.io/docs/articles/ImageSharp/MemoryManagement.html
?

I also recommend you to rethink your scaling strategy, if possible. A memory limit of 1GB is too low for image processing applications, specially if you need to handle very large images.

[mabead]

@antonfirsov It totally helps!

Given, how mainstream image processing / resizing has become these days, we forgot (or didn't know) how CPU and memory intensive it could be. Doh!

Note that I tried the ArrayPoolMemoryManager.CreateWithModeratePooling() strategy but it wasn't sufficient. I still have out of memory problems. I will try to move our resizing logic outside of this container and put it somewhere that scales better. I'll possibly use this approach.

Thanks for your quick support!

BTW: the ImageSharp documentation and articles is not obvious to find. Here's what I did:

• Went to the github project page since it is usually the starting point to find everything.
• At the very top, I see a description saying A cross-platform library for the processing of image files; written in C# that redirects me to the project site. This looks like a good place to dig further!
• On the project site, I see links to "Getting Started" and "A Better API". None of these pages point to the full ImageSharp documentation.
• I then used the project site search box and searched for memory or doc or documentation but didn't find anything.

I finally was able to find the documentation by going back on the github page, not following the link on the top, and reading the full README to find "API documentation is available". Sorry, TLDR... shame on me.

I suggest that you link your github.io and https://sixlabors.com/projects/imagesharp/. It would make it much easier to find your documentation.

[JimBobSquarePants]

That’s an excellent suggestion regarding the documentation link. We’ll definitely do that, thanks!

There’s also the SimpleGcMemoryManager that doesn’t pool so give that a try also.

We’re gonna try everything in our power to reduce the number of allocations for V1 so maybe we can bring the numbers down slightly.

We’re also going to work hard on writing the most comprehensive documentation we can think make things as easy as possible to discover and understand.

[mabead]

@JimBobSquarePants I'll give a try to SimpleGcMemoryManager next week.

[antonfirsov]

I'm really curious if falling back to SimpleGcMemoryManager could help here.

An MMF MemoryManager could definitely solve this.

[mabead]

@JimBobSquarePants I'm sorry but I won't try SimpleGcMemoryManager since I moved the code to an AWS Lambda. That way, I am sure of the memory that I have available for the current operation and I am sure that even if I have 50 resize operations in parallel, I won't run out of memory.