large memory used when infer

[tensor-tang]

This is an issue of NLP online service.

When run inference, the memory usage is always kept as about 6G, which is definitely larger than actually needed.

[ChinaLiuHao]

I meet this situation too. In addition, when i use the inference by multi-thread way with "export OPENBLAS_NUM_THREADS=1", the program may end with the "Aborted" error!

[tensor-tang]

@ChinaLiuHao
And as an addition, the "Abort" error is randomly encountered, not always appears.

[luotao1]

The OCR CRNN_CTC service also has a large memory:

[tensor-tang]

Paddle/paddle/fluid/memory/detail/buddy_allocator.cc

Lines 188 to 192 in 666c94e

	// Allocate a new maximum sized block
	size_t index = 0;
	void* p = system_allocator_->Alloc(&index, max_chunk_size_);

This should be the reason.
Paddle would alloc max chunk size at the first time.

[tensor-tang]

After debugging we can found there is a flag to choose how much memory we would like to use at the first time. Default it would use about 3.2%(1/32) of your total memory.

usage:

your_app --fraction_of_cpu_memory_to_use=0.1 # it would use 3.2% * 0.1 of total

The track back should be like this:

Paddle/paddle/fluid/platform/cpu_info.cc

Lines 26 to 28 in 666c94e

	DEFINE_double(fraction_of_cpu_memory_to_use, 1,
	"Default use 100% of CPU memory for PaddlePaddle,"
	"reserve the rest for page tables, etc");

Paddle/paddle/fluid/platform/cpu_info.cc

Lines 54 to 58 in 666c94e

	size_t CpuMaxAllocSize() {
	// For distributed systems, it requires configuring and limiting
	// the fraction of memory to use.
	return FLAGS_fraction_of_cpu_memory_to_use * CpuTotalPhysicalMemory();
	}

Paddle/paddle/fluid/platform/cpu_info.cc

Lines 65 to 69 in 666c94e

	size_t CpuMaxChunkSize() {
	// Allow to allocate the maximum chunk size is roughly 3% of CPU memory.
	return CpuMaxAllocSize() / 32;
	}

Paddle/paddle/fluid/memory/malloc.cc

Lines 32 to 36 in 666c94e

	if (a == nullptr) {
	a = new detail::BuddyAllocator(new detail::CPUAllocator,
	platform::CpuMinChunkSize(),
	platform::CpuMaxChunkSize());
	}

[tensor-tang]

@ChinaLiuHao
About the "Abort" issue, we can open another issue to discuss it. Thanks.