nodejs CPU usage spike when saving large documents with mongoose

[evgenyfedorenko]

Do you want to request a feature or report a bug?
Bug probably

What is the current behavior?
When inserting many large documents experiencing a spike in CPU

If the current behavior is a bug, please provide the steps to reproduce.

Here is the schema I use:

{
  ticker: String,
  asks: [[Number]],
  bids: [[Number]],
  timestamp: String,
}

Asks and bids field can have up to 1000 arrays inside the outer array.
Both .save and insertMany cause this behavior.
Edit: If I use MongoDB Node.js Driver the CPU stays at normal levels

What is the expected behavior?
CPU spike should not occur.

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.
Node v12.19.0
Mongoose 5.10.1
MongoDB 4.4.0

Thank you for help!

[vkarpov15]

I've managed to confirm that the below script takes 10x longer when using nested arrays [[Number]] than plain arrays [Number]:

'use strict';

const mongoose = require('mongoose');

mongoose.set('useFindAndModify', false);

const { Schema } = mongoose;

run().catch(err => console.log(err));

async function run() {
  await mongoose.connect('mongodb://localhost:27017/test', {
    useNewUrlParser: true,
    useUnifiedTopology: true
  });

  await mongoose.connection.dropDatabase();

  const bookSchema = Schema({
  ticker: String,
  asks: [[Number]],
  bids: [[Number]],
  timestamp: String,
  });
  const Book = mongoose.model('Book', bookSchema);

  let doc = { asks: [], bids: [] };
  for (let i = 0; i < 5000; ++i) {
    doc.asks.push([i]);
    doc.bids.push([i]);
  }

  /*
  const bookSchema = Schema({
  ticker: String,
  asks: [Number],
  bids: [Number],
  timestamp: String,
  });
  const Book = mongoose.model('Book', bookSchema);

  let doc = { asks: [], bids: [] };
  for (let i = 0; i < 5000; ++i) {
    doc.asks.push(i);
    doc.bids.push(i);
  }
*/

  let start = Date.now();
  for (let i = 0; i < 100; ++i) {
    const _doc = new Book(doc);
  }
  console.log(Date.now() - start);
}

I'm still investigating why this is happening and how we can improve this, but I've found a couple low-effort low-risk fixes that can make this a bit faster. Will keep working on this 👍

[evgenyfedorenko]

Thanks @vkarpov15. I appreciate it.

[evgenyfedorenko]

Tbh array of arrays type field is not very useful when it comes to querying it - basically I don't even know if it is possible. I am likely not going to use that format going forward but the issue surely exists.

[evgenyfedorenko]

Any ETA on this ?