Unmodified fields being marked as modified.

[jjacksonjc]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

6.3.5

Node.js version

16.x

MongoDB server version

5.x

Description

When a child field is modified, all children sharing a common parent are getting marked as modified, whether or not they were actually modified. This is a deviation from mongoose v5 and lower behavior and seems to be unintentional since it doesn't appear that any unit tests are checking for this case.

Steps to Reproduce

With mongo listening on 27017 run the following script and observe the output.

'use strict';

const mongoose = require('./lib');

const person = new mongoose.Schema({
  name: {
    first: String,
    last: String
  },
  haircolor: String,
  eyecolor: String
});

const Person = mongoose.model('Person', person);

(async function() {
  await mongoose.connect('mongodb://localhost:27017/example');

  const h = {
    name: {
      first: 'bob',
      last: 'jones'
    },
    haircolor: 'brown',
    eyecolor: 'brown'
  };

  const human = new Person(h);

  await human.save();

  h.name.first = 'Larry';
  human.set(h);

  console.log(`expect 'name,name.first': ${human.modifiedPaths({ includeChildren: true })}`);
  console.log(`expect 'false': ${human.isModified('haircolor')}`);
  console.log(`expect 'false': ${human.isModified('eyecolor')}`);
  console.log(`expect 'true': ${human.isModified('name.first')}`);
  console.log(`expect 'false': ${human.isModified('name.last')}`);

  await mongoose.disconnect();
})();

Expected Behavior

In the above example name.last should not be marked as modified.

[jjacksonjc]

Here's another repro case. Looks like this behavior is occurring when trying to set via an object. Using the args it's fine.

'use strict';

const mongoose = require('./lib');

const person = new mongoose.Schema({
  name: {
    first: String,
    last: String
  },
  haircolor: String,
  eyecolor: String
});

const Person = mongoose.model('Person', person);

(async function() {
  await mongoose.connect('mongodb://localhost:27017/example');

  const bob = 'bob';

  const h = {
    name: {
      first: bob,
      last: 'jones'
    },
    haircolor: 'brown',
    eyecolor: 'brown'
  };

  const human = new Person(h);

  await human.save();

  human.set('name.first', bob);
  console.log(`human.isModified via args: ${human.isModified('name.first')}`);

  human.set({ name: { first: bob } });
  console.log(`human.isModified via obj: ${human.isModified('name.first')}`);

  await mongoose.disconnect();
})();

[jjacksonjc]

@vkarpov15 It looks like 6.3.8 was released a couple hours ago, but 6.3.7 was skipped in releases though the milestone was closed, so maybe either this isn't the correct milestone, or their was an error in the version numbering?

[vkarpov15]

@jjacksonjc we had to release 6.3.8 shortly after 6.3.7, because we accidentally shipped 6.3.7 with failing tests.

In Mongoose, this is exopected behavior, because isModified() returns true if Mongoose will need to update this value in MongoDB, not necessarily if the value has changed. And set() on a nested path modifies the whole path by default, unless the value you're setting is deep equal to the current value.

So human.set({ name: { first: bob } }); means name is modified, because you're unsetting name.last. And, when you do human.set(h); in your original example, you're modifying name because { first: 'bob', last: 'jones' } is not deep equal to { first: 'Larry', last: 'jones' }.

The workaround is to set the merge option on set(), which merges rather than overwriting nested paths. The below script gives you the expected result:

'use strict';

const mongoose = require('mongoose');

const person = new mongoose.Schema({
  name: {
    first: String,
    last: String
  },
  haircolor: String,
  eyecolor: String
});

const Person = mongoose.model('Person', person);

(async function() {
  await mongoose.connect('mongodb://localhost:27017/example');

  const h = {
    name: {
      first: 'bob',
      last: 'jones'
    },
    haircolor: 'brown',
    eyecolor: 'brown'
  };

  const human = new Person(h);
  await human.save();

  h.name.first = 'Larry';
  human.set(h, null, null, { merge: true });

    console.log(`expect 'name,name.first': ${human.modifiedPaths({ includeChildren: true })}`);
  console.log(`expect 'false': ${human.isModified('haircolor')}`);
  console.log(`expect 'false': ${human.isModified('eyecolor')}`);
  console.log(`expect 'true': ${human.isModified('name.first')}`);
  console.log(`expect 'false': ${human.isModified('name.last')}`);

  await mongoose.disconnect();
})();

Does that work for you?

[jjacksonjc]

That did work, but it brings up the question: where are the available options documented? A docs site search for merge kind of makes mention about the new expected behavior you're talking about, though it doesn't say how to get that old behavior. It's not listed as an option to Document.prototype.set(). https://mongoosejs.com/docs/search.html?q=merge

[johnmangan]

@vkarpov15 Your example code for this feature above, #11913 (comment), broke in later versions of 6.x (we tested in 6.13.8, but we did not track down exactly which version broke this). The same test did work back in 6.3.8. Importantly, it is still not working in the latest 7.x.

The output when running that code now is that neither name nor name.first are marked as modified paths, and all 4 isModified checks return false (despite the intent of the 3rd to return true).

Assumption: This appears related to the _skipMarkModified option that is automatically set to true in a couple locations within lib/document.js :: Document.prototype.$set. In my team's specific use-case, functionality was as designed after changing the hardcoded option passed into recursive calls for 'nested' pathtypes from { _skipMarkModified: true } to { _skipMarkModified: !merge }, although I am not familiar enough with the framework to know what other side-effects such a change might cause.