Skip to content

feat: Apply waiting time bonus to pupils #1370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JeangelLF merged 2 commits into from
Mar 13, 2026
Merged

feat: Apply waiting time bonus to pupils #1370

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1d3383a
feat: Apply waiting time bonus to pupils
JeangelLF Mar 9, 2026
fd4db4f
feat: Update subject bonus calculation / update score results
JeangelLF Mar 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions common/match/matching.perf.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,33 +146,33 @@ describe('Real World Matching Performance', () => {
10,
Copy link
Member

@Jonasdoubleyou Jonasdoubleyou Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How often do we match currently? If we match too often, the score does not have any effect at all ...

Copy link
Contributor Author

@JeangelLF JeangelLF Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How often do we match currently?

Angebote team does it on Fridays EOD.

If we match too often, the score does not have any effect at all ...

Could you elaborate a bit on this? I'm worried I might be misunderstanding how the matching algorithm works / missing something extremely obvious.

For example, at the moment, there are around 70 pupils in the pool. Most of them need help in similar subjects, and there aren’t many students available right now. Our intention with this change is to prioritise pupils who have been waiting for weeks, rather than those who have just entered the pool with a similar matching profile.

Copy link
Member

@Jonasdoubleyou Jonasdoubleyou Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, if you have one pupil and one student that can be matched due to constraints, then the score is entirely useless as we will match those and do not have any other option. If on the other hand you take all pupils and all students, then there are many many possible matches and the score becomes highly relevant. Thus there is a big trade off between waiting time and match quality, which is also reflected in the small simulation we do here. You can see the results here for adding pupils and students as they arrived in the match pool from our historical data, and then running the matching algorithm every 1 day, every 10 days or every 1000 days. As you can see we get the best results if we match every 1000 days (many many users in the match pool, but the average waiting time is 500 days), quite okay results if we match every 10 days, and terrible results if we match on a daily basis.

So in general, it holds that match_quality ~ possible_matches ~ pupils_to_match * students_to_match. If we have an unbalanced matching as currently it is actually quite okay to match very often as even for pupil_to_match = 70 && students_to_match = 1 we have an okayish number of combinations.

You generally have the following trade-offs:

  • More constraints reduce match quality according to the score (as less matches are actually possible) vs. More constraints are enforced
  • A bigger match pool improves match quality according to the score, at the expense of waiting time

Using the score to balance out the waiting time (i.e. let users wait a bit longer by matching not too often, but prevent them from waiting for a long time through the score) is generally a good idea which I already tried before (the commented out code), but I wasn't too happy with the results so I abandoned it. Seems you've found a heuristic which is okayish, although the actual impact is also quite moderate (according to our historical data).

Copy link
Member

@Jonasdoubleyou Jonasdoubleyou Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you can see we get the best results if we match every 1000 days (many many users in the match pool, but the average waiting time is 500 days), quite okay results if we match every 10 days, and terrible results if we match on a daily basis

Ah, in the current state one can already see the impact of the constraints, as all three variants are currently pretty much equal in the simulation, so the score has basically no real world impact.

{
matchCountSum: 1045,
matchingSubjectsAvg: 1.6794258373205742,
matchingSubjectsAvg: 1.6803827751196172,
matchingSubjects: {
'>= 1': 1045,
'>= 2': 555,
'>= 3': 119,
'>= 2': 538,
'>= 3': 136,
'>= 4': 31,
'>= 5': 4,
'>= 5': 5,
},
// matchingState: 0.11100478468899522,
pupilWaitingTimeAvg: 8.342127835891821,
pupilWaitingTimeAvg: 8.297880297592158,
pupilWaitingTime: {
'>= 0': 1045,
'>= 1': 959,
'>= 7': 429,
'>= 14': 72,
'>= 1': 962,
'>= 7': 439,
'>= 14': 82,
'>= 21': 53,
'>= 28': 41,
'>= 28': 35,
},
studentWaitingTimeAvg: 53.936342931009634,
studentWaitingTimeAvg: 53.57448000132908,
studentWaitingTime: {
'>= 0': 1045,
'>= 1': 1034,
'>= 7': 901,
'>= 14': 776,
'>= 21': 715,
'>= 28': 654,
'>= 56': 465,
'>= 1': 1029,
'>= 7': 880,
'>= 14': 754,
'>= 21': 695,
'>= 28': 651,
'>= 56': 469,
},
},
],
Expand Down
64 changes: 41 additions & 23 deletions common/match/matching.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
import { pupil_languages_enum, student_languages_enum } from '@prisma/client';
import moment from 'moment';
import { computeMatchings, Matching, MatchOffer, MatchRequest, matchScore, NO_MATCH } from './matching';

const TODAY_TEST_DATE = new Date('2026-02-01T00:00:00Z');

function testScore(name: string, request: MatchRequest, offer: MatchOffer, expected: number) {
it(name, () => {
const actual = matchScore(request, offer);
const actual = matchScore(request, offer, TODAY_TEST_DATE);
expect(actual).toEqual(expected);
});
}

function test(name: string, requests: MatchRequest[], offers: MatchOffer[], expected: Matching, excludeMatchings: Set<string> = new Set()) {
it(name, () => {
const actual = computeMatchings(requests, offers, excludeMatchings, new Date(100000000000));
const actual = computeMatchings(requests, offers, excludeMatchings, TODAY_TEST_DATE);
expect(actual).toEqual(expected);
});
}
Expand All @@ -20,7 +23,7 @@ const requestOne = {
pupilId: 1,
state: 'at' as const,
subjects: [{ name: 'Deutsch', mandatory: false }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [pupil_languages_enum.Englisch, pupil_languages_enum.Spanisch],
};

Expand All @@ -29,7 +32,7 @@ const requestTwo = {
pupilId: 2,
state: 'at' as const,
subjects: [{ name: 'Mathematik', mandatory: false }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [],
};

Expand All @@ -38,7 +41,7 @@ const requestThree = {
pupilId: 3,
state: 'at' as const,
subjects: [{ name: 'Klingonisch', mandatory: false }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [],
};

Expand All @@ -50,7 +53,7 @@ const requestFour = {
{ name: 'Mathematik', mandatory: false },
{ name: 'Deutsch', mandatory: false },
],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [],
};

Expand All @@ -62,7 +65,7 @@ const requestFive = {
{ name: 'Mathematik', mandatory: true },
{ name: 'Deutsch', mandatory: false },
],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
calendarPreferences: {
weeklyAvailability: {
monday: [{ from: 600, to: 660 }],
Expand All @@ -82,7 +85,7 @@ const requestSix = {
pupilId: 5,
state: 'at' as const,
subjects: [{ name: 'Mathematik' }, { name: 'Deutsch' }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
onlyMatchWith: 'female' as const,
languages: [],
};
Expand All @@ -92,7 +95,7 @@ const requestSeven = {
pupilId: 5,
state: 'at' as const,
subjects: [{ name: 'Mathematik' }, { name: 'Deutsch' }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
hasSpecialNeeds: true,
languages: [],
};
Expand All @@ -102,7 +105,7 @@ const requestEight = {
pupilId: 8,
state: 'at' as const,
subjects: [{ name: 'Mathematik' }, { name: 'Deutsch' }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
calendarPreferences: {
weeklyAvailability: {
monday: [
Expand All @@ -125,23 +128,23 @@ const offerOne = {
studentId: 1,
state: 'at' as const,
subjects: [{ name: 'Deutsch', grade: { min: 1, max: 10 } }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [],
};

const offerTwo = {
studentId: 2,
state: 'at' as const,
subjects: [{ name: 'Mathematik', grade: { min: 1, max: 10 } }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [],
};

const offerThree = {
studentId: 3,
state: 'at' as const,
subjects: [{ name: 'Klingonisch', grade: { min: 1, max: 10 } }],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
languages: [],
};

Expand All @@ -152,7 +155,7 @@ const offerFour = {
{ name: 'Deutsch', grade: { min: 1, max: 10 } },
{ name: 'Mathematik', mandatory: false, grade: { min: 1, max: 10 } },
],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
gender: 'male' as const,
hasSpecialExperience: false,
languages: [],
Expand All @@ -165,7 +168,7 @@ const offerFive = {
{ name: 'Deutsch', grade: { min: 1, max: 10 } },
{ name: 'Mathematik', mandatory: false, grade: { min: 1, max: 10 } },
],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
gender: 'female' as const,
hasSpecialExperience: true,
languages: [student_languages_enum.Deutsch, student_languages_enum.Spanisch],
Expand All @@ -178,7 +181,7 @@ const offerSix = {
{ name: 'Deutsch', grade: { min: 1, max: 10 } },
{ name: 'Mathematik', mandatory: false, grade: { min: 1, max: 10 } },
],
requestAt: new Date(0),
requestAt: TODAY_TEST_DATE,
calendarPreferences: {
weeklyAvailability: {
monday: [
Expand All @@ -199,18 +202,33 @@ const offerSix = {

describe('Matching Score Basics', () => {
testScore('no subject', requestOne, offerTwo, NO_MATCH);
testScore('one subject', requestOne, offerOne, 0.6000000000000001);
testScore('two subjects', requestFour, offerFour, 0.9046376623823058);
testScore('two requested one offered', requestFour, offerOne, 0.6000000000000001);
testScore('one requested two offered', requestOne, offerFour, 0.6000000000000001);
testScore('one subject', requestOne, offerOne, 0.3);
testScore('two subjects', requestFour, offerFour, 0.41423912339336466);
testScore('two requested one offered', requestFour, offerOne, 0.3);
testScore('one requested two offered', requestOne, offerFour, 0.3);
// testScore('one requested two offered - different state', requestOne, offerFive, 0.495);
testScore('one requested two offered - share english', requestOne, offerSix, 0.625);
testScore('one requested two offered - share spanish', requestOne, offerFive, 0.65);
testScore('one requested two offered - share english', requestOne, offerSix, 0.35);
testScore('one requested two offered - share spanish', requestOne, offerFive, 0.4);
});

describe('Matching Score Mandatory', () => {
testScore('mandatory not offered', requestFive, offerOne, NO_MATCH);
testScore('mandatory offered', requestFive, offerTwo, 0.6000000000000001);
testScore('mandatory offered', requestFive, offerTwo, 0.3);
});

describe('Matching Score Waiting Bonus', () => {
testScore(
'pupil waiting for less than 14 days',
{ ...requestOne, requestAt: moment(TODAY_TEST_DATE).subtract(10, 'days').toDate() },
offerOne,
0.4499863806393892
);
testScore(
'pupil waiting for more than 14 days',
{ ...requestOne, requestAt: moment(TODAY_TEST_DATE).subtract(20, 'days').toDate() },
offerOne,
0.7499999987633078
);
});

describe('Matching Basics', () => {
Expand Down
6 changes: 3 additions & 3 deletions common/match/matching.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,11 +202,11 @@ export function matchScore(request: MatchRequest, offer: MatchOffer, currentDate
}
}

const offerWaitDays = (+currentDate - +offer.requestAt) / MS_PER_DAY;
const offerWaitingBonus = offerWaitDays > 20 ? sigmoid(offerWaitDays - 20) : 0;
const requestWaitDays = (+currentDate - +request.requestAt) / MS_PER_DAY;
const requestWaitingBonus = requestWaitDays >= 14 ? sigmoid(requestWaitDays) : sigmoid(requestWaitDays) * 0.5;

// how good a match is in (0, 1)
const score = 0.8 * subjectBonus + 0.05 * languageBonus /* + 0.02 * stateBonus + */ + 0.2 * offerWaitingBonus;
const score = 0.3 * subjectBonus + 0.1 * languageBonus + 0.6 * requestWaitingBonus;

// TODO: Fix retention for matches with only few subjects (e.g. both helper and helpee only have math as subject)
// in that case the score is not so high, and thus they are retained for a long time, although the match is perfect
Expand Down