Expression Builder API feedback

[ethanresnick]

Thanks again for all the great work you're doing here! I just started using the new(ish) ExpressionBuilder API, and wanted to give a bit of feedback, which I hope will be helpful.

In general, I think the new design makes a lot of sense, and the new names apply better in the various contexts where expressions can be used than the old ones did. My only suggestion has to do with the abbreviated method names cmpr and bxp. Two things struck me about these:

1. Using abbreviations felt a bit out of place with the rest of the kysely API, which generally seems favors explicitness and readability over maximum concision (e.g., selectFrom rather than from, or something even shorter). To me, the three letters saved by using cmpr as opposed to compare doesn't seem worth it, given that cmpr is not an especially familiar abbreviation.

2. However, if the expression builder API is going to use abbreviations — and I can see a case for that in this API — then I think it might be worth adding abbreviated methods for specific, common binary comparisons, like eq, neq, gt, gte, etc. Having methods like that is, to me, both more intuitive (because eq is a much more common/familiar abbreviation than cmpr) and more concise (since the operator can be omitted).

So, I guess the above suggest three possible APIs:

1.  qb.where(({ compare, and }) => and([
      compare('first_name', '=', 'Jennifer'),
      compare('last_name', '!=', 'Tremblay'),
      compare('updated_at', '>', lastMonth)
    ])

2.  qb.where(({ cmpr, and }) => and([
      cmpr('first_name', '=', 'Jennifer'),
      cmpr('last_name', '!=', 'Tremblay'),
      cmpr('updated_at', '>', lastMonth)
    ])

3.  qb.where(({ eq, neq, gt, and }) => and([
      eq('first_name', 'Jennifer'),
      neq('last_name', 'Tremblay'),
      gt('updated_at', lastMonth)
    ])

Personally, I think I prefer (1), for maximum clarity without too much verbosity. But, if maximal concision is the goal, then (3) makes more sense to me than two.

[igalklebanov]

Hey 👋

Thank you for taking the time to write this.

We went with such abbreviations because:

1. we didn't want these names to collide with SQL keywords, known and unknown. e.g. compare sounds like something some dialect might have, either full-match, or as part of a built-in function name.
2. we wanted the function names to "get out of the way" when reading/writing the code. The most important part of a cmpr(...) is its arguments, which should read like SQL to align with WYSIWYG. We don't need a fully qualified compare to realize there's a comparison going on, the arguments speak for themselves.

We thought about eq, neq, etc. and decided to not introduce these:

1. They're not aligned with WYSIWYG.
2. More code to maintain for ",'=',".length less characters you type/"copilot"?
3. They're redundant. Too many options to choose from that do the same thing.

[ethanresnick]

Thanks @igalklebanov! I think I agree with not adding eq, neq, etc... I'm curious if you all considered something like this?

 qb.where(({ $, and }) => and([
   $('first_name', '=', 'Jennifer'),
   $('last_name', '!=', 'Tremblay'),
   $('updated_at', '>', lastMonth)
 ]))

db.updateTable('person')
  .set(({ $ }) => ({  
     age: $('age', '+', 1)
  }))

I'm not necessarily arguing for that, but it would seem to be the logical extension of the current design.

I.e., some of the downsides with cmpr/bxp are that they are not names that a user would know to search for (e.g., in the API docs) or that they'd necessarily recognize as being the function that they're looking for (e.g., in an autocomplete dropdown). $ has both of those downsides, but then seems to do better at "getting out of the way" and letting the arguments speak for themselves.

In the above example, $ would probably be a shorthand for bxp, as bxp seems like the more general one, which people could use almost all the time (unless they run into TS performance/type depth issues).

[koskimas]

If we renamed cmpr to compare, what would bxp be renamed to? binaryExpression? That's way too long and would dominate the expressions. I think we should either abbreviate all or none of them.

We need separate binary expression and a comparison expression. If we only had bxp, you couldn't use an unknown operator without specifying the output type explictly.

cmpr produces a boolean expression by default
bxp produces an expression that has the same type as the operands by default

$ is already reserved for stuff that's not related to SQL ($call, $if, etc).

Another reason for the abbreviated names was that people won't find the methods by searching no matter what we name them. They should be discovered from examples. The API documentation is filled with examples (which people don't seem to find). The new site will have those examples in a more accessible/searchable place.

[koskimas]

We could replace bxp and cmpr with a single function if we accept that using custom operators needs an explicit output type. We support most of the operators the built-in dialects have, but third party dialects might suffer.

I kind of like the idea of replacing both with $ but unfortunately that's not an option as stated in the previous comment. What else could we use?

[ethanresnick]

What else could we use?

Hmm…I think the obvious candidate would be _, since that’s the only other legal non-alphabetic character to start an identifier with, but that feels pretty overloaded to me in backend JS land (with lodash and the like) and also a bit hard to type constantly. So maybe better would be a very short alphabetic string that gestures at ‘expression’…maybe just e?

qb.where(({ e, and }) => and([
   e('first_name', '=', 'Jennifer'),
   e('last_name', '!=', 'Tremblay'),
   e('updated_at', '>', lastMonth)
 ]))

db.updateTable('person')
  .set(({ e }) => ({  
     age: e('age', '+', 1)
  }))

Or possibly xp, which sorta sounds like expression?

qb.where(({ xp, and }) => and([
   xp('first_name', '=', 'Jennifer'),
   xp('last_name', '!=', 'Tremblay'),
   xp('updated_at', '>', lastMonth)
 ]))

db.updateTable('person')
  .set(({ xp }) => ({  
     age: xp('age', '+', 1)
  }))

Could also do just x, which also kinda sounds like expression and has the secondary meaning of ‘a generic placeholder name’, which kinda applies here.

qb.where(({ x, and }) => and([
   x('first_name', '=', 'Jennifer'),
   x('last_name', '!=', 'Tremblay'),
   x('updated_at', '>', lastMonth)
 ]))

db.updateTable('person')
  .set(({ x }) => ({  
     age: x('age', '+', 1)
  }))

[ethanresnick]

Another direction would be is, which I actually quite like…makes sense in the boolean context especially, but works decently in other contexts too:

qb.where(({ is, and }) => and([
   is('first_name', '=', 'Jennifer'),
   is('last_name', '!=', 'Tremblay'),
   is('updated_at', '>', lastMonth)
 ]))

db.updateTable('person')
  .set(({ is }) => ({  
     age: is('age', '+', 1)
  }))

I think that’s my favorite by far: it’s actually a full word, while still being shorter than the current names. If we don’t like is in the non-boolean contexts, we could also make is the replacement for cmpr and think of something different for bxp

[ethanresnick]

I suspect an objection to is will be that it could be confused with the SQL operator, and I agree that’s an issue.

So here’s one more option: get, which is also a full and appropriate word, and is short enough that, with a two argument overload, it might be able to replace unary and all the shorthand methods (not, exists, neg) as well.

qb.where(({ get, and }) => and([
   get('first_name', '=', 'Jennifer'),
   get('last_name', '!=', 'Tremblay'),
   get('updated_at', '>', lastMonth),
   get('not', 'is_deleted')
 ]))

db.updateTable('person')
  .set(({ get }) => ({  
     age: get('age', '+', 1),
     backwards: get('-', 'age')
  }))

Of course, we can mix and match between all these options (eg, unifying cmpr, bxp, and unary, but calling that unified method e or exp or whatever).

I’m also not sure about the compiler performance hit of unifying unary with the other methods via a two-argument overload, but it does seem nice from an API POV.

[igalklebanov]

Left field idea:

eb.b('ref', '+', 5) // `b` for binary.
eb.b01('ref', '=', 'moshe') // cmpr replacement. `b` for binary, `01` for SqlBool hint :)

[ethanresnick]

Hmm, I think I'd rather try to unify bxp and unary, in which case leaning into "binary" in the name doesn't make sense. But, if that unification doesn't work for some reason, then b seems like a decent option. I'm not a fan of the 01, though: I think kysely should infer the return type from the operator (at least for the built-in operators, which seem like they cover 95%+ of the usage).

[igalklebanov]

Another one:

eb('ref', '+', 5)
eb('ref', '=', 'moshe')

[ethanresnick]

Love that!

Although it does mean that every other property (and, or, fn) can't be destructured.... assuming eb is the whole builder object, and not a function called eb on the builder. But maybe we could pass the builder twice?

[igalklebanov]

@ethanresnick

Love that!

Although it does mean that every other property (and, or, fn) can't be destructured.... assuming eb is the whole builder object,
and not a function called eb on the builder. But maybe we could pass the builder twice?

Yeah that could work.

interface EB {
  (): {}
  eb: Omit<EB, 'eb'>
  fn: () => {}
  ...
}

({ eb, fn }) => ...

[ethanresnick]

I'm loving this!

But why not:

interface EB {
  (): {}
  eb: EB // no omit here
  fn: () => {}
  ...
}

To me, it makes sense that the eb property should just be a reference to the full EB object (since that's what it's named after, after all).

Combining that with unifying bxp/unary/cmpr and we'd have:

qb.where(({ eb, and }) => and([
   eb('first_name', '=', 'Jennifer'),
   eb('last_name', '!=', 'Tremblay'),
   eb('updated_at', '>', lastMonth),
   eb('not', 'is_deleted')
 ]))

db.updateTable('person')
  .set(eb => ({  
     age: eb('age', '+', 1),
     backwards: eb('-', 'age')
  }))