Port to more backends with narwhals?

[NickCrews]

Hi! I really want to use this tool. But, my data lives in a postgres database, and I want the queries to be pushed down into the backend, I don't want to have to materialize the entire result set, pass that to the the in-memory duckdb, and go from there. The rest of my codebase also is based around ibis, so it would be great if there was interoperability there.

Would it be possible to port the implementation of this to https://github.com/narwhals-dev/narwhals? After a 30 second browse of the codebase, it looks like the python layer here is quite small, it just creates a duckdb connection and then almost all of the meat is in the typescript layer. So my initial thought would be no, this wouldn't be possible. But I wanted to double check!

[domoritz]

This probably mostly depends on the support in mosaic. Right now, we generate SQL specific for DuckDB but quak uses fairly generic features so it might not be too difficult to support other databases (as long as the queries are SQL).

Related: uwdata/mosaic#399

[manzt]

Yup, quak is based on mosaic, dispatching DuckDB-flavored SQL from the front end to a DuckDB backend.

In theory, given that the SQL is pretty generic, it might be possible to pass the SQL through something like sqlglot for a subset of supported backends. Maybe ibis could help out there too?

EDIT: Just seeing uwdata/mosaic#399 (comment)