DataFusion connector for setupset

This project add Datafusion support in superset. Leveraging the usage of parquet file, csv, for graphing

10 mins setup

docker compose up --build

Once superset is up, one can add a datafusion database, using the following connection string :

datafusion://app/setup.sql

setup.sql is a sql configuration to setup the datasources used by datafusion, this can be :


CREATE EXTERNAL TABLE users STORED AS PARQUET LOCATION 'users.parquet';

-- Set configuration
SET datafusion.execution.batch_size = 4096;

to configure a parquet access

git clone --recursive https://github.com/frett27/superset-datafusion

see more information for development deployment here :

./setup_superset_datafusion.sh

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
.github/workflows		.github/workflows
python		python
src		src
superset @ 3b8bc9b		superset @ 3b8bc9b
.gitignore		.gitignore
.gitmodules		.gitmodules
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
Dockerfile		Dockerfile
LICENSE		LICENSE
README-Docker.md		README-Docker.md
README.md		README.md
cleanup_and_restart.sh		cleanup_and_restart.sh
docker-compose.yml		docker-compose.yml
example.sql		example.sql
init_superset.sh		init_superset.sh
pyproject.toml		pyproject.toml
quick_restart.sh		quick_restart.sh
requierments.txt		requierments.txt
setup.sql		setup.sql
setup_superset_datafusion.sh		setup_superset_datafusion.sh
start_with_init.sh		start_with_init.sh
superset_config_datafusion.py		superset_config_datafusion.py
users.parquet		users.parquet