Skip to content

Latest commit

 

History

History
291 lines (211 loc) · 9.5 KB

BUILD.md

File metadata and controls

291 lines (211 loc) · 9.5 KB

Building Rerun

This is a guide to how to build Rerun.

See also

Getting started with the repository

First, install the Rust toolchain using the installer from https://rustup.rs/.

Then, clone the repository:

git clone [email protected]:rerun-io/rerun.git
cd rerun

Now install the pixi package manager: https://github.com/prefix-dev/pixi?tab=readme-ov-file#installation

Make sure cargo --version prints 1.88.0 once you are done.

If you are using an Apple-silicon Mac (M1, M2), make sure rustc -vV outputs host: aarch64-apple-darwin. If not, this should fix it:

rustup set default-host aarch64-apple-darwin && rustup install 1.88.0

Git-lfs

We use git-lfs to store big files in the repository, such as UI test snapshots. We aim to keep this project buildable without the need of git-lfs (for example, icons and similar assets are checked in to the repo as regular files). However, git-lfs is generally required for a proper development environment, e.g. to run tests.

Setting up git-lfs

The TL;DR is to install git-lfs via your favorite package manager (apt, Homebrew, MacPorts, etc.) and run git lfs install. See the many resources available online more details.

You can ensure that everything is correctly installed by running git lfs ls-files from the repository root. It should list some test snapshot files.

Useful git-lfs commands

# Install git-lfs in the repo (installs git hooks)
git lfs install

# Move a file to git lfs
git lfs track "path/to/file/or/pattern" # OR manually edit .gitattributes
git add --renormalize . # Moves already added files to lfs (according to .gitattributes)

# Move a file from lfs to regular git
git lfs untrack "path/to/file/or/pattern" # OR manually edit .gitattributes
git add --renormalize . # Moves already added files to regular git (according to .gitattributes)

# Push to a contributor remote (see https://github.com/cli/cli/discussions/8794#discussioncomment-8695076)
git push --no-verify

# Push git lfs files to contributor remote:
git push origin $(git branch --show-current) && git push --no-verify && git push origin --delete $(git branch --show-current)

See also this section in the egui docs.

Validating your environment

You can validate your environment is set up correctly by running:

pixi run check-env

Building and running the Viewer

Use this command for building and running the viewer:

pixi run rerun

Running the Rust examples

All Rust examples are set up as separate executables, so they can be run by specifying the corresponding package, for example:

cargo run -p dna

They will either connect to an already running rerun viewer, or spawn a new one. In debug builds, it will spawn target/debug/rerun if it exists, otherwise look for rerun on PATH.

Building and installing the Rerun Python SDK

Rerun is available as a package on PyPi and can be installed with pip install rerun-sdk.

Additionally, nightly dev wheels from head of main are available at https://github.com/rerun-io/rerun/releases/tag/prerelease.

If you want to build from source, you can do so easily in the Pixi environment:

  • Run pixi run py-build --release to build SDK & Viewer for Python (or pixi run py-build for a debug build)
  • Then you can run examples from the repository, either by making the Pixi shell active with pixi shell and then running Python or by using pixi run, e.g. pixi run Python examples/python/minimal/minimal.py

Tests & tooling

# Run the unit tests
pixi run py-test

# Run the linting checks
pixi run py-lint

# Run the formatter
pixi run py-fmt

Building an installable Python wheel

The py-build-wheels-sdk-only command builds a whl file:

pixi run py-build-wheels-sdk-only

Which you can then install in your own Python environment:

pip install ./dist/CURRENT_ARCHITECTURE/*.whl

IMPORTANT: unlike the official wheels, wheels produced by this method do not contain the viewer, so they may only be used for logging purposes.

Building and installing the Rerun C++ SDK

On Windows you have to have a system install of Visual Studio 2022 in order to compile the SDK and samples.

All other dependencies are downloaded by Pixi! You can run tests with:

pixi run -e cpp cpp-test

and build all C++ artifacts with:

pixi run -e cpp cpp-build-all

Building the docs

High-level documentation for Rerun can be found at http://rerun.io/docs. It is built from the separate repository rerun-docs.

Building for the web

If you want to build a standalone Rerun executable that contains the web-viewer and a gRPC server, you need to install the wasm32-unknown-unknown Rust target and ensure the web_viewer feature flag is set when building rerun. This is automatically done by this shortcut which builds & runs the web viewer:

pixi run rerun-web

If you're on Windows you have to make sure that your git client creates symlinks, otherwise you may get errors during the build. Run git config --show-scope --show-origin core.symlinks to check if symlinks are enabled. You may need to turn on Windows developer mode in order to give the mklink command sufficient permissions. See also this Stack Overflow reply on the issue.

Improving compile times

As of today, we link everything statically in both debug and release builds, which makes custom linkers and split debuginfo the two most impactful tools we have at our disposal in order to improve compile times.

These tools can be configured through your Cargo configuration, available at $HOME/.cargo/config.toml.

macOS

On x64 macOS, use the zld linker and keep debuginfo in a single separate file.

Pre-requisites:

  • Install zld: brew install michaeleisel/zld/zld.

config.toml (x64):

[target.x86_64-apple-darwin]
rustflags = [
    "-C",
    "link-arg=-fuse-ld=/usr/local/bin/zld",
    "-C",
    "split-debuginfo=packed",
]

On Apple-silicon Mac (M1, M2), the default settings are already pretty good. The default linker is just as good as zld. Do NOT set split-debuginfo=packed, as that will make linking a lot slower. You can set split-debuginfo=unpacked for a small improvement.

config.toml (M1, M2):

[target.aarch64-apple-darwin]
rustflags = [
    "-C",
    "split-debuginfo=unpacked",
]

Linux

On Linux, use the mold linker and keep DWARF debuginfo in separate files.

Pre-requisites:

  • Install mold through your package manager.

config.toml:

[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = [
    "-C",
    "link-arg=-fuse-ld=/usr/bin/mold",
    "-C",
    "split-debuginfo=unpacked",
]

Windows

On Windows, use LLVM's lld linker and keep debuginfo in a single separate file.

Pre-requisites:

  • Install lld:
cargo install -f cargo-binutils
rustup component add llvm-tools-preview

config.toml:

[target.x86_64-pc-windows-msvc]
linker = "rust-lld.exe"
rustflags = [
    "-C",
    "split-debuginfo=packed",
]

Python build variables (macOS or linux)

When building the python package you can greatly speed up your rebuilding by defining a PYO3_CONFIG_FILE environment variable pointing to a file that contains your build environment. You will need to update this file if there are major changes to your build, including when there are version changes in the PyO3 dependencies.

To set this variable, first run the following command:

PYO3_PRINT_CONFIG=1 cargo build

It should include an output similar to the following:

implementation=CPython
version=3.9
shared=true
abi3=true
lib_name=python3.12
lib_dir=/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.12/lib
executable=/Users/myusername/src/datafusion-python/.venv/bin/python
pointer_width=64
build_flags=
suppress_build_script_link_lines=false

Copy the output in your terminal into a file and set this environment variable.

export PYO3_CONFIG_FILE="/Users/myusername/some/path/pyo3_build.config"

If you are using VS Code for your IDE you may need to set these variables in your settings:

"rust-analyzer.cargo.extraEnv": {
    "PYO3_CONFIG_FILE": "/Users/myusername/some/path/pyo3_build.config"
},
"rust-analyzer.runnables.extraEnv": {
    "PYO3_CONFIG_FILE": "/Users/myusername/some/path/pyo3_build.config"
}