fix: configure pre-commit.ci and clean up codebase formatting

[abhizipstack]

What

• Get pre-commit.ci fully working on the OSS repo with a clean baseline
• Fix pre-commit hook configurations for monorepo structure
• Clean up trailing whitespace and end-of-file issues across 90+ files

Why

• pre-commit.ci was failing due to incompatible hooks (docformatter python_venv, absolufy-imports breaking relative imports)
• Pre-existing whitespace/formatting violations across the codebase blocked CI hooks
• Need automated linting on PRs for code quality before public launch

How

• docformatter: Bumped v1.7.5 → v1.7.7 (fixes python_venv language error)
• Config paths: Point black, pycln, isort to backend/pyproject.toml
• Whitespace cleanup: Fixed trailing whitespace across 90+ files, end-of-file newlines across 50+ files
• yamllint: Fixed long line in docker-compose.yaml
• actionlint: Removed empty core-backend-full-tests-parallel.yaml
• SVG exclusion: Exclude SVG icons from whitespace hooks (design assets)
• Disabled autofix_prs: Prevent auto-commits that break code (absolufy-imports incident)
• Skipped hooks: Hooks with pre-existing violations (black, isort, pyupgrade, flake8, markdownlint, etc.) skipped in CI until dedicated cleanup

Active hooks in CI

trailing-whitespace, check-yaml, check-json, check-toml, check-ast, detect-private-key, check-merge-conflict, gitleaks, htmlhint, yamllint, actionlint

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

• No — only whitespace changes to existing files (no logic changes), CI config updates, and removal of an empty workflow file
• Reverted all SVG icon changes that were unnecessarily modified
• Reverted absolufy-imports auto-fixes that broke relative imports

Database Migrations

• None

Env Config

• None

Relevant Docs

• pre-commit.ci dashboard: https://results.pre-commit.ci/repo/github/Zipstack/visitran

Related Issues or PRs

• Unblocks PR fix: fix Minimal Tests workflow for OSS repo structure #18 (Minimal Tests workflow)
• Follow-up: OR-1405 (enable remaining skipped hooks)

Dependencies Versions

• docformatter v1.7.7 (bumped from v1.7.5)

Notes on Testing

• pre-commit.ci passes on this PR
• Server runs fine locally — verified
• No application code changes

Screenshots

N/A — CI config and whitespace changes only

Checklist

I have read and understood the Contribution Guidelines.

[greptile-apps]

Greptile Summary

This PR gets pre-commit.ci operational on the OSS repo through a series of targeted fixes to .pre-commit-config.yaml, accompanied by a large sweep of trailing-whitespace cleanup across 100+ files. The main structural changes are: adding 10 hooks to ci.skip to avoid pre-existing violations, correcting pyproject.toml config paths to backend/pyproject.toml, bumping docformatter to v1.7.7, excluding sample.env from the detect-private-key check, adding SVG exclusions, and deleting an entirely-commented-out workflow file. Integration test CSV files were also migrated from Git LFS pointers to inline data, making the OSS repo usable without LFS.

Key changes:

• .pre-commit-config.yaml: 10 hooks added to ci.skip; config paths fixed; autofix_prs changed to false; docformatter bumped to v1.7.7; detect-private-key excludes sample.env
• core-backend-full-tests-parallel.yaml: deleted (was entirely commented out, causing actionlint failures)
• validate_new_formulas.py: made executable (mode 0644 → 0755) to satisfy check-executables-have-shebangs
• rental.csv, film.csv, inventory.csv, store.csv: converted from Git LFS pointers to inline CSV data
• 100+ files: trailing-whitespace cleanup (blank lines with trailing spaces removed)

Issues found:

• yamllint is stated in the PR description as needing to be skipped (pre-existing line length violations) but is absent from ci.skip — it will run in CI and is likely to fail
• autofix_prs: false disables pre-commit.ci's auto-fix PR capability going forward; worth re-enabling once the cleanup work is done

Confidence Score: 4/5

Safe to merge after confirming yamllint is either passing or added to ci.skip — one P1 config gap could cause CI to fail immediately after merge

The P1 finding (yamllint missing from ci.skip despite pre-existing violations called out in the PR description) means CI may fail right after merge. All other changes are either correct config fixes, whitespace cleanup, or intentional policy decisions. No application logic was changed.

.pre-commit-config.yaml — confirm yamllint and actionlint handling matches intent

Important Files Changed

Filename	Overview
.pre-commit-config.yaml	Core config change: adds 10 hooks to ci.skip, fixes pyproject.toml config paths, bumps docformatter to v1.7.7, excludes sample.env from detect-private-key, adds SVG to excluded types — but yamllint and actionlint are missing from ci.skip despite being called out in the PR description, and autofix_prs is disabled
.github/workflows/core-backend-full-tests-parallel.yaml	File deleted — it was entirely commented out (87 lines of comments), likely causing actionlint failures; removal is appropriate cleanup
backend/formulasql/tests/validate_new_formulas.py	Mode change only (0644 → 0755) to make the script executable, fixing the check-executables-have-shebangs hook
backend/tests/integration_tests/data/sakila/rental.csv	Migrated from Git LFS pointer (3 lines) to inline CSV data (~1000 rows); similar migration done for film.csv, inventory.csv, and store.csv — makes the OSS repo usable without LFS
backend/backend/utils/decryption_utils.py	Whitespace-only changes: trailing spaces removed from blank lines inside functions and docstrings — no logic changes
docker/docker-compose.yaml	Whitespace/formatting only: long healthcheck test string split across two lines for readability

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[git push / PR] --> B[pre-commit.ci runs]
    B --> C{Hook in ci.skip?}
    C -- Yes --> D[Skip hook in CI\nstill runs locally]
    C -- No --> E[Hook executes in CI]
    E --> F{Hook passes?}
    F -- Yes --> G[CI green]
    F -- No --> H[CI red — no autofix PR\nautofix_prs: false]

    subgraph ci_skip [ci.skip list - new]
        S1[mypy] & S2[protolint-docker] & S3[hadolint-docker]
        S4[pycln] & S5[end-of-file-fixer] & S6[flake8]
        S7[markdownlint] & S8[absolufy-imports] & S9[black]
        S10[pyupgrade] & S11[isort] & S12[yesqa] & S13[docformatter]
    end

    subgraph running [Hooks running in CI]
        R1[trailing-whitespace]
        R2[check-yaml / check-json / check-toml]
        R3[detect-private-key]
        R4[gitleaks]
        R5[yamllint - pre-existing violations NOT skipped]
        R6[actionlint]
        R7[htmlhint]
    end

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: .pre-commit-config.yaml
Line: 6-19

Comment:
**`yamllint` and `actionlint` missing from `ci.skip`**

The PR description explicitly lists both `yamllint` (pre-existing line length violations) and `actionlint` (empty/malformed workflow file) as hooks that should be skipped in CI, but neither appears in the `ci.skip` block here. This means both hooks will be executed by pre-commit.ci.

- `yamllint` — acknowledged in the PR description as having pre-existing violations. It will run against all YAML files (including the YAML in this PR) and is very likely to fail CI.
- `actionlint` — the problematic fully-commented workflow file was deleted in this PR, so it may now pass; but the PR description suggests it should be skipped regardless.

```suggestion
  skip:
    - mypy # Uses language: system, not available in pre-commit.ci sandbox
    - protolint-docker # Needs Docker, not available in pre-commit.ci
    - hadolint-docker # Needs Docker, not available in pre-commit.ci
    - pycln # Path resolution bug in pre-commit.ci sandbox
    - end-of-file-fixer # Inconsistent behavior between local and CI environments
    - flake8 # Pre-existing violations — will clean up separately
    - markdownlint # Pre-existing violations — will clean up separately
    - absolufy-imports # Incorrectly rewrites relative imports in monorepo structure
    - black # Large-scale reformatting — will clean up in dedicated PR
    - pyupgrade # Pre-existing across 43 files
    - isort # Pre-existing import ordering across codebase
    - yesqa # Pre-existing unnecessary noqa comments
    - docformatter # Pre-existing docstring formatting
    - yamllint # Pre-existing line length violations — will clean up separately
    - actionlint # Verify actionlint passes after workflow file deletion, skip if not
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: .pre-commit-config.yaml
Line: 20

Comment:
**`autofix_prs` disabled removes CI auto-fix capability**

Changing `autofix_prs` from `true` to `false` means pre-commit.ci will no longer push fix commits when auto-fixable hooks (e.g., `trailing-whitespace`, `end-of-file-fixer`) detect violations. Going forward, CI will only report failures without ever applying the fix — contributors must run pre-commit locally to resolve them.

This may be intentional while the codebase is being cleaned up (to avoid noisy auto-fix PRs), but it reduces the main value proposition of pre-commit.ci. Consider re-enabling it once the bulk cleanup is done, or leave a comment documenting the intent to re-enable.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (11): Last reviewed commit: "fix: revert unnecessary whitespace chang..." | Re-trigger Greptile}

[Deepak-Gnanasekar]

⚠️ Merge Order Recommendation

This PR should be merged last, after all other currently open PRs are merged.

Why

This PR touches 331 files (mostly mechanical Black reformatting to line-length = 120). Merging it now will cause merge conflicts on every open PR that touches any of those ~265 Python files.

Suggested order:

1. Merge all other open PRs first
2. Rebase this PR on latest main
3. Re-run the formatters to pick up any new code from merged PRs
4. Merge this PR last

This avoids unnecessary rebase churn for the rest of the team.

[wicky-zipstack]

LGTM. Merge after PR #18 to avoid CI config conflicts.

[tahierhussain]

LGTM.

Hope this PR was tested locally, as it is very difficult to review a PR with 112 file changes.