Skip to content

Conversation

@shivasurya
Copy link
Owner

This PR implements comprehensive relative import resolution for Python using a 3-pass algorithm. It extends the import extraction system to handle Python's relative import syntax with dot notation.

Key Changes:

  1. Added FileToModule reverse mapping to ModuleRegistry

    • Enables O(1) lookup from file path to module path
    • Required for resolving relative imports
    • Updated AddModule() to maintain bidirectional mapping
  2. Implemented resolveRelativeImport() function

    • Handles single dot (.) for current package
    • Handles multiple dots (.., ...) for parent/grandparent packages
    • Navigates package hierarchy using module path components
    • Clamps excessive dots to root package level
    • Falls back gracefully when file not in registry
  3. Enhanced processImportFromStatement() for relative imports

    • Detects relative_import nodes in tree-sitter AST
    • Extracts import_prefix (dots) and optional module suffix
    • Resolves relative paths to absolute module paths before adding to ImportMap
  4. Comprehensive test coverage (94.5% overall)

    • Unit tests for resolveRelativeImport with various dot counts
    • Integration tests with ExtractImports
    • Tests for deeply nested packages
    • Tests for mixed absolute and relative imports
    • Real fixture files with project structure

Relative Import Examples:

  • from . import utils → "currentpackage.utils"
  • from .. import config → "parentpackage.config"
  • from ..utils import helper → "parentpackage.utils.helper"
  • from ...db import query → "grandparent.db.query"

Test Fixtures:

  • Created myapp/submodule/handler.py with all relative import styles
  • Created supporting package structure with init.py files
  • Tests verify correct resolution across package hierarchy

Checklist:

  • Tests passing (gradle testGo)?
  • Lint passing (golangci-lint run this requires golangci-lint)?

shivasurya and others added 4 commits October 25, 2025 22:47
Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@shivasurya shivasurya self-assigned this Oct 26, 2025
@shivasurya shivasurya added enhancement New feature or request go Pull requests that update go code labels Oct 26, 2025
@safedep
Copy link

safedep bot commented Oct 26, 2025

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

This report is generated by SafeDep Github App

@shivasurya shivasurya changed the title feat: Implement relative import resolution - Pass 2 Part B cpf/enhancement: Implement relative import resolution Oct 26, 2025
@codecov
Copy link

codecov bot commented Oct 26, 2025

Codecov Report

❌ Patch coverage is 96.29630% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.34%. Comparing base (5454e99) to head (722556b).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
sourcecode-parser/graph/callgraph/imports.go 96.15% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #325      +/-   ##
==========================================
+ Coverage   73.94%   74.34%   +0.40%     
==========================================
  Files          26       26              
  Lines        2675     2717      +42     
==========================================
+ Hits         1978     2020      +42     
  Misses        647      647              
  Partials       50       50              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Base automatically changed from shiva/callgraph-infra-3 to main October 29, 2025 02:12
@shivasurya shivasurya merged commit 7ba6de4 into main Oct 29, 2025
5 checks passed
@shivasurya shivasurya deleted the shiva/callgraph-infra-4 branch October 29, 2025 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants