Skip to content

Conversation

@shivasurya
Copy link
Owner

First Huge WIN 🏅

Path-Finder Query Console:
>FIND method WHERE name = 'onCreate'
------Results------
@Override
    public void onCreate(SQLiteDatabase db) {
        db.execSQL(DATABASE_CREATE);
    }
@Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_movie_detail);
        Intent intent = getIntent();

        getSupportActionBar().setDisplayHomeAsUpEnabled(true);
        getSupportActionBar().setDisplayShowHomeEnabled(true);

        movieGeneralModal moviegeneralModal = (movieGeneralModal) intent.getSerializableExtra("DATA_MOVIE");

        if (savedInstanceState == null) {

            movieDetailFragment fragment = new movieDetailFragment();
            fragment.setMovieData(moviegeneralModal);
            getSupportFragmentManager().beginTransaction()
                    .add(R.id.movie_detail_container, fragment)
                    .commit();
        }
    }
-------------------

Path-Finder Query Console:
>FIND method WHERE name = 'onCreate'

yields results of methods named as oncreate
@shivasurya shivasurya self-assigned this Apr 19, 2024
@shivasurya shivasurya merged commit 4e6d01a into main Apr 19, 2024
@shivasurya shivasurya deleted the shiva/java-method-query-support branch April 19, 2024 16:51
shivasurya added a commit that referenced this pull request Oct 26, 2025
This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 26, 2025
This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 26, 2025
This PR completes the 3-pass algorithm for building Python call graphs
by implementing the final pass that resolves call targets and constructs
the complete graph structure with edges linking callers to callees.

## Changes

### Core Implementation (builder.go)

1. **BuildCallGraph()**: Main entry point for Pass 3
   - Indexes all function definitions from code graph
   - Iterates through all Python files in the registry
   - Extracts imports and call sites for each file
   - Resolves each call site to its target function
   - Builds edges and stores call site details
   - Returns complete CallGraph with all relationships

2. **indexFunctions()**: Function indexing
   - Scans code graph for all function/method definitions
   - Maps each function to its FQN using module registry
   - Populates CallGraph.Functions map for quick lookup

3. **getFunctionsInFile()**: File-scoped function retrieval
   - Filters code graph nodes by file path
   - Returns only function/method definitions in that file
   - Used for finding containing functions of call sites

4. **findContainingFunction()**: Call site parent resolution
   - Determines which function contains a given call site
   - Uses line number comparison with nearest-match algorithm
   - Finds function with highest line number ≤ call line
   - Returns empty string for module-level calls

5. **resolveCallTarget()**: Core resolution logic
   - Handles simple names: sanitize() → myapp.utils.sanitize
   - Handles qualified names: utils.sanitize() → myapp.utils.sanitize
   - Resolves through import maps first
   - Falls back to same-module resolution
   - Validates FQNs against module registry
   - Returns (FQN, resolved bool) tuple

6. **validateFQN()**: FQN validation
   - Checks if a fully qualified name exists in registry
   - Handles both modules and functions within modules
   - Validates parent module for function FQNs

7. **readFileBytes()**: File reading helper
   - Reads source files for parsing
   - Handles absolute path conversion

### Comprehensive Tests (builder_test.go)

Created 15 test functions covering:

**Resolution Tests:**
- Simple imported function resolution
- Qualified import resolution (module.function)
- Same-module function resolution
- Unresolved method calls (obj.method)
- Non-existent function handling

**Validation Tests:**
- Module existence validation
- Function-in-module validation
- Non-existent module handling

**Helper Function Tests:**
- Function indexing from code graph
- Functions-in-file filtering
- Containing function detection with edge cases

**Integration Tests:**
- Simple single-file call graph
- Multi-file call graph with imports
- Real test fixture integration

## Test Coverage

- Overall: 91.8%
- BuildCallGraph: 80.8%
- indexFunctions: 87.5%
- getFunctionsInFile: 100.0%
- findContainingFunction: 100.0%
- resolveCallTarget: 85.0%
- validateFQN: 100.0%
- readFileBytes: 75.0%

## Algorithm Overview

Pass 3 ties together all previous work:

### Pass 1 (PR #2): BuildModuleRegistry
- Maps file paths to module paths
- Enables FQN generation

### Pass 2 (PRs #3-5): Import & Call Site Extraction
- ExtractImports: Maps local names to FQNs
- ExtractCallSites: Finds all function calls in AST

### Pass 3 (This PR): Call Graph Construction
- Resolves call targets using import maps
- Links callers to callees with edges
- Validates resolutions against registry
- Stores detailed call site information

## Resolution Strategy

The resolver uses a multi-step approach:

1. **Simple names** (no dots):
   - Check import map first
   - Fall back to same-module lookup
   - Return unresolved if neither works

2. **Qualified names** (with dots):
   - Split into base + rest
   - Resolve base through imports
   - Append rest to get full FQN
   - Try current module if not imported

3. **Validation**:
   - Check if target exists in registry
   - For functions, validate parent module exists
   - Mark resolution success/failure

## Design Decisions

1. **Containing function detection**:
   - Uses nearest-match algorithm based on line numbers
   - Finds function with highest line number ≤ call line
   - Handles module-level calls by returning empty FQN

2. **Resolution priority**:
   - Import map takes precedence over same-module
   - Explicit imports always respected even if unresolved
   - Same-module only tried when not in imports

3. **Validation vs Resolution**:
   - Resolution finds FQN from imports/context
   - Validation checks if FQN exists in registry
   - Both pieces of information stored in CallSite

4. **Error handling**:
   - Continues processing even if some files fail
   - Marks individual call sites as unresolved
   - Returns partial graph instead of failing completely

## Next Steps

The call graph infrastructure is now complete. Future PRs will:

- PR #7: Add CFG data structures for control flow analysis
- PR #8: Implement pattern matching for security rules
- PR #9: Integrate into main initialization pipeline
- PR #10: Add comprehensive documentation and examples
- PR #11: Performance optimizations (caching, pooling)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call site extraction from AST - Pass 2 Part C

This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call site extraction from AST - Pass 2 Part C

This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call graph builder - Pass 3

This PR completes the 3-pass algorithm for building Python call graphs
by implementing the final pass that resolves call targets and constructs
the complete graph structure with edges linking callers to callees.

## Changes

### Core Implementation (builder.go)

1. **BuildCallGraph()**: Main entry point for Pass 3
   - Indexes all function definitions from code graph
   - Iterates through all Python files in the registry
   - Extracts imports and call sites for each file
   - Resolves each call site to its target function
   - Builds edges and stores call site details
   - Returns complete CallGraph with all relationships

2. **indexFunctions()**: Function indexing
   - Scans code graph for all function/method definitions
   - Maps each function to its FQN using module registry
   - Populates CallGraph.Functions map for quick lookup

3. **getFunctionsInFile()**: File-scoped function retrieval
   - Filters code graph nodes by file path
   - Returns only function/method definitions in that file
   - Used for finding containing functions of call sites

4. **findContainingFunction()**: Call site parent resolution
   - Determines which function contains a given call site
   - Uses line number comparison with nearest-match algorithm
   - Finds function with highest line number ≤ call line
   - Returns empty string for module-level calls

5. **resolveCallTarget()**: Core resolution logic
   - Handles simple names: sanitize() → myapp.utils.sanitize
   - Handles qualified names: utils.sanitize() → myapp.utils.sanitize
   - Resolves through import maps first
   - Falls back to same-module resolution
   - Validates FQNs against module registry
   - Returns (FQN, resolved bool) tuple

6. **validateFQN()**: FQN validation
   - Checks if a fully qualified name exists in registry
   - Handles both modules and functions within modules
   - Validates parent module for function FQNs

7. **readFileBytes()**: File reading helper
   - Reads source files for parsing
   - Handles absolute path conversion

### Comprehensive Tests (builder_test.go)

Created 15 test functions covering:

**Resolution Tests:**
- Simple imported function resolution
- Qualified import resolution (module.function)
- Same-module function resolution
- Unresolved method calls (obj.method)
- Non-existent function handling

**Validation Tests:**
- Module existence validation
- Function-in-module validation
- Non-existent module handling

**Helper Function Tests:**
- Function indexing from code graph
- Functions-in-file filtering
- Containing function detection with edge cases

**Integration Tests:**
- Simple single-file call graph
- Multi-file call graph with imports
- Real test fixture integration

## Test Coverage

- Overall: 91.8%
- BuildCallGraph: 80.8%
- indexFunctions: 87.5%
- getFunctionsInFile: 100.0%
- findContainingFunction: 100.0%
- resolveCallTarget: 85.0%
- validateFQN: 100.0%
- readFileBytes: 75.0%

## Algorithm Overview

Pass 3 ties together all previous work:

### Pass 1 (PR #2): BuildModuleRegistry
- Maps file paths to module paths
- Enables FQN generation

### Pass 2 (PRs #3-5): Import & Call Site Extraction
- ExtractImports: Maps local names to FQNs
- ExtractCallSites: Finds all function calls in AST

### Pass 3 (This PR): Call Graph Construction
- Resolves call targets using import maps
- Links callers to callees with edges
- Validates resolutions against registry
- Stores detailed call site information

## Resolution Strategy

The resolver uses a multi-step approach:

1. **Simple names** (no dots):
   - Check import map first
   - Fall back to same-module lookup
   - Return unresolved if neither works

2. **Qualified names** (with dots):
   - Split into base + rest
   - Resolve base through imports
   - Append rest to get full FQN
   - Try current module if not imported

3. **Validation**:
   - Check if target exists in registry
   - For functions, validate parent module exists
   - Mark resolution success/failure

## Design Decisions

1. **Containing function detection**:
   - Uses nearest-match algorithm based on line numbers
   - Finds function with highest line number ≤ call line
   - Handles module-level calls by returning empty FQN

2. **Resolution priority**:
   - Import map takes precedence over same-module
   - Explicit imports always respected even if unresolved
   - Same-module only tried when not in imports

3. **Validation vs Resolution**:
   - Resolution finds FQN from imports/context
   - Validation checks if FQN exists in registry
   - Both pieces of information stored in CallSite

4. **Error handling**:
   - Continues processing even if some files fail
   - Marks individual call sites as unresolved
   - Returns partial graph instead of failing completely

## Next Steps

The call graph infrastructure is now complete. Future PRs will:

- PR #7: Add CFG data structures for control flow analysis
- PR #8: Implement pattern matching for security rules
- PR #9: Integrate into main initialization pipeline
- PR #10: Add comprehensive documentation and examples
- PR #11: Performance optimizations (caching, pooling)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
…#328)

* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call site extraction from AST - Pass 2 Part C

This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call graph builder - Pass 3

This PR completes the 3-pass algorithm for building Python call graphs
by implementing the final pass that resolves call targets and constructs
the complete graph structure with edges linking callers to callees.

## Changes

### Core Implementation (builder.go)

1. **BuildCallGraph()**: Main entry point for Pass 3
   - Indexes all function definitions from code graph
   - Iterates through all Python files in the registry
   - Extracts imports and call sites for each file
   - Resolves each call site to its target function
   - Builds edges and stores call site details
   - Returns complete CallGraph with all relationships

2. **indexFunctions()**: Function indexing
   - Scans code graph for all function/method definitions
   - Maps each function to its FQN using module registry
   - Populates CallGraph.Functions map for quick lookup

3. **getFunctionsInFile()**: File-scoped function retrieval
   - Filters code graph nodes by file path
   - Returns only function/method definitions in that file
   - Used for finding containing functions of call sites

4. **findContainingFunction()**: Call site parent resolution
   - Determines which function contains a given call site
   - Uses line number comparison with nearest-match algorithm
   - Finds function with highest line number ≤ call line
   - Returns empty string for module-level calls

5. **resolveCallTarget()**: Core resolution logic
   - Handles simple names: sanitize() → myapp.utils.sanitize
   - Handles qualified names: utils.sanitize() → myapp.utils.sanitize
   - Resolves through import maps first
   - Falls back to same-module resolution
   - Validates FQNs against module registry
   - Returns (FQN, resolved bool) tuple

6. **validateFQN()**: FQN validation
   - Checks if a fully qualified name exists in registry
   - Handles both modules and functions within modules
   - Validates parent module for function FQNs

7. **readFileBytes()**: File reading helper
   - Reads source files for parsing
   - Handles absolute path conversion

### Comprehensive Tests (builder_test.go)

Created 15 test functions covering:

**Resolution Tests:**
- Simple imported function resolution
- Qualified import resolution (module.function)
- Same-module function resolution
- Unresolved method calls (obj.method)
- Non-existent function handling

**Validation Tests:**
- Module existence validation
- Function-in-module validation
- Non-existent module handling

**Helper Function Tests:**
- Function indexing from code graph
- Functions-in-file filtering
- Containing function detection with edge cases

**Integration Tests:**
- Simple single-file call graph
- Multi-file call graph with imports
- Real test fixture integration

## Test Coverage

- Overall: 91.8%
- BuildCallGraph: 80.8%
- indexFunctions: 87.5%
- getFunctionsInFile: 100.0%
- findContainingFunction: 100.0%
- resolveCallTarget: 85.0%
- validateFQN: 100.0%
- readFileBytes: 75.0%

## Algorithm Overview

Pass 3 ties together all previous work:

### Pass 1 (PR #2): BuildModuleRegistry
- Maps file paths to module paths
- Enables FQN generation

### Pass 2 (PRs #3-5): Import & Call Site Extraction
- ExtractImports: Maps local names to FQNs
- ExtractCallSites: Finds all function calls in AST

### Pass 3 (This PR): Call Graph Construction
- Resolves call targets using import maps
- Links callers to callees with edges
- Validates resolutions against registry
- Stores detailed call site information

## Resolution Strategy

The resolver uses a multi-step approach:

1. **Simple names** (no dots):
   - Check import map first
   - Fall back to same-module lookup
   - Return unresolved if neither works

2. **Qualified names** (with dots):
   - Split into base + rest
   - Resolve base through imports
   - Append rest to get full FQN
   - Try current module if not imported

3. **Validation**:
   - Check if target exists in registry
   - For functions, validate parent module exists
   - Mark resolution success/failure

## Design Decisions

1. **Containing function detection**:
   - Uses nearest-match algorithm based on line numbers
   - Finds function with highest line number ≤ call line
   - Handles module-level calls by returning empty FQN

2. **Resolution priority**:
   - Import map takes precedence over same-module
   - Explicit imports always respected even if unresolved
   - Same-module only tried when not in imports

3. **Validation vs Resolution**:
   - Resolution finds FQN from imports/context
   - Validation checks if FQN exists in registry
   - Both pieces of information stored in CallSite

4. **Error handling**:
   - Continues processing even if some files fail
   - Marks individual call sites as unresolved
   - Returns partial graph instead of failing completely

## Next Steps

The call graph infrastructure is now complete. Future PRs will:

- PR #7: Add CFG data structures for control flow analysis
- PR #8: Implement pattern matching for security rules
- PR #9: Integrate into main initialization pipeline
- PR #10: Add comprehensive documentation and examples
- PR #11: Performance optimizations (caching, pooling)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Create CFG data structures for control flow analysis

This PR implements Control Flow Graph (CFG) data structures to enable
intra-procedural analysis of execution paths through functions. CFGs are
essential for security analysis patterns like taint tracking and detecting
missing sanitization on all paths.

## Changes

### Core Implementation (cfg.go)

1. **BlockType**: Enumeration of basic block types
   - Entry: Function entry point
   - Exit: Function exit point
   - Normal: Sequential execution block
   - Conditional: Branch blocks (if/else)
   - Loop: Loop header blocks (while/for)
   - Switch: Switch/match statement blocks
   - Try/Catch/Finally: Exception handling blocks

2. **BasicBlock**: Represents a single basic block
   - ID: Unique identifier within CFG
   - Type: Block category for analysis
   - StartLine/EndLine: Source code location
   - Instructions: CallSites occurring in this block
   - Successors: Blocks that can execute next
   - Predecessors: Blocks that can execute before
   - Condition: Condition expression (for conditional blocks)
   - Dominators: Blocks that always execute before this one

3. **ControlFlowGraph**: Complete CFG for a function
   - FunctionFQN: Fully qualified function name
   - Blocks: Map of block ID to BasicBlock
   - EntryBlockID/ExitBlockID: Special block identifiers
   - CallGraph: Reference for inter-procedural analysis

4. **CFG Operations**:
   - NewControlFlowGraph(): Creates CFG with entry/exit blocks
   - AddBlock(): Adds basic block to CFG
   - AddEdge(): Connects blocks with control flow edges
   - GetBlock(): Retrieves block by ID
   - GetSuccessors(): Returns successor blocks
   - GetPredecessors(): Returns predecessor blocks

5. **Dominator Analysis**:
   - ComputeDominators(): Calculates dominator sets using iterative data flow
   - IsDominator(): Checks if one block dominates another
   - Used to verify sanitization always occurs before usage

6. **Path Analysis**:
   - GetAllPaths(): Enumerates all execution paths from entry to exit
   - dfsAllPaths(): DFS-based path enumeration
   - Used for exhaustive security analysis

7. **Helper Functions**:
   - intersect(): Set intersection for dominator computation
   - slicesEqual(): Compare string slices for fixed-point detection

### Comprehensive Tests (cfg_test.go)

Created 23 test functions covering:

**Construction Tests:**
- CFG creation with entry/exit blocks
- Basic block creation with all fields
- Block addition to CFG

**Edge Management Tests:**
- Adding edges between blocks
- Duplicate edge handling
- Non-existent block edge handling

**Graph Navigation Tests:**
- Block retrieval by ID
- Successor block retrieval
- Predecessor block retrieval

**Dominator Analysis Tests:**
- Linear CFG dominators (A→B→C)
- Branching CFG dominators (if/else merge)
- Dominator checking

**Path Analysis Tests:**
- All paths in linear CFG
- All paths in branching CFG

**Helper Function Tests:**
- Set intersection operations
- Slice equality checking

**Complex Integration Test:**
- Realistic function CFG with branches
- Multiple blocks and paths
- Dominator relationships verification

## Test Coverage

- Overall: 92.7%
- NewControlFlowGraph: 100.0%
- AddBlock: 100.0%
- AddEdge: 100.0%
- GetBlock: 100.0%
- GetSuccessors: 87.5%
- GetPredecessors: 87.5%
- ComputeDominators: 100.0%
- IsDominator: 75.0%
- GetAllPaths: 100.0%
- dfsAllPaths: 91.7%
- intersect: 100.0%
- slicesEqual: 100.0%

## Design Decisions

1. **Entry/Exit blocks always created**:
   - Simplifies analysis by providing single entry/exit points
   - Standard CFG construction practice

2. **Dominator computation uses iterative algorithm**:
   - Simple fixed-point iteration
   - Converges quickly for most real-world CFGs
   - More efficient than other dominator algorithms for small graphs

3. **Path enumeration with cycle detection**:
   - Avoids infinite loops in cyclic CFGs
   - Uses visited tracking during DFS
   - WARNING: Can be exponential for complex CFGs

4. **Blocks store CallSites as instructions**:
   - Links CFG to call graph for inter-procedural analysis
   - Enables tracking tainted data through function calls

5. **Condition stored as string**:
   - Simple representation for conditional blocks
   - Could be enhanced with AST expression nodes later

## Use Cases

CFGs enable several security analysis patterns:

**Taint Analysis:**
- Track data flow through execution paths
- Detect if tainted data reaches sensitive sinks

**Sanitization Verification:**
- Use dominators to check if sanitization always occurs
- Detect missing sanitization on some paths

**Dead Code Detection:**
- Find unreachable blocks
- Identify code that never executes

**Inter-Procedural Analysis:**
- Combine CFG with call graph
- Track data flow across function boundaries

## Example CFG

```python
def process_user(user_id):
    user = get_user(user_id)        # Block 1 (entry)
    if user.is_admin():              # Block 2 (conditional)
        grant_access()               # Block 3 (true branch)
    else:
        deny_access()                # Block 4 (false branch)
    log_action(user)                 # Block 5 (merge point)
    return                           # Block 6 (exit)
```

CFG Structure:
```
Entry → Block1 → Block2 → Block3 → Block5 → Exit
                       ↘ Block4 ↗
```

Dominators:
- Block1 dominates all blocks (always executes)
- Block2 dominates Block3, Block4, Block5
- Block3 does NOT dominate Block5 (false branch skips it)
- Block4 does NOT dominate Block5 (true branch skips it)

## Next Steps

Future PRs will:
- PR #8: Implement pattern registry for security rules
- Use CFG to detect missing sanitization patterns
- Implement taint tracking across CFG paths
- Combine CFG with call graph for full analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
…xample (#329)

* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call site extraction from AST - Pass 2 Part C

This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call graph builder - Pass 3

This PR completes the 3-pass algorithm for building Python call graphs
by implementing the final pass that resolves call targets and constructs
the complete graph structure with edges linking callers to callees.

## Changes

### Core Implementation (builder.go)

1. **BuildCallGraph()**: Main entry point for Pass 3
   - Indexes all function definitions from code graph
   - Iterates through all Python files in the registry
   - Extracts imports and call sites for each file
   - Resolves each call site to its target function
   - Builds edges and stores call site details
   - Returns complete CallGraph with all relationships

2. **indexFunctions()**: Function indexing
   - Scans code graph for all function/method definitions
   - Maps each function to its FQN using module registry
   - Populates CallGraph.Functions map for quick lookup

3. **getFunctionsInFile()**: File-scoped function retrieval
   - Filters code graph nodes by file path
   - Returns only function/method definitions in that file
   - Used for finding containing functions of call sites

4. **findContainingFunction()**: Call site parent resolution
   - Determines which function contains a given call site
   - Uses line number comparison with nearest-match algorithm
   - Finds function with highest line number ≤ call line
   - Returns empty string for module-level calls

5. **resolveCallTarget()**: Core resolution logic
   - Handles simple names: sanitize() → myapp.utils.sanitize
   - Handles qualified names: utils.sanitize() → myapp.utils.sanitize
   - Resolves through import maps first
   - Falls back to same-module resolution
   - Validates FQNs against module registry
   - Returns (FQN, resolved bool) tuple

6. **validateFQN()**: FQN validation
   - Checks if a fully qualified name exists in registry
   - Handles both modules and functions within modules
   - Validates parent module for function FQNs

7. **readFileBytes()**: File reading helper
   - Reads source files for parsing
   - Handles absolute path conversion

### Comprehensive Tests (builder_test.go)

Created 15 test functions covering:

**Resolution Tests:**
- Simple imported function resolution
- Qualified import resolution (module.function)
- Same-module function resolution
- Unresolved method calls (obj.method)
- Non-existent function handling

**Validation Tests:**
- Module existence validation
- Function-in-module validation
- Non-existent module handling

**Helper Function Tests:**
- Function indexing from code graph
- Functions-in-file filtering
- Containing function detection with edge cases

**Integration Tests:**
- Simple single-file call graph
- Multi-file call graph with imports
- Real test fixture integration

## Test Coverage

- Overall: 91.8%
- BuildCallGraph: 80.8%
- indexFunctions: 87.5%
- getFunctionsInFile: 100.0%
- findContainingFunction: 100.0%
- resolveCallTarget: 85.0%
- validateFQN: 100.0%
- readFileBytes: 75.0%

## Algorithm Overview

Pass 3 ties together all previous work:

### Pass 1 (PR #2): BuildModuleRegistry
- Maps file paths to module paths
- Enables FQN generation

### Pass 2 (PRs #3-5): Import & Call Site Extraction
- ExtractImports: Maps local names to FQNs
- ExtractCallSites: Finds all function calls in AST

### Pass 3 (This PR): Call Graph Construction
- Resolves call targets using import maps
- Links callers to callees with edges
- Validates resolutions against registry
- Stores detailed call site information

## Resolution Strategy

The resolver uses a multi-step approach:

1. **Simple names** (no dots):
   - Check import map first
   - Fall back to same-module lookup
   - Return unresolved if neither works

2. **Qualified names** (with dots):
   - Split into base + rest
   - Resolve base through imports
   - Append rest to get full FQN
   - Try current module if not imported

3. **Validation**:
   - Check if target exists in registry
   - For functions, validate parent module exists
   - Mark resolution success/failure

## Design Decisions

1. **Containing function detection**:
   - Uses nearest-match algorithm based on line numbers
   - Finds function with highest line number ≤ call line
   - Handles module-level calls by returning empty FQN

2. **Resolution priority**:
   - Import map takes precedence over same-module
   - Explicit imports always respected even if unresolved
   - Same-module only tried when not in imports

3. **Validation vs Resolution**:
   - Resolution finds FQN from imports/context
   - Validation checks if FQN exists in registry
   - Both pieces of information stored in CallSite

4. **Error handling**:
   - Continues processing even if some files fail
   - Marks individual call sites as unresolved
   - Returns partial graph instead of failing completely

## Next Steps

The call graph infrastructure is now complete. Future PRs will:

- PR #7: Add CFG data structures for control flow analysis
- PR #8: Implement pattern matching for security rules
- PR #9: Integrate into main initialization pipeline
- PR #10: Add comprehensive documentation and examples
- PR #11: Performance optimizations (caching, pooling)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Create CFG data structures for control flow analysis

This PR implements Control Flow Graph (CFG) data structures to enable
intra-procedural analysis of execution paths through functions. CFGs are
essential for security analysis patterns like taint tracking and detecting
missing sanitization on all paths.

## Changes

### Core Implementation (cfg.go)

1. **BlockType**: Enumeration of basic block types
   - Entry: Function entry point
   - Exit: Function exit point
   - Normal: Sequential execution block
   - Conditional: Branch blocks (if/else)
   - Loop: Loop header blocks (while/for)
   - Switch: Switch/match statement blocks
   - Try/Catch/Finally: Exception handling blocks

2. **BasicBlock**: Represents a single basic block
   - ID: Unique identifier within CFG
   - Type: Block category for analysis
   - StartLine/EndLine: Source code location
   - Instructions: CallSites occurring in this block
   - Successors: Blocks that can execute next
   - Predecessors: Blocks that can execute before
   - Condition: Condition expression (for conditional blocks)
   - Dominators: Blocks that always execute before this one

3. **ControlFlowGraph**: Complete CFG for a function
   - FunctionFQN: Fully qualified function name
   - Blocks: Map of block ID to BasicBlock
   - EntryBlockID/ExitBlockID: Special block identifiers
   - CallGraph: Reference for inter-procedural analysis

4. **CFG Operations**:
   - NewControlFlowGraph(): Creates CFG with entry/exit blocks
   - AddBlock(): Adds basic block to CFG
   - AddEdge(): Connects blocks with control flow edges
   - GetBlock(): Retrieves block by ID
   - GetSuccessors(): Returns successor blocks
   - GetPredecessors(): Returns predecessor blocks

5. **Dominator Analysis**:
   - ComputeDominators(): Calculates dominator sets using iterative data flow
   - IsDominator(): Checks if one block dominates another
   - Used to verify sanitization always occurs before usage

6. **Path Analysis**:
   - GetAllPaths(): Enumerates all execution paths from entry to exit
   - dfsAllPaths(): DFS-based path enumeration
   - Used for exhaustive security analysis

7. **Helper Functions**:
   - intersect(): Set intersection for dominator computation
   - slicesEqual(): Compare string slices for fixed-point detection

### Comprehensive Tests (cfg_test.go)

Created 23 test functions covering:

**Construction Tests:**
- CFG creation with entry/exit blocks
- Basic block creation with all fields
- Block addition to CFG

**Edge Management Tests:**
- Adding edges between blocks
- Duplicate edge handling
- Non-existent block edge handling

**Graph Navigation Tests:**
- Block retrieval by ID
- Successor block retrieval
- Predecessor block retrieval

**Dominator Analysis Tests:**
- Linear CFG dominators (A→B→C)
- Branching CFG dominators (if/else merge)
- Dominator checking

**Path Analysis Tests:**
- All paths in linear CFG
- All paths in branching CFG

**Helper Function Tests:**
- Set intersection operations
- Slice equality checking

**Complex Integration Test:**
- Realistic function CFG with branches
- Multiple blocks and paths
- Dominator relationships verification

## Test Coverage

- Overall: 92.7%
- NewControlFlowGraph: 100.0%
- AddBlock: 100.0%
- AddEdge: 100.0%
- GetBlock: 100.0%
- GetSuccessors: 87.5%
- GetPredecessors: 87.5%
- ComputeDominators: 100.0%
- IsDominator: 75.0%
- GetAllPaths: 100.0%
- dfsAllPaths: 91.7%
- intersect: 100.0%
- slicesEqual: 100.0%

## Design Decisions

1. **Entry/Exit blocks always created**:
   - Simplifies analysis by providing single entry/exit points
   - Standard CFG construction practice

2. **Dominator computation uses iterative algorithm**:
   - Simple fixed-point iteration
   - Converges quickly for most real-world CFGs
   - More efficient than other dominator algorithms for small graphs

3. **Path enumeration with cycle detection**:
   - Avoids infinite loops in cyclic CFGs
   - Uses visited tracking during DFS
   - WARNING: Can be exponential for complex CFGs

4. **Blocks store CallSites as instructions**:
   - Links CFG to call graph for inter-procedural analysis
   - Enables tracking tainted data through function calls

5. **Condition stored as string**:
   - Simple representation for conditional blocks
   - Could be enhanced with AST expression nodes later

## Use Cases

CFGs enable several security analysis patterns:

**Taint Analysis:**
- Track data flow through execution paths
- Detect if tainted data reaches sensitive sinks

**Sanitization Verification:**
- Use dominators to check if sanitization always occurs
- Detect missing sanitization on some paths

**Dead Code Detection:**
- Find unreachable blocks
- Identify code that never executes

**Inter-Procedural Analysis:**
- Combine CFG with call graph
- Track data flow across function boundaries

## Example CFG

```python
def process_user(user_id):
    user = get_user(user_id)        # Block 1 (entry)
    if user.is_admin():              # Block 2 (conditional)
        grant_access()               # Block 3 (true branch)
    else:
        deny_access()                # Block 4 (false branch)
    log_action(user)                 # Block 5 (merge point)
    return                           # Block 6 (exit)
```

CFG Structure:
```
Entry → Block1 → Block2 → Block3 → Block5 → Exit
                       ↘ Block4 ↗
```

Dominators:
- Block1 dominates all blocks (always executes)
- Block2 dominates Block3, Block4, Block5
- Block3 does NOT dominate Block5 (false branch skips it)
- Block4 does NOT dominate Block5 (true branch skips it)

## Next Steps

Future PRs will:
- PR #8: Implement pattern registry for security rules
- Use CFG to detect missing sanitization patterns
- Implement taint tracking across CFG paths
- Combine CFG with call graph for full analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Add pattern registry with hardcoded code injection example

Implements pattern matching infrastructure for security analysis with one example pattern (code injection via eval). Additional patterns will be loaded from queries in future PRs. Includes pattern types (source-sink, missing-sanitizer, dangerous-function) and matching algorithms with 92.4% test coverage.

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call site extraction from AST - Pass 2 Part C

This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call graph builder - Pass 3

This PR completes the 3-pass algorithm for building Python call graphs
by implementing the final pass that resolves call targets and constructs
the complete graph structure with edges linking callers to callees.

## Changes

### Core Implementation (builder.go)

1. **BuildCallGraph()**: Main entry point for Pass 3
   - Indexes all function definitions from code graph
   - Iterates through all Python files in the registry
   - Extracts imports and call sites for each file
   - Resolves each call site to its target function
   - Builds edges and stores call site details
   - Returns complete CallGraph with all relationships

2. **indexFunctions()**: Function indexing
   - Scans code graph for all function/method definitions
   - Maps each function to its FQN using module registry
   - Populates CallGraph.Functions map for quick lookup

3. **getFunctionsInFile()**: File-scoped function retrieval
   - Filters code graph nodes by file path
   - Returns only function/method definitions in that file
   - Used for finding containing functions of call sites

4. **findContainingFunction()**: Call site parent resolution
   - Determines which function contains a given call site
   - Uses line number comparison with nearest-match algorithm
   - Finds function with highest line number ≤ call line
   - Returns empty string for module-level calls

5. **resolveCallTarget()**: Core resolution logic
   - Handles simple names: sanitize() → myapp.utils.sanitize
   - Handles qualified names: utils.sanitize() → myapp.utils.sanitize
   - Resolves through import maps first
   - Falls back to same-module resolution
   - Validates FQNs against module registry
   - Returns (FQN, resolved bool) tuple

6. **validateFQN()**: FQN validation
   - Checks if a fully qualified name exists in registry
   - Handles both modules and functions within modules
   - Validates parent module for function FQNs

7. **readFileBytes()**: File reading helper
   - Reads source files for parsing
   - Handles absolute path conversion

### Comprehensive Tests (builder_test.go)

Created 15 test functions covering:

**Resolution Tests:**
- Simple imported function resolution
- Qualified import resolution (module.function)
- Same-module function resolution
- Unresolved method calls (obj.method)
- Non-existent function handling

**Validation Tests:**
- Module existence validation
- Function-in-module validation
- Non-existent module handling

**Helper Function Tests:**
- Function indexing from code graph
- Functions-in-file filtering
- Containing function detection with edge cases

**Integration Tests:**
- Simple single-file call graph
- Multi-file call graph with imports
- Real test fixture integration

## Test Coverage

- Overall: 91.8%
- BuildCallGraph: 80.8%
- indexFunctions: 87.5%
- getFunctionsInFile: 100.0%
- findContainingFunction: 100.0%
- resolveCallTarget: 85.0%
- validateFQN: 100.0%
- readFileBytes: 75.0%

## Algorithm Overview

Pass 3 ties together all previous work:

### Pass 1 (PR #2): BuildModuleRegistry
- Maps file paths to module paths
- Enables FQN generation

### Pass 2 (PRs #3-5): Import & Call Site Extraction
- ExtractImports: Maps local names to FQNs
- ExtractCallSites: Finds all function calls in AST

### Pass 3 (This PR): Call Graph Construction
- Resolves call targets using import maps
- Links callers to callees with edges
- Validates resolutions against registry
- Stores detailed call site information

## Resolution Strategy

The resolver uses a multi-step approach:

1. **Simple names** (no dots):
   - Check import map first
   - Fall back to same-module lookup
   - Return unresolved if neither works

2. **Qualified names** (with dots):
   - Split into base + rest
   - Resolve base through imports
   - Append rest to get full FQN
   - Try current module if not imported

3. **Validation**:
   - Check if target exists in registry
   - For functions, validate parent module exists
   - Mark resolution success/failure

## Design Decisions

1. **Containing function detection**:
   - Uses nearest-match algorithm based on line numbers
   - Finds function with highest line number ≤ call line
   - Handles module-level calls by returning empty FQN

2. **Resolution priority**:
   - Import map takes precedence over same-module
   - Explicit imports always respected even if unresolved
   - Same-module only tried when not in imports

3. **Validation vs Resolution**:
   - Resolution finds FQN from imports/context
   - Validation checks if FQN exists in registry
   - Both pieces of information stored in CallSite

4. **Error handling**:
   - Continues processing even if some files fail
   - Marks individual call sites as unresolved
   - Returns partial graph instead of failing completely

## Next Steps

The call graph infrastructure is now complete. Future PRs will:

- PR #7: Add CFG data structures for control flow analysis
- PR #8: Implement pattern matching for security rules
- PR #9: Integrate into main initialization pipeline
- PR #10: Add comprehensive documentation and examples
- PR #11: Performance optimizations (caching, pooling)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Create CFG data structures for control flow analysis

This PR implements Control Flow Graph (CFG) data structures to enable
intra-procedural analysis of execution paths through functions. CFGs are
essential for security analysis patterns like taint tracking and detecting
missing sanitization on all paths.

## Changes

### Core Implementation (cfg.go)

1. **BlockType**: Enumeration of basic block types
   - Entry: Function entry point
   - Exit: Function exit point
   - Normal: Sequential execution block
   - Conditional: Branch blocks (if/else)
   - Loop: Loop header blocks (while/for)
   - Switch: Switch/match statement blocks
   - Try/Catch/Finally: Exception handling blocks

2. **BasicBlock**: Represents a single basic block
   - ID: Unique identifier within CFG
   - Type: Block category for analysis
   - StartLine/EndLine: Source code location
   - Instructions: CallSites occurring in this block
   - Successors: Blocks that can execute next
   - Predecessors: Blocks that can execute before
   - Condition: Condition expression (for conditional blocks)
   - Dominators: Blocks that always execute before this one

3. **ControlFlowGraph**: Complete CFG for a function
   - FunctionFQN: Fully qualified function name
   - Blocks: Map of block ID to BasicBlock
   - EntryBlockID/ExitBlockID: Special block identifiers
   - CallGraph: Reference for inter-procedural analysis

4. **CFG Operations**:
   - NewControlFlowGraph(): Creates CFG with entry/exit blocks
   - AddBlock(): Adds basic block to CFG
   - AddEdge(): Connects blocks with control flow edges
   - GetBlock(): Retrieves block by ID
   - GetSuccessors(): Returns successor blocks
   - GetPredecessors(): Returns predecessor blocks

5. **Dominator Analysis**:
   - ComputeDominators(): Calculates dominator sets using iterative data flow
   - IsDominator(): Checks if one block dominates another
   - Used to verify sanitization always occurs before usage

6. **Path Analysis**:
   - GetAllPaths(): Enumerates all execution paths from entry to exit
   - dfsAllPaths(): DFS-based path enumeration
   - Used for exhaustive security analysis

7. **Helper Functions**:
   - intersect(): Set intersection for dominator computation
   - slicesEqual(): Compare string slices for fixed-point detection

### Comprehensive Tests (cfg_test.go)

Created 23 test functions covering:

**Construction Tests:**
- CFG creation with entry/exit blocks
- Basic block creation with all fields
- Block addition to CFG

**Edge Management Tests:**
- Adding edges between blocks
- Duplicate edge handling
- Non-existent block edge handling

**Graph Navigation Tests:**
- Block retrieval by ID
- Successor block retrieval
- Predecessor block retrieval

**Dominator Analysis Tests:**
- Linear CFG dominators (A→B→C)
- Branching CFG dominators (if/else merge)
- Dominator checking

**Path Analysis Tests:**
- All paths in linear CFG
- All paths in branching CFG

**Helper Function Tests:**
- Set intersection operations
- Slice equality checking

**Complex Integration Test:**
- Realistic function CFG with branches
- Multiple blocks and paths
- Dominator relationships verification

## Test Coverage

- Overall: 92.7%
- NewControlFlowGraph: 100.0%
- AddBlock: 100.0%
- AddEdge: 100.0%
- GetBlock: 100.0%
- GetSuccessors: 87.5%
- GetPredecessors: 87.5%
- ComputeDominators: 100.0%
- IsDominator: 75.0%
- GetAllPaths: 100.0%
- dfsAllPaths: 91.7%
- intersect: 100.0%
- slicesEqual: 100.0%

## Design Decisions

1. **Entry/Exit blocks always created**:
   - Simplifies analysis by providing single entry/exit points
   - Standard CFG construction practice

2. **Dominator computation uses iterative algorithm**:
   - Simple fixed-point iteration
   - Converges quickly for most real-world CFGs
   - More efficient than other dominator algorithms for small graphs

3. **Path enumeration with cycle detection**:
   - Avoids infinite loops in cyclic CFGs
   - Uses visited tracking during DFS
   - WARNING: Can be exponential for complex CFGs

4. **Blocks store CallSites as instructions**:
   - Links CFG to call graph for inter-procedural analysis
   - Enables tracking tainted data through function calls

5. **Condition stored as string**:
   - Simple representation for conditional blocks
   - Could be enhanced with AST expression nodes later

## Use Cases

CFGs enable several security analysis patterns:

**Taint Analysis:**
- Track data flow through execution paths
- Detect if tainted data reaches sensitive sinks

**Sanitization Verification:**
- Use dominators to check if sanitization always occurs
- Detect missing sanitization on some paths

**Dead Code Detection:**
- Find unreachable blocks
- Identify code that never executes

**Inter-Procedural Analysis:**
- Combine CFG with call graph
- Track data flow across function boundaries

## Example CFG

```python
def process_user(user_id):
    user = get_user(user_id)        # Block 1 (entry)
    if user.is_admin():              # Block 2 (conditional)
        grant_access()               # Block 3 (true branch)
    else:
        deny_access()                # Block 4 (false branch)
    log_action(user)                 # Block 5 (merge point)
    return                           # Block 6 (exit)
```

CFG Structure:
```
Entry → Block1 → Block2 → Block3 → Block5 → Exit
                       ↘ Block4 ↗
```

Dominators:
- Block1 dominates all blocks (always executes)
- Block2 dominates Block3, Block4, Block5
- Block3 does NOT dominate Block5 (false branch skips it)
- Block4 does NOT dominate Block5 (true branch skips it)

## Next Steps

Future PRs will:
- PR #8: Implement pattern registry for security rules
- Use CFG to detect missing sanitization patterns
- Implement taint tracking across CFG paths
- Combine CFG with call graph for full analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Add pattern registry with hardcoded code injection example

Implements pattern matching infrastructure for security analysis with one example pattern (code injection via eval). Additional patterns will be loaded from queries in future PRs. Includes pattern types (source-sink, missing-sanitizer, dangerous-function) and matching algorithms with 92.4% test coverage.

* feat: Integrate call graph into initialization pipeline

Adds InitializeCallGraph() to wire together the 3-pass algorithm (module registry, call graph building, pattern loading) and AnalyzePatterns() for security pattern detection. Includes end-to-end integration tests with 92.6% coverage.

* add callgraph integration

* chore: comment the debugging code

* cpf/enhancement: Benchmark suite test for callgraph (#331)

* feat: Add comprehensive benchmark suite for performance testing

This commit adds a complete benchmark suite to measure performance across
small, medium, and large Python projects. The benchmarks establish baseline
metrics for future optimization work.

Changes:
- Add benchmark_test.go with benchmarks for:
  * Module registry building (Pass 1)
  * Import extraction (Pass 2A)
  * Call site extraction (Pass 2B)
  * Call target resolution
  * Pattern matching
- Test against 3 real-world codebases:
  * Small: simple_project (~5 files)
  * Medium: label-studio (~1000 files)
  * Large: salt (~10,000 files)
- Fix patterns_test.go assertions for PatternMatchDetails return type
- Fix godot lint errors in builder.go

Baseline Performance Results (Apple M2 Max, 5 iterations):
- BuildModuleRegistry_Small: 80µs (target: <10ms) ✓
- BuildModuleRegistry_Medium: 6.5ms (target: <500ms) ✓
- BuildModuleRegistry_Large: 3.3ms (target: <2s) ✓
- ExtractImports_Small: 101µs (target: <20ms) ✓
- ExtractImports_Medium: 433ms (target: <2s) ✓
- ExtractCallSites_Small: 91µs (target: <30ms) ✓
- ResolveCallTarget: 533ns (target: <1µs) ✓

All benchmarks meet performance targets. Medium/Large project benchmarks
are skipped by default to keep CI fast. Enable manually with:
  go test -bench=Medium -run=^$
  go test -bench=Large -run=^$

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Add ImportMap caching with sync.RWMutex for performance

This commit implements thread-safe caching of ImportMap instances to avoid
re-parsing imports from the same file multiple times. This provides significant
performance improvements when the same imports are needed repeatedly.

Changes:
- Add ImportMapCache struct with RWMutex-protected cache map
- Implement Get(), Put(), and GetOrExtract() cache methods
- Update BuildCallGraph to use import caching
- Add comprehensive cache_test.go with:
  * Basic CRUD operations tests
  * Cache hit/miss scenarios
  * Concurrent access safety tests
  * Performance benchmarks

Performance characteristics:
- Get operation: O(1) with read lock (allows concurrent reads)
- Put operation: O(1) with write lock (exclusive access)
- Thread-safe for concurrent access from multiple goroutines
- Cache hit avoids expensive tree-sitter parsing

Test coverage:
- NewImportMapCache: 100%
- Get: 100%
- Put: 100%
- GetOrExtract: 85.7%
- All tests pass including concurrent access tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Correct matchesFunctionName test expectations

The test was incorrectly expecting 'evaluation' to match 'eval' via
substring matching, but the implementation correctly only supports:
- Exact matches: 'eval' == 'eval'
- Suffix matches: 'myapp.utils.eval' ends with '.eval'
- Prefix matches: 'request.GET.get' starts with 'request.GET.'

This prevents false positives like matching 'evaluation' to 'eval'.

Updated test case to expect false for 'evaluation' vs 'eval' match.
All tests now pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Update main_test.go to include analyze command in expected output

The analyze command was added in a previous commit (cmd/analyze.go) but the
main_test.go wasn't updated to reflect this new command in the help output.

This caused TestExecute/Successful_execution to fail because it expected
the old command list without 'analyze'.

Updated expected output to include:
  analyze     Analyze source code for security vulnerabilities using call graph

All tests now pass with gradle testGo.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feature: add diagnostic report command for callgraph resolution

* cpf/enhancement: added resolution for framework and its corresponding support (#332)

* feature: added resolution for framework and its corresponding support

* chore: fixed lint issues

---------

Co-authored-by: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Oct 29, 2025
* feat: Add core data structures for call graph (PR #1)

Add foundational data structures for Python call graph construction:

New Types:
- CallSite: Represents function call locations with arguments and resolution status
- CallGraph: Maps functions to callees with forward/reverse edges
- ModuleRegistry: Maps Python file paths to module paths
- ImportMap: Tracks imports per file for name resolution
- Location: Source code position tracking
- Argument: Function call argument metadata

Features:
- 100% test coverage with comprehensive unit tests
- Bidirectional call graph edges (forward and reverse)
- Support for ambiguous short names in module registry
- Helper functions for module path manipulation

This establishes the foundation for 3-pass call graph algorithm:
- Pass 1 (next PR): Module registry builder
- Pass 2 (next PR): Import extraction and resolution
- Pass 3 (next PR): Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement module registry - Pass 1 of 3-pass algorithm (PR #2)

Implement the first pass of the call graph construction algorithm: building
a complete registry of Python modules by walking the directory tree.

New Features:
- BuildModuleRegistry: Walks directory tree and maps file paths to module paths
- convertToModulePath: Converts file system paths to Python import paths
- shouldSkipDirectory: Filters out venv, __pycache__, build dirs, etc.

Module Path Conversion:
- Handles regular files: myapp/views.py → myapp.views
- Handles packages: myapp/utils/__init__.py → myapp.utils
- Supports deep nesting: myapp/api/v1/endpoints/users.py → myapp.api.v1.endpoints.users
- Cross-platform: Normalizes Windows/Unix path separators

Performance Optimizations:
- Skips 15+ common non-source directories (venv, __pycache__, .git, dist, build, etc.)
- Avoids scanning thousands of dependency files
- Indexes both full module paths and short names for ambiguity detection

Test Coverage: 93%
- Comprehensive unit tests for all conversion scenarios
- Integration tests with real Python project structure
- Edge case handling: empty dirs, non-Python files, deep nesting, permissions
- Error path testing: walk errors, invalid paths, system errors
- Test fixtures: test-src/python/simple_project/ with realistic structure
- Documented: Remaining 7% are untestable OS-level errors (filepath.Abs failures)

This establishes Pass 1 of 3:
- ✅ Pass 1: Module registry (this PR)
- Next: Pass 2 - Import extraction and resolution
- Next: Pass 3 - Call graph construction

Related: Phase 1 - Call Graph Construction & 3-Pass Algorithm
Base Branch: shiva/callgraph-infra-1 (PR #1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement import extraction with tree-sitter - Pass 2 Part A

This PR implements comprehensive import extraction for Python code using
tree-sitter AST parsing. It handles all three main import styles:

1. Simple imports: `import module`
2. From imports: `from module import name`
3. Aliased imports: `import module as alias` and `from module import name as alias`

The implementation uses direct AST traversal instead of tree-sitter queries
for better compatibility and control. It properly handles:
- Multiple imports per line (`from json import dumps, loads`)
- Nested module paths (`import xml.etree.ElementTree`)
- Whitespace variations
- Invalid/malformed syntax (fault-tolerant parsing)

Key functions:
- ExtractImports(): Main entry point that parses code and builds ImportMap
- traverseForImports(): Recursively traverses AST to find import statements
- processImportStatement(): Handles simple and aliased imports
- processImportFromStatement(): Handles from-import statements with proper
  module name skipping to avoid duplicate entries

Test coverage: 92.8% overall, 90-95% for import extraction functions

Test fixtures include:
- simple_imports.py: Basic import statements
- from_imports.py: From import statements with multiple names
- aliased_imports.py: Aliased imports (both simple and from)
- mixed_imports.py: Mixed import styles

All tests passing, linting clean, builds successfully.

This is Pass 2 Part A of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement relative import resolution - Pass 2 Part B

This PR implements comprehensive relative import resolution for Python using
a 3-pass algorithm. It extends the import extraction system from PR #3 to handle
Python's relative import syntax with dot notation.

Key Changes:

1. **Added FileToModule reverse mapping to ModuleRegistry**
   - Enables O(1) lookup from file path to module path
   - Required for resolving relative imports
   - Updated AddModule() to maintain bidirectional mapping

2. **Implemented resolveRelativeImport() function**
   - Handles single dot (.) for current package
   - Handles multiple dots (.., ...) for parent/grandparent packages
   - Navigates package hierarchy using module path components
   - Clamps excessive dots to root package level
   - Falls back gracefully when file not in registry

3. **Enhanced processImportFromStatement() for relative imports**
   - Detects relative_import nodes in tree-sitter AST
   - Extracts import_prefix (dots) and optional module suffix
   - Resolves relative paths to absolute module paths before adding to ImportMap

4. **Comprehensive test coverage (94.5% overall)**
   - Unit tests for resolveRelativeImport with various dot counts
   - Integration tests with ExtractImports
   - Tests for deeply nested packages
   - Tests for mixed absolute and relative imports
   - Real fixture files with project structure

Relative Import Examples:
- `from . import utils` → "currentpackage.utils"
- `from .. import config` → "parentpackage.config"
- `from ..utils import helper` → "parentpackage.utils.helper"
- `from ...db import query` → "grandparent.db.query"

Test Fixtures:
- Created myapp/submodule/handler.py with all relative import styles
- Created supporting package structure with __init__.py files
- Tests verify correct resolution across package hierarchy

All tests passing, linting clean, builds successfully.

This is Pass 2 Part B of the 3-pass call graph algorithm.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call site extraction from AST - Pass 2 Part C

This PR implements call site extraction from Python source code using
tree-sitter AST parsing. It builds on the import resolution work from
PRs #3 and #4 to prepare for call graph construction in Pass 3.

## Changes

### Core Implementation (callsites.go)

1. **ExtractCallSites()**: Main entry point for extracting call sites
   - Parses Python source with tree-sitter
   - Traverses AST to find all call expressions
   - Returns slice of CallSite objects with location information

2. **traverseForCalls()**: Recursive AST traversal
   - Tracks function context while traversing
   - Updates context when entering function definitions
   - Finds and processes call expressions

3. **processCallExpression()**: Call site processing
   - Extracts callee name (function/method being called)
   - Parses arguments (positional and keyword)
   - Creates CallSite with source location
   - Parameters for importMap and caller reserved for Pass 3

4. **extractCalleeName()**: Callee name extraction
   - Handles simple identifiers: foo()
   - Handles attributes: obj.method(), obj.attr.method()
   - Recursively builds dotted names

5. **extractArguments()**: Argument parsing
   - Extracts all positional arguments
   - Preserves keyword arguments as "name=value" in Value field
   - Tracks argument position and variable status

6. **convertArgumentsToSlice()**: Helper for struct conversion
   - Converts []*Argument to []Argument for CallSite struct

### Comprehensive Tests (callsites_test.go)

Created 17 test functions covering:
- Simple function calls: foo(), bar()
- Method calls: obj.method(), self.helper()
- Arguments: positional, keyword, mixed
- Nested calls: foo(bar(x))
- Multiple functions in one file
- Class methods
- Chained calls: obj.method1().method2()
- Module-level calls (no function context)
- Source location tracking
- Empty files
- Complex arguments: expressions, lists, dicts, lambdas
- Nested method calls: obj.attr.method()
- Real file fixture integration

### Test Fixture (simple_calls.py)

Created realistic test file with:
- Function definitions with various call patterns
- Method calls on objects
- Calls with arguments (positional and keyword)
- Nested calls
- Class methods with self references

## Test Coverage

- Overall: 93.3%
- ExtractCallSites: 90.0%
- traverseForCalls: 93.3%
- processCallExpression: 83.3%
- extractCalleeName: 91.7%
- extractArguments: 87.5%
- convertArgumentsToSlice: 100.0%

## Design Decisions

1. **Keyword argument handling**: Store as "name=value" in Value field
   - Tree-sitter provides full keyword_argument node content
   - Preserves complete argument information for later analysis
   - Separating name/value would require additional parsing

2. **Caller context tracking**: Parameter reserved but not used yet
   - Will be populated in Pass 3 during call graph construction
   - Enables linking call sites to their containing functions

3. **Import map parameter**: Reserved for Pass 3 resolution
   - Will be used to resolve qualified names to FQNs
   - Enables cross-file call graph construction

4. **Location tracking**: Store exact position for each call site
   - File, line, column information
   - Enables precise error reporting and code navigation

## Testing Strategy

- Unit tests for each extraction function
- Integration tests with tree-sitter AST
- Real file fixture for end-to-end validation
- Edge cases: empty files, no context, nested structures

## Next Steps (PR #6)

Pass 3 will use this call site data to:
1. Build the complete call graph structure
2. Resolve call targets to function definitions
3. Link caller and callee through edges
4. Handle disambiguation for overloaded names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Implement call graph builder - Pass 3

This PR completes the 3-pass algorithm for building Python call graphs
by implementing the final pass that resolves call targets and constructs
the complete graph structure with edges linking callers to callees.

## Changes

### Core Implementation (builder.go)

1. **BuildCallGraph()**: Main entry point for Pass 3
   - Indexes all function definitions from code graph
   - Iterates through all Python files in the registry
   - Extracts imports and call sites for each file
   - Resolves each call site to its target function
   - Builds edges and stores call site details
   - Returns complete CallGraph with all relationships

2. **indexFunctions()**: Function indexing
   - Scans code graph for all function/method definitions
   - Maps each function to its FQN using module registry
   - Populates CallGraph.Functions map for quick lookup

3. **getFunctionsInFile()**: File-scoped function retrieval
   - Filters code graph nodes by file path
   - Returns only function/method definitions in that file
   - Used for finding containing functions of call sites

4. **findContainingFunction()**: Call site parent resolution
   - Determines which function contains a given call site
   - Uses line number comparison with nearest-match algorithm
   - Finds function with highest line number ≤ call line
   - Returns empty string for module-level calls

5. **resolveCallTarget()**: Core resolution logic
   - Handles simple names: sanitize() → myapp.utils.sanitize
   - Handles qualified names: utils.sanitize() → myapp.utils.sanitize
   - Resolves through import maps first
   - Falls back to same-module resolution
   - Validates FQNs against module registry
   - Returns (FQN, resolved bool) tuple

6. **validateFQN()**: FQN validation
   - Checks if a fully qualified name exists in registry
   - Handles both modules and functions within modules
   - Validates parent module for function FQNs

7. **readFileBytes()**: File reading helper
   - Reads source files for parsing
   - Handles absolute path conversion

### Comprehensive Tests (builder_test.go)

Created 15 test functions covering:

**Resolution Tests:**
- Simple imported function resolution
- Qualified import resolution (module.function)
- Same-module function resolution
- Unresolved method calls (obj.method)
- Non-existent function handling

**Validation Tests:**
- Module existence validation
- Function-in-module validation
- Non-existent module handling

**Helper Function Tests:**
- Function indexing from code graph
- Functions-in-file filtering
- Containing function detection with edge cases

**Integration Tests:**
- Simple single-file call graph
- Multi-file call graph with imports
- Real test fixture integration

## Test Coverage

- Overall: 91.8%
- BuildCallGraph: 80.8%
- indexFunctions: 87.5%
- getFunctionsInFile: 100.0%
- findContainingFunction: 100.0%
- resolveCallTarget: 85.0%
- validateFQN: 100.0%
- readFileBytes: 75.0%

## Algorithm Overview

Pass 3 ties together all previous work:

### Pass 1 (PR #2): BuildModuleRegistry
- Maps file paths to module paths
- Enables FQN generation

### Pass 2 (PRs #3-5): Import & Call Site Extraction
- ExtractImports: Maps local names to FQNs
- ExtractCallSites: Finds all function calls in AST

### Pass 3 (This PR): Call Graph Construction
- Resolves call targets using import maps
- Links callers to callees with edges
- Validates resolutions against registry
- Stores detailed call site information

## Resolution Strategy

The resolver uses a multi-step approach:

1. **Simple names** (no dots):
   - Check import map first
   - Fall back to same-module lookup
   - Return unresolved if neither works

2. **Qualified names** (with dots):
   - Split into base + rest
   - Resolve base through imports
   - Append rest to get full FQN
   - Try current module if not imported

3. **Validation**:
   - Check if target exists in registry
   - For functions, validate parent module exists
   - Mark resolution success/failure

## Design Decisions

1. **Containing function detection**:
   - Uses nearest-match algorithm based on line numbers
   - Finds function with highest line number ≤ call line
   - Handles module-level calls by returning empty FQN

2. **Resolution priority**:
   - Import map takes precedence over same-module
   - Explicit imports always respected even if unresolved
   - Same-module only tried when not in imports

3. **Validation vs Resolution**:
   - Resolution finds FQN from imports/context
   - Validation checks if FQN exists in registry
   - Both pieces of information stored in CallSite

4. **Error handling**:
   - Continues processing even if some files fail
   - Marks individual call sites as unresolved
   - Returns partial graph instead of failing completely

## Next Steps

The call graph infrastructure is now complete. Future PRs will:

- PR #7: Add CFG data structures for control flow analysis
- PR #8: Implement pattern matching for security rules
- PR #9: Integrate into main initialization pipeline
- PR #10: Add comprehensive documentation and examples
- PR #11: Performance optimizations (caching, pooling)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Create CFG data structures for control flow analysis

This PR implements Control Flow Graph (CFG) data structures to enable
intra-procedural analysis of execution paths through functions. CFGs are
essential for security analysis patterns like taint tracking and detecting
missing sanitization on all paths.

## Changes

### Core Implementation (cfg.go)

1. **BlockType**: Enumeration of basic block types
   - Entry: Function entry point
   - Exit: Function exit point
   - Normal: Sequential execution block
   - Conditional: Branch blocks (if/else)
   - Loop: Loop header blocks (while/for)
   - Switch: Switch/match statement blocks
   - Try/Catch/Finally: Exception handling blocks

2. **BasicBlock**: Represents a single basic block
   - ID: Unique identifier within CFG
   - Type: Block category for analysis
   - StartLine/EndLine: Source code location
   - Instructions: CallSites occurring in this block
   - Successors: Blocks that can execute next
   - Predecessors: Blocks that can execute before
   - Condition: Condition expression (for conditional blocks)
   - Dominators: Blocks that always execute before this one

3. **ControlFlowGraph**: Complete CFG for a function
   - FunctionFQN: Fully qualified function name
   - Blocks: Map of block ID to BasicBlock
   - EntryBlockID/ExitBlockID: Special block identifiers
   - CallGraph: Reference for inter-procedural analysis

4. **CFG Operations**:
   - NewControlFlowGraph(): Creates CFG with entry/exit blocks
   - AddBlock(): Adds basic block to CFG
   - AddEdge(): Connects blocks with control flow edges
   - GetBlock(): Retrieves block by ID
   - GetSuccessors(): Returns successor blocks
   - GetPredecessors(): Returns predecessor blocks

5. **Dominator Analysis**:
   - ComputeDominators(): Calculates dominator sets using iterative data flow
   - IsDominator(): Checks if one block dominates another
   - Used to verify sanitization always occurs before usage

6. **Path Analysis**:
   - GetAllPaths(): Enumerates all execution paths from entry to exit
   - dfsAllPaths(): DFS-based path enumeration
   - Used for exhaustive security analysis

7. **Helper Functions**:
   - intersect(): Set intersection for dominator computation
   - slicesEqual(): Compare string slices for fixed-point detection

### Comprehensive Tests (cfg_test.go)

Created 23 test functions covering:

**Construction Tests:**
- CFG creation with entry/exit blocks
- Basic block creation with all fields
- Block addition to CFG

**Edge Management Tests:**
- Adding edges between blocks
- Duplicate edge handling
- Non-existent block edge handling

**Graph Navigation Tests:**
- Block retrieval by ID
- Successor block retrieval
- Predecessor block retrieval

**Dominator Analysis Tests:**
- Linear CFG dominators (A→B→C)
- Branching CFG dominators (if/else merge)
- Dominator checking

**Path Analysis Tests:**
- All paths in linear CFG
- All paths in branching CFG

**Helper Function Tests:**
- Set intersection operations
- Slice equality checking

**Complex Integration Test:**
- Realistic function CFG with branches
- Multiple blocks and paths
- Dominator relationships verification

## Test Coverage

- Overall: 92.7%
- NewControlFlowGraph: 100.0%
- AddBlock: 100.0%
- AddEdge: 100.0%
- GetBlock: 100.0%
- GetSuccessors: 87.5%
- GetPredecessors: 87.5%
- ComputeDominators: 100.0%
- IsDominator: 75.0%
- GetAllPaths: 100.0%
- dfsAllPaths: 91.7%
- intersect: 100.0%
- slicesEqual: 100.0%

## Design Decisions

1. **Entry/Exit blocks always created**:
   - Simplifies analysis by providing single entry/exit points
   - Standard CFG construction practice

2. **Dominator computation uses iterative algorithm**:
   - Simple fixed-point iteration
   - Converges quickly for most real-world CFGs
   - More efficient than other dominator algorithms for small graphs

3. **Path enumeration with cycle detection**:
   - Avoids infinite loops in cyclic CFGs
   - Uses visited tracking during DFS
   - WARNING: Can be exponential for complex CFGs

4. **Blocks store CallSites as instructions**:
   - Links CFG to call graph for inter-procedural analysis
   - Enables tracking tainted data through function calls

5. **Condition stored as string**:
   - Simple representation for conditional blocks
   - Could be enhanced with AST expression nodes later

## Use Cases

CFGs enable several security analysis patterns:

**Taint Analysis:**
- Track data flow through execution paths
- Detect if tainted data reaches sensitive sinks

**Sanitization Verification:**
- Use dominators to check if sanitization always occurs
- Detect missing sanitization on some paths

**Dead Code Detection:**
- Find unreachable blocks
- Identify code that never executes

**Inter-Procedural Analysis:**
- Combine CFG with call graph
- Track data flow across function boundaries

## Example CFG

```python
def process_user(user_id):
    user = get_user(user_id)        # Block 1 (entry)
    if user.is_admin():              # Block 2 (conditional)
        grant_access()               # Block 3 (true branch)
    else:
        deny_access()                # Block 4 (false branch)
    log_action(user)                 # Block 5 (merge point)
    return                           # Block 6 (exit)
```

CFG Structure:
```
Entry → Block1 → Block2 → Block3 → Block5 → Exit
                       ↘ Block4 ↗
```

Dominators:
- Block1 dominates all blocks (always executes)
- Block2 dominates Block3, Block4, Block5
- Block3 does NOT dominate Block5 (false branch skips it)
- Block4 does NOT dominate Block5 (true branch skips it)

## Next Steps

Future PRs will:
- PR #8: Implement pattern registry for security rules
- Use CFG to detect missing sanitization patterns
- Implement taint tracking across CFG paths
- Combine CFG with call graph for full analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Add pattern registry with hardcoded code injection example

Implements pattern matching infrastructure for security analysis with one example pattern (code injection via eval). Additional patterns will be loaded from queries in future PRs. Includes pattern types (source-sink, missing-sanitizer, dangerous-function) and matching algorithms with 92.4% test coverage.

* feat: Integrate call graph into initialization pipeline

Adds InitializeCallGraph() to wire together the 3-pass algorithm (module registry, call graph building, pattern loading) and AnalyzePatterns() for security pattern detection. Includes end-to-end integration tests with 92.6% coverage.

* add callgraph integration

* chore: comment the debugging code

* feat: Add comprehensive benchmark suite for performance testing

This commit adds a complete benchmark suite to measure performance across
small, medium, and large Python projects. The benchmarks establish baseline
metrics for future optimization work.

Changes:
- Add benchmark_test.go with benchmarks for:
  * Module registry building (Pass 1)
  * Import extraction (Pass 2A)
  * Call site extraction (Pass 2B)
  * Call target resolution
  * Pattern matching
- Test against 3 real-world codebases:
  * Small: simple_project (~5 files)
  * Medium: label-studio (~1000 files)
  * Large: salt (~10,000 files)
- Fix patterns_test.go assertions for PatternMatchDetails return type
- Fix godot lint errors in builder.go

Baseline Performance Results (Apple M2 Max, 5 iterations):
- BuildModuleRegistry_Small: 80µs (target: <10ms) ✓
- BuildModuleRegistry_Medium: 6.5ms (target: <500ms) ✓
- BuildModuleRegistry_Large: 3.3ms (target: <2s) ✓
- ExtractImports_Small: 101µs (target: <20ms) ✓
- ExtractImports_Medium: 433ms (target: <2s) ✓
- ExtractCallSites_Small: 91µs (target: <30ms) ✓
- ResolveCallTarget: 533ns (target: <1µs) ✓

All benchmarks meet performance targets. Medium/Large project benchmarks
are skipped by default to keep CI fast. Enable manually with:
  go test -bench=Medium -run=^$
  go test -bench=Large -run=^$

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feat: Add ImportMap caching with sync.RWMutex for performance

This commit implements thread-safe caching of ImportMap instances to avoid
re-parsing imports from the same file multiple times. This provides significant
performance improvements when the same imports are needed repeatedly.

Changes:
- Add ImportMapCache struct with RWMutex-protected cache map
- Implement Get(), Put(), and GetOrExtract() cache methods
- Update BuildCallGraph to use import caching
- Add comprehensive cache_test.go with:
  * Basic CRUD operations tests
  * Cache hit/miss scenarios
  * Concurrent access safety tests
  * Performance benchmarks

Performance characteristics:
- Get operation: O(1) with read lock (allows concurrent reads)
- Put operation: O(1) with write lock (exclusive access)
- Thread-safe for concurrent access from multiple goroutines
- Cache hit avoids expensive tree-sitter parsing

Test coverage:
- NewImportMapCache: 100%
- Get: 100%
- Put: 100%
- GetOrExtract: 85.7%
- All tests pass including concurrent access tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Correct matchesFunctionName test expectations

The test was incorrectly expecting 'evaluation' to match 'eval' via
substring matching, but the implementation correctly only supports:
- Exact matches: 'eval' == 'eval'
- Suffix matches: 'myapp.utils.eval' ends with '.eval'
- Prefix matches: 'request.GET.get' starts with 'request.GET.'

This prevents false positives like matching 'evaluation' to 'eval'.

Updated test case to expect false for 'evaluation' vs 'eval' match.
All tests now pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Update main_test.go to include analyze command in expected output

The analyze command was added in a previous commit (cmd/analyze.go) but the
main_test.go wasn't updated to reflect this new command in the help output.

This caused TestExecute/Successful_execution to fail because it expected
the old command list without 'analyze'.

Updated expected output to include:
  analyze     Analyze source code for security vulnerabilities using call graph

All tests now pass with gradle testGo.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* feature: add diagnostic report command for callgraph resolution

* feature: added resolution for framework and its corresponding support

* chore: fixed lint issues

* added orm related resolutions with framework support

---------

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 2, 2025
Implements PR #2: Local filesystem loader for Python stdlib registries.

**Core Implementation:**
- Add stdlib_registry.go with data structures for Python 3.14 stdlib
  - StdlibRegistry, StdlibModule, StdlibFunction, StdlibClass
  - StdlibConstant, StdlibAttribute with full JSON mapping
  - Snake_case JSON tags to match Python-generated format
- Add stdlib_registry_loader.go for local file loading
  - Loads manifest.json and all module JSON files
  - SHA256 checksum verification for data integrity
  - Graceful error handling (logs warnings, continues without stdlib)
- Add stdlib_registry_loader_test.go with comprehensive coverage
  - Tests manifest loading, module loading, checksum validation
  - Tests with actual generated registries (188 modules)
  - Edge case handling (missing/corrupted files)

**Resolution Integration:**
- Add validateStdlibFQN() helper with module alias support
  - Handles os.path -> posixpath platform-specific aliasing
  - Checks functions, classes, constants, and attributes
- Integrate stdlib validation into resolveCallTarget()
  - Checks stdlib before user project registry
  - Non-blocking: stdlib load failures don't break analysis

**Test Results:**
- All tests passing (gradle testGo)
- Zero lint issues (gradle lintGo)
- Successfully loads 188 modules from registries/python3.14/stdlib/v1/
- Resolution improvement: 64.7% -> 66.3% (+90 resolutions)

**Integration:**
- TypeInferenceEngine.StdlibRegistry field added
- Loaded in BuildCallGraph() after builtin registry
- Logs success: "Loaded stdlib registry: 188 modules"

Related: PR #1 (Python stdlib registry generator)
Next: PR #3 (remote registry hosting + deployment)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 2, 2025
Implements PR #2: Local filesystem loader for Python stdlib registries.

**Core Implementation:**
- Add stdlib_registry.go with data structures for Python 3.14 stdlib
  - StdlibRegistry, StdlibModule, StdlibFunction, StdlibClass
  - StdlibConstant, StdlibAttribute with full JSON mapping
  - Snake_case JSON tags to match Python-generated format
- Add stdlib_registry_loader.go for local file loading
  - Loads manifest.json and all module JSON files
  - SHA256 checksum verification for data integrity
  - Graceful error handling (logs warnings, continues without stdlib)
- Add stdlib_registry_loader_test.go with comprehensive coverage
  - Tests manifest loading, module loading, checksum validation
  - Tests with actual generated registries (188 modules)
  - Edge case handling (missing/corrupted files)

**Resolution Integration:**
- Add validateStdlibFQN() helper with module alias support
  - Handles os.path -> posixpath platform-specific aliasing
  - Checks functions, classes, constants, and attributes
- Integrate stdlib validation into resolveCallTarget()
  - Checks stdlib before user project registry
  - Non-blocking: stdlib load failures don't break analysis

**Test Results:**
- All tests passing (gradle testGo)
- Zero lint issues (gradle lintGo)
- Successfully loads 188 modules from registries/python3.14/stdlib/v1/
- Resolution improvement: 64.7% -> 66.3% (+90 resolutions)

**Integration:**
- TypeInferenceEngine.StdlibRegistry field added
- Loaded in BuildCallGraph() after builtin registry
- Logs success: "Loaded stdlib registry: 188 modules"

Related: PR #1 (Python stdlib registry generator)
Next: PR #3 (remote registry hosting + deployment)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 2, 2025
Resolves issue where stdlib calls like os.path.join were not being
resolved when modules were imported directly (import os.path).

**Problem:**
- Code like `import os.path; os.path.join()` was not resolving
- importMap.Resolve("os") fails because import is "os.path", not "os"
- Target "os.path.join" never reached stdlib validation logic
- Result: 310+ stdlib calls marked as unresolved "external_framework"

**Solution:**
- Add fallback stdlib check before giving up on resolution
- Check if target itself is a stdlib call (e.g., os.path.join)
- Uses existing validateStdlibFQN() with os.path -> posixpath aliasing

**Impact:**
- Resolution improved: 66.3% -> 72.1% (+310 calls, +5.8%)
- os.* unresolved: 311 -> 1 (99.7% reduction)
- os.path.join no longer in top 20 unresolved patterns

**Test Results:**
- All tests passing (gradle testGo)
- Zero lint issues (gradle lintGo)
- Validated on pre-commit codebase (5,367 calls)

**Remaining Unresolved Stdlib Calls:**
Analysis shows ~135 stdlib calls still unresolved, all requiring
type inference:
- f.read/f.write (62 calls) - need to infer file object type
- parser.add_argument (22 calls) - need function param annotations
- logger.warning (18 calls) - need assignment type inference
- cfg.read/write (33 calls) - need configparser type inference

These require Phase 3 type inference enhancements (not in PR #2 scope).

Related: PR #2 (stdlib registry loader)
Next: PR #3 (remote registry hosting)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 2, 2025
…339)

Resolves issue where stdlib calls like os.path.join were not being
resolved when modules were imported directly (import os.path).

**Problem:**
- Code like `import os.path; os.path.join()` was not resolving
- importMap.Resolve("os") fails because import is "os.path", not "os"
- Target "os.path.join" never reached stdlib validation logic
- Result: 310+ stdlib calls marked as unresolved "external_framework"

**Solution:**
- Add fallback stdlib check before giving up on resolution
- Check if target itself is a stdlib call (e.g., os.path.join)
- Uses existing validateStdlibFQN() with os.path -> posixpath aliasing

**Impact:**
- Resolution improved: 66.3% -> 72.1% (+310 calls, +5.8%)
- os.* unresolved: 311 -> 1 (99.7% reduction)
- os.path.join no longer in top 20 unresolved patterns

**Test Results:**
- All tests passing (gradle testGo)
- Zero lint issues (gradle lintGo)
- Validated on pre-commit codebase (5,367 calls)

**Remaining Unresolved Stdlib Calls:**
Analysis shows ~135 stdlib calls still unresolved, all requiring
type inference:
- f.read/f.write (62 calls) - need to infer file object type
- parser.add_argument (22 calls) - need function param annotations
- logger.warning (18 calls) - need assignment type inference
- cfg.read/write (33 calls) - need configparser type inference

These require Phase 3 type inference enhancements (not in PR #2 scope).

Related: PR #2 (stdlib registry loader)
Next: PR #3 (remote registry hosting)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 4, 2025
Implement BuildDefUseChains algorithm and ComputeStats diagnostics:

Core Implementation:
- BuildDefUseChains: O(n×m) single-pass construction from statements
- ComputeStats: Computes statistics (variables, defs, uses, max per var)
- GetDefs/GetUses: Return empty slices (not nil) for consistency

Test Coverage:
- 20+ comprehensive tests in defuse_test.go
- 100% coverage for BuildDefUseChains and ComputeStats
- Tests cover: empty cases, single/multiple defs/uses, complex flows
- Fixed statement_test.go to expect empty slices (per spec)

This implements PR #3 from the intra-procedural dataflow plan.
Enables efficient "where is X defined/used?" queries for taint analysis.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 4, 2025
Implement BuildDefUseChains algorithm and ComputeStats diagnostics:

Core Implementation:
- BuildDefUseChains: O(n×m) single-pass construction from statements
- ComputeStats: Computes statistics (variables, defs, uses, max per var)
- GetDefs/GetUses: Return empty slices (not nil) for consistency

Test Coverage:
- 20+ comprehensive tests in defuse_test.go
- 100% coverage for BuildDefUseChains and ComputeStats
- Tests cover: empty cases, single/multiple defs/uses, complex flows
- Fixed statement_test.go to expect empty slices (per spec)

This implements PR #3 from the intra-procedural dataflow plan.
Enables efficient "where is X defined/used?" queries for taint analysis.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 4, 2025
Implements BuildDefUseChains algorithm for O(1) variable lookup.

- BuildDefUseChains: Single-pass construction from statements
- ComputeStats: Diagnostics for variables, defs, uses
- 100% test coverage (20+ tests)

Part of intra-procedural dataflow analysis. Stacked on PR #2.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
shivasurya added a commit that referenced this pull request Nov 4, 2025
Implements forward taint analysis for intra-procedural flows.

- Track taint from sources (request.GET, os.getenv) to sinks (eval, exec)
- Assignment & call propagation with confidence decay
- Stdlib integration with hardcoded patterns
- 100% coverage on core algorithm

Stacked on PR #3.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
shivasurya added a commit that referenced this pull request Nov 9, 2025
## Summary
Introduces Python DSL foundation with `calls()` and `variable()` matchers. Generates JSON IR for Go executor (future PRs).

## New Features
- ✅ `pathfinder` Python package (codepathfinder on PyPI)
- ✅ `calls(*patterns)` - Match function/method calls (supports wildcards)
- ✅ `variable(pattern)` - Match variable references (supports wildcards)
- ✅ `@rule` decorator - Mark functions as security rules
- ✅ JSON IR serialization (Python → Go protocol)

## Files Added
- `python-dsl/pathfinder/*.py` (matchers, IR, decorators)
- `python-dsl/tests/*.py` (36 tests, 97% coverage)
- `python-dsl/setup.py`, `pyproject.toml`
- `python-dsl/README.md`
- `sourcecode-parser/build.gradle` (Python tasks)

## Testing
- ✅ `pytest` passes: 36 tests, 97% coverage
- ✅ `black`, `ruff`, `mypy` clean
- ✅ `gradle testPython` passes

## Other Changes
- Moved class_memory.csv and test_memory.csv to perf_tools/

## Next PR
PR #3 will add `flows()` dataflow matcher (taint analysis).

🤖 Generated with Claude Code
shivasurya added a commit that referenced this pull request Nov 10, 2025
## Summary
Introduces Python DSL foundation with `calls()` and `variable()` matchers. Generates JSON IR for Go executor (future PRs).

## New Features
- ✅ `pathfinder` Python package (codepathfinder on PyPI)
- ✅ `calls(*patterns)` - Match function/method calls (supports wildcards)
- ✅ `variable(pattern)` - Match variable references (supports wildcards)
- ✅ `@rule` decorator - Mark functions as security rules
- ✅ JSON IR serialization (Python → Go protocol)

## Files Added
- `python-dsl/pathfinder/*.py` (matchers, IR, decorators)
- `python-dsl/tests/*.py` (36 tests, 97% coverage)
- `python-dsl/setup.py`, `pyproject.toml`
- `python-dsl/README.md`
- `sourcecode-parser/build.gradle` (Python tasks)

## Testing
- ✅ `pytest` passes: 36 tests, 97% coverage
- ✅ `black`, `ruff`, `mypy` clean
- ✅ `gradle testPython` passes

## Other Changes
- Moved class_memory.csv and test_memory.csv to perf_tools/

## Next PR
PR #3 will add `flows()` dataflow matcher (taint analysis).

🤖 Generated with Claude Code
shivasurya added a commit that referenced this pull request Nov 10, 2025
## Summary
Introduces Python DSL foundation with `calls()` and `variable()` matchers. Generates JSON IR for Go executor (future PRs).

## New Features
- ✅ `pathfinder` Python package (codepathfinder on PyPI)
- ✅ `calls(*patterns)` - Match function/method calls (supports wildcards)
- ✅ `variable(pattern)` - Match variable references (supports wildcards)
- ✅ `@rule` decorator - Mark functions as security rules
- ✅ JSON IR serialization (Python → Go protocol)

## Files Added
- `python-dsl/pathfinder/*.py` (matchers, IR, decorators)
- `python-dsl/tests/*.py` (36 tests, 97% coverage)
- `python-dsl/setup.py`, `pyproject.toml`
- `python-dsl/README.md`
- `sourcecode-parser/build.gradle` (Python tasks)

## Testing
- ✅ `pytest` passes: 36 tests, 97% coverage
- ✅ `black`, `ruff`, `mypy` clean
- ✅ `gradle testPython` passes

## Other Changes
- Moved class_memory.csv and test_memory.csv to perf_tools/

## Next PR
PR #3 will add `flows()` dataflow matcher (taint analysis).

🤖 Generated with Claude Code
shivasurya added a commit that referenced this pull request Nov 16, 2025
…n package (PR #4) (#375)

## Summary
Complete Phase 2 PR #4: AST Extraction Features by creating the `resolution` package and completing the `extraction` package. This PR migrates ~2000 LOC from 6 files into a clean hierarchical structure while maintaining full backward compatibility.

## New Package Structure

### resolution/ Package (1,127 lines)
Consolidates all resolution logic for imports, callsites, and type inference:
- **imports.go** (297 lines) - Import extraction with relative import resolution (`from .. import module`)
- **callsites.go** (271 lines) - Call site extraction with full argument tracking
- **inference.go** (155 lines) - Type inference engine managing function scopes and variable bindings
- **return_type.go** (404 lines) - Return type analysis and class instantiation resolution

### extraction/ Package (961 lines)
Completes the extraction layer for Python AST analysis:
- **attributes.go** (540 lines) - Class attribute extraction with type inference integration
- **variables.go** (421 lines) - Variable assignment extraction and type tracking

## Backward Compatibility

All original files updated with wrapper functions and type aliases:
```go
// imports.go - Simple wrapper
func ExtractImports(...) (*core.ImportMap, error) {
    return resolution.ExtractImports(...)
}

// type_inference.go - Type aliases
type TypeInferenceEngine = resolution.TypeInferenceEngine
type FunctionScope = resolution.FunctionScope
```

**Note**: `return_type.go` contains documentation only (no wrapper) due to signature changes requiring direct migration to `resolution` package.

## Test Migration

Moved 10 test files to new packages with full updates:
- **resolution/** (7 files): imports_test.go, imports_relative_test.go, callsites_test.go, inference_test.go, return_type_test.go, return_type_class_test.go
- **extraction/** (3 files): attributes_simple_test.go, attributes_coverage_test.go, variables_test.go

All test fixtures and relative paths adjusted for new locations.

## Build Verification

✅ **gradle buildGo** - SUCCESS  
✅ **gradle testGo** - ALL PASSING (100% pass rate)  
✅ **gradle lintGo** - 0 issues  

## Files Changed (26 files, +2389/-2241 lines)

### New Files
- `resolution/imports.go`
- `resolution/callsites.go`
- `resolution/inference.go`
- `resolution/return_type.go`
- `extraction/attributes.go`
- `extraction/variables.go`

### Modified Files (Backward Compatibility)
- `imports.go` → wrapper to resolution
- `callsites.go` → wrapper to resolution
- `type_inference.go` → type aliases to resolution
- `attribute_extraction.go` → wrapper to extraction
- `variable_extraction.go` → wrapper to extraction
- `return_type.go` → documentation only
- `builder.go` → updated all imports and calls

### Test Migrations
- 7 tests moved to resolution/
- 3 tests moved to extraction/
- 1 test kept in parent (import cycle prevention)

## Breaking Changes

None for most users. Direct imports of functions continue to work through wrappers.

**Only breaking change**: Code directly using `ExtractReturnTypes` or `ResolveClassInstantiation` must update to:
```go
import "github.com/shivasurya/code-pathfinder/sourcecode-parser/graph/callgraph/resolution"

resolution.ExtractReturnTypes(...)
resolution.ResolveClassInstantiation(...)
```

## Dependencies

Built on top of:
- PR #3 (refactor/03-stdlib-taint) - stdlib foundation
- PR #2 (refactor/02-infrastructure-core) - core types  
- PR #1 (refactor/01-foundation-types) - base types

## Related

- Implements specification: `/Users/shiva/src/shivasurya/cpf_plans/pr-details/refactor/pr-04-ast-extraction.md`
- Part of Phase 2 refactoring plan

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 16, 2025
#5) (#376)

## Summary
Complete Phase 2 PR #5: Advanced Resolution by moving ORM patterns, attribute resolution, and method chaining to the `resolution` package. This PR consolidates ~1135 LOC of advanced resolution features into a cohesive package structure.

## New Package Structure

### resolution/ Package (+1135 lines)
Advanced resolution features now centralized:
- **orm.go** (283 lines) - ORM pattern detection and resolution
- **attribute.go** (382 lines) - Attribute resolution with type inference
- **chaining.go** (470 lines) - Method chain parsing and resolution

## Changes by Component

### ORM Resolution (resolution/orm.go)
**Django ORM Support:**
- Pattern detection: `Model.objects.filter()`, `.get()`, `.all()`, etc.
- 29 Django ORM methods recognized
- Model validation against code graph
- Synthetic FQN generation

**SQLAlchemy Support:**
- Pattern detection: `.filter()`, `.filter_by()`, `.first()`, etc.
- 17 SQLAlchemy query methods recognized
- Query builder pattern support

**Exported Functions:**
- `IsDjangoORMPattern(target string) (bool, string)`
- `IsSQLAlchemyORMPattern(target string) (bool, string)`
- `IsORMPattern(target string) (bool, string, string)`
- `ValidateDjangoModel(modelName string, codeGraph *graph.CodeGraph) bool`
- `ResolveDjangoORMCall(...) (string, bool)`
- `ResolveSQLAlchemyORMCall(...) (string, bool)`
- `ResolveORMCall(...) (string, bool)`

### Attribute Resolution (resolution/attribute.go)
**Self-Attribute Calls:**
- Resolves `self.attr.method()` patterns
- Type inference integration for attribute types
- Builtin method resolution
- Call graph integration

**Attribute Placeholders:**
- Resolves `__ATTR__` placeholders in call targets
- Class attribute registry integration
- Failure statistics tracking

**Exported Functions:**
- `ResolveSelfAttributeCall(...) (string, bool, *core.TypeInfo)`
- `PrintAttributeFailureStats()`
- `ResolveAttributePlaceholders(...)`

### Method Chaining (resolution/chaining.go)
**Chain Parsing:**
- Parses `a.b().c()` into individual steps
- Distinguishes function calls from attribute access
- Tracks type through each step

**Chain Resolution:**
- Type propagation across chain steps
- Builtin method integration
- Return type registry integration
- Confidence decay calculation

**Exported Types:**
- `ChainStep` - Represents one link in a chain

**Exported Functions:**
- `ParseChain(target string) []ChainStep`
- `ResolveChainedCall(...) (string, bool, *core.TypeInfo)`

## Backward Compatibility

All original files converted to wrappers:

**orm_patterns.go** (49 lines):
```go
func ResolveORMCall(target string, modulePath string, registry *core.ModuleRegistry, codeGraph *graph.CodeGraph) (string, bool) {
    return resolution.ResolveORMCall(target, modulePath, registry, codeGraph)
}
```

**attribute_resolution.go** (38 lines):
```go
func ResolveSelfAttributeCall(...) (string, bool, *core.TypeInfo) {
    return resolution.ResolveSelfAttributeCall(...)
}
```

**chaining.go** (34 lines):
```go
type ChainStep = resolution.ChainStep

func ResolveChainedCall(...) (string, bool, *core.TypeInfo) {
    return resolution.ResolveChainedCall(...)
}
```

All wrappers include deprecation notices.

## Test Migration

**Moved 2 test files** to resolution package with full updates:
- **resolution/orm_test.go** - 7 tests for ORM pattern detection
- **resolution/chaining_test.go** - 5 tests for chain parsing and resolution

Test updates:
- Package changed to `resolution`
- Imports updated with `core.`, `registry.` prefixes
- Direct calls to resolution functions (no wrappers)
- Tests for unexported functions now work (same package)

## Build Verification

✅ **gradle buildGo** - SUCCESS  
✅ **gradle testGo** - ALL PASSING (100% pass rate)  
✅ **gradle lintGo** - 0 issues  

## Files Changed (9 files, +1226/-1101 lines)

### New Files
- `resolution/orm.go` (283 lines)
- `resolution/attribute.go` (382 lines)
- `resolution/chaining.go` (470 lines)

### Modified Files (Backward Compatibility)
- `orm_patterns.go` → wrapper (49 lines, -234 lines)
- `attribute_resolution.go` → wrapper (38 lines, -344 lines)
- `chaining.go` → wrapper (34 lines, -436 lines)
- `builder.go` → updated function signature

### Test Migrations
- `orm_patterns_test.go` → `resolution/orm_test.go`
- `chaining_test.go` → `resolution/chaining_test.go`

## Breaking Changes

**None for most users.** Existing code continues to work through wrappers.

**Optional migration** (recommended):
```go
// Old
import "github.com/shivasurya/code-pathfinder/sourcecode-parser/graph/callgraph"
callgraph.ResolveORMCall(...)

// New
import "github.com/shivasurya/code-pathfinder/sourcecode-parser/graph/callgraph/resolution"
resolution.ResolveORMCall(...)
```

## Dependencies

Built on top of:
- PR #4 (refactor/04-ast-extraction) - resolution package foundation
- PR #3 (refactor/03-stdlib-taint) - stdlib foundation
- PR #2 (refactor/02-infrastructure-core) - core types
- PR #1 (refactor/01-foundation-types) - base types

## Related

- Implements specification: `/Users/shiva/src/shivasurya/cpf_plans/pr-details/refactor/pr-05-advanced-resolution.md`
- Part of Phase 2 refactoring plan
- Total LOC moved: ~1135 lines

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 21, 2025
Found during PR #6 validation. These bugs prevented argument constraints
from working correctly in production.

## Bug #1: Python DSL - Argument wildcard inheritance
File: python-dsl/codepathfinder/matchers.py:81

The _make_constraint() method was inheriting the pattern wildcard flag,
causing all argument constraints to use wildcard matching when the
pattern itself had wildcards.

Example:
  calls("*.bind", match_position={"0[0]": "0.0.0.0"})

  Before: Would match ALL .bind() calls (wrong!)
  After: Only matches .bind(("0.0.0.0", ...)) (correct)

## Bug #2: Go Executor - Missing argument validation
File: sourcecode-parser/dsl/call_matcher.go:155-162

The getMatchedPattern() method only checked function name patterns,
completely ignoring argument constraints. Since the scan command uses
ExecuteWithContext() which calls getMatchedPattern(), all argument
checking was bypassed in production!

Impact: ALL argument constraints were ignored during scans.

## Bug #3: Tuple extraction - Empty string ambiguity
File: sourcecode-parser/dsl/call_matcher.go:318-350

The extractTupleElement() function returned "" for both "index out of
bounds" and "extracted value is empty string", making them indistinguishable.

Example:
  s.bind(("", 8080))  # Empty string is valid!

  Before: Treated as "out of bounds" error (wrong!)
  After: Returns ("", true) indicating success (correct)

Changed signature to return (string, bool) to distinguish error from
valid empty string.

## Validation Results

Test rule: avoid_bind_to_all_interfaces
Before: 6/6 matches (100% false positives)
After: 3/6 matches (100% accurate)

## Test Changes

- Updated extractTupleElement tests for new (string, bool) signature
- Added test case for tuple with empty string element
- All existing tests pass

🤖 Generated with Claude Code
shivasurya added a commit that referenced this pull request Nov 21, 2025
This introduces the Python DSL API surface for specifying argument constraints in security rules. Rule authors can now use match_name and match_position parameters in the calls matcher to define expected keyword and positional argument values. The DSL automatically generates the appropriate JSON IR structures that the Go executor uses for validation during code analysis.
shivasurya added a commit that referenced this pull request Nov 21, 2025
Found during PR #6 validation. These bugs prevented argument constraints
from working correctly in production.

## Bug #1: Python DSL - Argument wildcard inheritance
File: python-dsl/codepathfinder/matchers.py:81

The _make_constraint() method was inheriting the pattern wildcard flag,
causing all argument constraints to use wildcard matching when the
pattern itself had wildcards.

Example:
  calls("*.bind", match_position={"0[0]": "0.0.0.0"})

  Before: Would match ALL .bind() calls (wrong!)
  After: Only matches .bind(("0.0.0.0", ...)) (correct)

## Bug #2: Go Executor - Missing argument validation
File: sourcecode-parser/dsl/call_matcher.go:155-162

The getMatchedPattern() method only checked function name patterns,
completely ignoring argument constraints. Since the scan command uses
ExecuteWithContext() which calls getMatchedPattern(), all argument
checking was bypassed in production!

Impact: ALL argument constraints were ignored during scans.

## Bug #3: Tuple extraction - Empty string ambiguity
File: sourcecode-parser/dsl/call_matcher.go:318-350

The extractTupleElement() function returned "" for both "index out of
bounds" and "extracted value is empty string", making them indistinguishable.

Example:
  s.bind(("", 8080))  # Empty string is valid!

  Before: Treated as "out of bounds" error (wrong!)
  After: Returns ("", true) indicating success (correct)

Changed signature to return (string, bool) to distinguish error from
valid empty string.

## Validation Results

Test rule: avoid_bind_to_all_interfaces
Before: 6/6 matches (100% false positives)
After: 3/6 matches (100% accurate)

## Test Changes

- Updated extractTupleElement tests for new (string, bool) signature
- Added test case for tuple with empty string element
- All existing tests pass

🤖 Generated with Claude Code
shivasurya added a commit that referenced this pull request Nov 21, 2025
- Detection type badges (Pattern, Taint-Local, Taint-Global)
- Severity-based grouping with detail levels
- Code snippets with line numbers and highlight
- Taint flow visualization
- Summary statistics
- Comprehensive tests with 100% coverage

Part of output standardization feature.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 21, 2025
- Replace old printDetections() with enrichment pipeline
- Add enricher to add context and metadata to detections
- Connect enricher -> formatter flow for rich output
- Keep printDetections() for query command compatibility

Part of output standardization feature.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 22, 2025
- Detection type badges (Pattern, Taint-Local, Taint-Global)
- Severity-based grouping with detail levels
- Code snippets with line numbers and highlight
- Taint flow visualization
- Summary statistics
- Comprehensive tests with 100% coverage

Part of output standardization feature.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 22, 2025
- Replace old printDetections() with enrichment pipeline
- Add enricher to add context and metadata to detections
- Connect enricher -> formatter flow for rich output
- Keep printDetections() for query command compatibility

Part of output standardization feature.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 22, 2025
## Objective

Implement human-readable text output for the scan command with detection type badges, code snippets, severity grouping, and taint flow visualization.

## Changes

### New Files
- `output/text_formatter.go` (268 lines) - TextFormatter implementation
- `output/text_formatter_test.go` (596 lines) - Comprehensive tests

### Modified Files
- `cmd/scan.go` - Integrated enrichment pipeline and text formatter

## Features

✅ **Detection Type Badges**: `[Pattern]`, `[Taint-Local]`, `[Taint-Global]`
✅ **Severity Grouping**: Critical → High → Medium → Low (ordered by priority)
✅ **Detail Levels**:
  - Critical/High: Full details with code snippets, taint flow, confidence
  - Medium/Low: Single-line abbreviated format
✅ **Code Snippets**: Line numbers with highlight markers (`>`)
✅ **Taint Flow Visualization**: Source → Sink with variable tracking
✅ **Summary Statistics**: Total findings, severity breakdown
✅ **Verbose Mode**: Detection method breakdown

## Test Results

- ✅ All Go tests passing (19 packages)
- ✅ Text formatter coverage: **100%**
- ✅ Output package coverage: **98.4%**
- ✅ All Python tests passing (185 tests)
- ✅ Linting: 0 issues

## Commits

1. **Add text formatter with rich output** - Core formatter implementation with 100% test coverage
2. **Integrate text formatter in scan command** - Replace old output with enrichment pipeline

## Example Output

```
Code Pathfinder Security Scan

Results:

Critical Issues (1):

  [critical] [Taint-Local] command-injection: Command Injection
    CWE-78 | A03:2021

    auth/login.py:10

      > 10 | eval(user_input)

    Flow: user_input (line 5) -> eval (line 10)
    Tainted variable 'user_input' reaches dangerous sink without sanitization

    Confidence: High | Detection: Intra-procedural taint analysis

Summary:
  1 findings across 5 rules
  1 critical
```

## Dependencies

- **Stacked on**: PR #2 (shiva/output-logging-system)
- **Blocks**: PR #4 (JSON & CSV Formatters)

## Tech Spec Reference

Implements Section 4.1 of [output-standardization tech spec](https://github.com/shivasurya/code-pathfinder/blob/main/docs/planning/output-standardization/tech-spec.md)

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
shivasurya added a commit that referenced this pull request Nov 22, 2025
## Summary
Implements JSON and CSV output formatters for the `ci` command, replacing the old inline JSON generation with a modular, well-tested implementation.

**Part of output-standardization tech spec (Stacked PRs)**
- ✅ PR #1: Logging System Infrastructure (#391) - **Merged**
- ✅ PR #2: Output Package Foundation (#392) - **In Review**
- ✅ PR #3: Text Formatter for Scan Command (#393) - **In Review**
- 🔄 PR #4: JSON and CSV Formatters ← **This PR**

## Changes

### New Files
- `output/json_formatter.go` (235 lines)
  - Enhanced JSON output with rich metadata structure
  - Tool, scan, results, summary, and errors sections
  - Code snippets with configurable context lines
  - Taint flow source/sink information
  - CWE, OWASP, and reference metadata
  
- `output/csv_formatter.go` (123 lines)
  - CSV output for CI/CD integration
  - 17 columns: severity, confidence, rule_id, rule_name, cwe, owasp, file, line, column, function, message, detection_type, detection_scope, source_line, sink_line, tainted_var, sink_call
  - Proper escaping via encoding/csv package

- `output/json_formatter_test.go` (415 lines)
  - Comprehensive tests achieving 100% coverage
  - Structure validation, snippet handling, metadata, pattern vs taint detection

- `output/csv_formatter_test.go` (395 lines)
  - Comprehensive tests achieving 100% coverage
  - Header validation, escaping, multiple rows, zero values

### Modified Files
- `cmd/ci.go`
  - Replaced old `generateJSONOutput()` with new formatter integration
  - Added enrichment pipeline using `output.NewEnricher()`
  - Updated output format validation to include "csv"
  - Added CSV formatter support
  - Updated help text and examples
  - Exit code 1 when vulnerabilities found (for CI/CD)

- `cmd/ci_test.go`
  - Skipped obsolete `TestGenerateJSONOutput` (replaced by new formatter tests)

- `main_test.go`
  - Updated expected help text to include CSV output format

## JSON Output Structure
```json
{
  "tool": {
    "name": "Code Pathfinder",
    "version": "1.0.0",
    "url": "https://codepathfinder.dev"
  },
  "scan": {
    "target": "/path/to/project",
    "timestamp": "2025-01-21T10:30:00Z",
    "duration": 5.43,
    "rules_executed": 12
  },
  "results": [{
    "rule_id": "sql-injection",
    "rule_name": "SQL Injection",
    "message": "Unsanitized user input flows to SQL query",
    "severity": "critical",
    "confidence": "high",
    "location": {
      "file": "src/main.py",
      "line": 42,
      "column": 8,
      "function": "process_user",
      "snippet": {
        "start_line": 40,
        "end_line": 44,
        "lines": ["...", "query = f\"SELECT * FROM users WHERE id={user_id}\"", "..."]
      }
    },
    "detection": {
      "type": "taint-local",
      "scope": "intra-procedural",
      "confidence_score": 0.95,
      "source": {"line": 38, "variable": "user_id"},
      "sink": {"line": 42, "call": "execute"}
    },
    "metadata": {
      "cwe": ["CWE-89"],
      "owasp": ["A03:2021"],
      "references": ["https://..."]
    }
  }],
  "summary": {
    "total": 5,
    "by_severity": {"critical": 2, "high": 3},
    "by_detection_type": {"taint-local": 4, "pattern": 1}
  },
  "errors": []
}
```

## CSV Output Format
```csv
severity,confidence,rule_id,rule_name,cwe,owasp,file,line,column,function,message,detection_type,detection_scope,source_line,sink_line,tainted_var,sink_call
critical,high,sql-injection,SQL Injection,CWE-89,A03:2021,src/main.py,42,8,process_user,Unsanitized user input flows to SQL query,taint-local,intra-procedural,38,42,user_id,execute
```

## Testing
- All tests passing (100% coverage for both formatters)
- Output package overall: 98.1% coverage
- Linting checks passed
- Integration tests with ci command verified

## Usage Examples
```bash
# Generate JSON report
pathfinder ci --rules rules/ --project . --output json > results.json

# Generate CSV report  
pathfinder ci --rules rules/ --project . --output csv > results.csv

# Generate SARIF report (existing)
pathfinder ci --rules rules/ --project . --output sarif > results.sarif
```

## Breaking Changes
- Old `generateJSONOutput()` function removed from cmd/ci.go
- JSON output structure changed to new rich format (snake_case fields)
- Exit code behavior unchanged (exits 1 when vulnerabilities found)

## Stack Status
This PR stacks on:
- **PR #3**: shiva/output-text-formatter (#393) ← base branch
- **PR #2**: shiva/output-logging-system (#392)
- **main**: Production branch

Next PR:
- PR #5: SARIF Formatter Enhancement (will stack on this PR)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
shivasurya added a commit that referenced this pull request Nov 22, 2025
## Summary
Implements enhanced SARIF formatter with code flows, related locations, and rich metadata for optimal GitHub Code Scanning integration.

**Part of output-standardization tech spec (Stacked PRs)**
- ✅ PR #1: Logging System Infrastructure (#391) - **Merged**
- ✅ PR #2: Output Package Foundation (#392) - **In Review**
- ✅ PR #3: Text Formatter for Scan Command (#393) - **In Review**
- ✅ PR #4: JSON and CSV Formatters (#394) - **In Review**
- 🔄 PR #5: Enhanced SARIF Formatter ← **This PR**

## Changes

### New Files
- `output/sarif_formatter.go` (290 lines)
  - SARIF 2.1.0 compliant output formatter
  - Code flows for taint path visualization (source → sink)
  - Related locations for taint sources
  - Help text with markdown and CWE references
  - Security severity scores (9.0, 7.0, 5.0, 3.0)
  - Rule properties: tags, precision
  - Deduplicates rules across multiple detections

- `output/sarif_formatter_test.go` (519 lines)
  - Comprehensive tests achieving 97.5% coverage
  - Tests for version, tool metadata, rules, results
  - Code flow generation tests (taint-local, taint-global)
  - Related locations validation
  - Pattern vs taint detection differentiation

### Modified Files
- `cmd/ci.go`
  - Replaced old `generateSARIFOutput()` with new formatter
  - Uses enriched detections for rich output
  - Removed unused imports (sarif library, json, encoding/json)
  - Consistent pattern with JSON and CSV formatters

- `cmd/ci_test.go`
  - Skipped obsolete SARIF tests
  - Removed unused helper functions

## Key Features

### Code Flows
Taint detections automatically include code flows showing the path from source to sink:

```json
{
  "codeFlows": [{
    "message": {"text": "Taint flow from line 10 to line 20"},
    "threadFlows": [{
      "locations": [
        {
          "location": {"physicalLocation": {"region": {"startLine": 10}}},
          "message": {"text": "Taint source: user_input"}
        },
        {
          "location": {"physicalLocation": {"region": {"startLine": 20}}},
          "message": {"text": "Taint sink: os.system"}
        }
      ]
    }]
  }]
}
```

### Help Text with Markdown
Rules include rich help text with CWE references:

```markdown
## Command Injection

User input flows to shell command without sanitization

### References
- [CWE-78](https://cwe.mitre.org/data/definitions/78.html)
```

### Security Severity Scores
GitHub-compatible severity scores for prioritization:
- Critical: 9.0
- High: 7.0
- Medium: 5.0
- Low: 3.0

### Rule Properties
```json
{
  "properties": {
    "tags": ["security"],
    "security-severity": "9.0",
    "precision": "high"
  }
}
```

## Benefits over Old Implementation

| Feature | Old | New |
|---------|-----|-----|
| Code flows | ❌ None | ✅ Source → Sink visualization |
| Related locations | ❌ None | ✅ Taint sources highlighted |
| Help text | ❌ Plain text | ✅ Markdown with references |
| Security severity | ❌ Level only | ✅ Numeric scores for GitHub |
| Rule properties | ❌ None | ✅ Tags, precision |
| Pattern detection | ❌ Same as taint | ✅ No code flows (correct) |
| Test coverage | ❌ ~60% | ✅ 97.5% |

## Testing
- All tests passing (97.5% coverage on SARIF formatter)
- Output package overall: 97.5% coverage
- Linting checks passed
- Integration with ci command verified

## Usage Examples
```bash
# Generate enhanced SARIF report with code flows
pathfinder ci --rules rules/ --project . --output sarif > results.sarif

# Upload to GitHub Code Scanning
gh api /repos/:owner/:repo/code-scanning/sarifs -F [email protected]

# View in GitHub UI with code flows highlighted
```

## SARIF Output Sample
```json
{
  "version": "2.1.0",
  "runs": [{
    "tool": {
      "driver": {
        "name": "Code Pathfinder",
        "version": "0.0.25",
        "rules": [{
          "id": "sql-injection",
          "name": "SQL Injection",
          "fullDescription": {"text": "Unsanitized user input flows to SQL query (CWE-89, A03:2021)"},
          "helpUri": "https://github.com/shivasurya/code-pathfinder",
          "defaultConfiguration": {"level": "error"},
          "properties": {
            "tags": ["security"],
            "security-severity": "9.0",
            "precision": "high"
          }
        }]
      }
    },
    "results": [{
      "ruleId": "sql-injection",
      "message": {"text": "Unsanitized user input flows to SQL query (sink: execute, confidence: 95%)"},
      "locations": [{
        "physicalLocation": {
          "artifactLocation": {"uri": "src/db/queries.py"},
          "region": {"startLine": 42, "startColumn": 8}
        }
      }],
      "codeFlows": [...],
      "relatedLocations": [...]
    }]
  }]
}
```

## Breaking Changes
- Old `generateSARIFOutput()` function removed
- SARIF output structure enhanced with additional fields
- Pattern matches no longer include code flows (correct behavior)

## Stack Status
This PR stacks on:
- **PR #4**: shiva/output-json-csv-formatters (#394) ← base branch
- **PR #3**: shiva/output-text-formatter (#393)
- **PR #2**: shiva/output-logging-system (#392)
- **main**: Production branch

Next PR:
- PR #6: Exit Code Standardization (will stack on this PR)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
shivasurya added a commit that referenced this pull request Dec 8, 2025
Adds tree-sitter-dockerfile integration for AST-based parsing:
- DockerfileParser with Parse() and ParseFile() methods
- AST traversal and instruction detection
- Basic instruction conversion (full impl in PR #3)
- Comprehensive test coverage for all 18 instruction types

All parsing has 100% test coverage.

Files added:
- sast-engine/graph/parser_dockerfile.go
- sast-engine/graph/parser_dockerfile_test.go

Dependencies:
- Uses github.com/smacker/go-tree-sitter/dockerfile

Part of: Dockerfile & Docker Compose Support
Depends on: PR #1 (Core Data Structures)
Next PR: #3 Instruction Converters
shivasurya added a commit that referenced this pull request Dec 8, 2025
Adds tree-sitter-dockerfile integration for AST-based parsing in docker/ subdirectory:
- DockerfileParser with Parse() and ParseFile() methods
- AST traversal and instruction detection
- Basic instruction conversion (full impl in PR #3)
- Comprehensive test coverage for all 18 instruction types

All parsing has 100% test coverage.

Files added:
- sast-engine/graph/docker/parser.go
- sast-engine/graph/docker/parser_test.go

Dependencies:
- Uses github.com/smacker/go-tree-sitter/dockerfile

Part of: Dockerfile & Docker Compose Support
Depends on: PR #1 (Core Data Structures)
Next PR: #3 Instruction Converters
shivasurya added a commit that referenced this pull request Dec 9, 2025
…kerfile instructions

Implement specialized converter functions for all Dockerfile instruction types:
- FROM: Extract image, tag, digest, and stage alias
- RUN/CMD/ENTRYPOINT: Parse shell and exec forms
- COPY/ADD: Extract source paths, destination, and flags (--from, --chown)
- ENV/ARG: Parse environment variables and build arguments
- USER: Parse user:group format
- EXPOSE: Extract ports and protocols
- WORKDIR: Track working directory and path type
- VOLUME/SHELL: Parse JSON arrays and paths
- HEALTHCHECK: Extract all health check options and command
- LABEL: Parse key-value label pairs
- ONBUILD/STOPSIGNAL/MAINTAINER: Extract instruction details

Key implementation details:
- Raw text parsing for robust handling of tree-sitter variations
- Helper functions for parameter/path/JSON array extraction
- 100% test coverage with comprehensive test suite
- All converters now return void instead of error

Part of PR #3: Instruction Converters
Stacked on: docker/02-tree-sitter-integration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
shivasurya added a commit that referenced this pull request Dec 10, 2025
## Executive Summary

This PR introduces the **foundational data structures** needed for Dockerfile parsing and analysis. It contains **ZERO behavioral changes** to the existing codebase - only new type definitions, constructors, and tests.

## File Structure

Following the existing pattern (`java/`, `python/`), files are organized in:
```
sast-engine/graph/docker/
├── node.go           (DockerfileNode - unified instruction representation)
├── graph.go          (DockerfileGraph + BuildStage - multi-stage support)
├── node_test.go      (Comprehensive tests)
└── graph_test.go     (Comprehensive tests)
```

## Why This is Safe

- ✅ No modifications to existing files
- ✅ No integration with existing logic
- ✅ All new code is isolated in new package
- ✅ 100% test coverage on all new code
- ✅ All tests pass: `gradle buildGo && gradle testGo && gradle lintGo`

## Quality Metrics

| Metric | Result |
|--------|--------|
| Build Status | ✅ BUILD SUCCESSFUL |
| Test Coverage | ✅ 100% |
| Linting | ✅ 0 issues |
| Test Execution | ✅ All tests PASS |

## Code Examples

### Creating a FROM instruction node:
```go
node := docker.NewDockerfileNode("FROM", 1)
node.BaseImage = "ubuntu"
node.ImageTag = "20.04"
node.StageAlias = "builder"
```

### Building a Dockerfile graph:
```go
graph := docker.NewDockerfileGraph("/path/to/Dockerfile")
graph.AddInstruction(fromNode)
graph.AddInstruction(runNode)

// Security check
if graph.IsRunningAsRoot() {
    // Container runs as root
}
```

### Multi-stage build analysis:
```go
stages := graph.GetStages()
for _, stage := range stages {
    fmt.Printf("Stage %s: %s:%s\n", 
        stage.Alias, stage.BaseImage, stage.ImageTag)
}
```

## Part of Stack

**Dockerfile & Docker Compose Support** implementation:
- ✅ **PR #1**: Core Data Structures (this PR)
- ⏳ **PR #2**: Tree-sitter Integration
- ⏳ **PR #3**: AST Conversion Layer
- ⏳ **PR #4**: Python DSL Extensions

## Testing Coverage

- ✅ Constructor and initialization tests
- ✅ Flag operations (GetFlag, HasFlag)
- ✅ Helper methods (IsRootUser, UsesLatestTag)
- ✅ Graph operations (AddInstruction, GetInstructions)
- ✅ Multi-stage analysis (AnalyzeBuildStages, GetStageByAlias)
- ✅ Edge cases (empty graph, single stage, no USER instruction)
shivasurya added a commit that referenced this pull request Dec 10, 2025
Adds tree-sitter-dockerfile integration for AST-based parsing in docker/ subdirectory:
- DockerfileParser with Parse() and ParseFile() methods
- AST traversal and instruction detection
- Basic instruction conversion (full impl in PR #3)
- Comprehensive test coverage for all 18 instruction types

All parsing has 100% test coverage.

Files added:
- sast-engine/graph/docker/parser.go
- sast-engine/graph/docker/parser_test.go

Dependencies:
- Uses github.com/smacker/go-tree-sitter/dockerfile

Part of: Dockerfile & Docker Compose Support
Depends on: PR #1 (Core Data Structures)
Next PR: #3 Instruction Converters
shivasurya added a commit that referenced this pull request Dec 10, 2025
## Executive Summary

This PR adds **tree-sitter-dockerfile integration** for AST-based parsing of Dockerfiles. It follows the existing pattern used for Python and Java parsers and provides the foundation for full instruction parsing in PR #3.

## File Structure

Following the existing pattern (`java/`, `python/`), files are organized in:
```
sast-engine/graph/docker/
├── node.go         (DockerfileNode - unified instruction representation)
├── graph.go        (DockerfileGraph + BuildStage - multi-stage support)
├── parser.go       (DockerfileParser with AST traversal)
├── node_test.go    (Tests for data structures)
├── graph_test.go   (Tests for graph operations)
└── parser_test.go  (Tests for parsing - all 18 instructions)
```

## Why This is Safe

- ✅ No modifications to existing files
- ✅ All new code isolated in docker/ subdirectory
- ✅ 100% test coverage on all new code
- ✅ Placeholder converters (full implementation in PR #3)
- ✅ All tests pass: `gradle buildGo && gradle testGo && gradle lintGo`

## Quality Metrics

| Metric | Result |
|--------|--------|
| Build Status | ✅ BUILD SUCCESSFUL |
| Test Coverage | ✅ 100% |
| Linting | ✅ 0 issues |
| Test Execution | ✅ All tests PASS |

## Key Features

### DockerfileParser
- `Parse(filePath, content)` - parses Dockerfile bytes into DockerfileGraph
- `ParseFile(path)` - convenience method for parsing from file
- AST traversal with instruction detection
- Multi-stage build support
- Handles syntax errors gracefully (continues with partial parse)

### Instruction Detection
- Recognizes all 18 Dockerfile instruction types
- FROM, RUN, COPY, ADD, ENV, ARG, USER, EXPOSE, WORKDIR
- CMD, ENTRYPOINT, VOLUME, SHELL, HEALTHCHECK, LABEL
- ONBUILD, STOPSIGNAL, MAINTAINER

### Current Implementation
- Basic instruction type detection and line number tracking
- Placeholder conversion logic (creates DockerfileNode with type and line)
- Full field population deferred to PR #3

## Code Examples

### Basic parsing:
```go
import "github.com/shivasurya/code-pathfinder/sast-engine/graph/docker"

parser := docker.NewDockerfileParser()
dockerfileGraph, err := parser.ParseFile("/path/to/Dockerfile")

// Check what instructions exist
if dockerfileGraph.HasInstruction("USER") {
    users := dockerfileGraph.GetInstructions("USER")
    // Process USER instructions
}
```

### Multi-stage detection:
```go
if dockerfileGraph.IsMultiStage() {
    stages := dockerfileGraph.GetStages()
    fmt.Printf("Found %d build stages\n", len(stages))
}
```

## Testing Coverage

- ✅ Parser initialization
- ✅ Simple Dockerfile parsing (4 instructions)
- ✅ Multi-stage Dockerfile parsing
- ✅ All 18 instruction types detected
- ✅ Empty Dockerfile handling
- ✅ Line number accuracy
- ✅ Instruction type extraction
- ✅ Comments and blank lines skipped

## Part of Stack

**Dockerfile & Docker Compose Support** implementation:
- ✅ **PR #1**: Core Data Structures
- ✅ **PR #2**: Tree-sitter Integration (this PR)
- ⏳ **PR #3**: AST Conversion Layer
- ⏳ **PR #4**: Python DSL Extensions

## Dependencies

Uses `github.com/smacker/go-tree-sitter/dockerfile` for Dockerfile grammar parsing (MIT license).
shivasurya added a commit that referenced this pull request Dec 10, 2025
…kerfile instructions

Implement specialized converter functions for all Dockerfile instruction types:
- FROM: Extract image, tag, digest, and stage alias
- RUN/CMD/ENTRYPOINT: Parse shell and exec forms
- COPY/ADD: Extract source paths, destination, and flags (--from, --chown)
- ENV/ARG: Parse environment variables and build arguments
- USER: Parse user:group format
- EXPOSE: Extract ports and protocols
- WORKDIR: Track working directory and path type
- VOLUME/SHELL: Parse JSON arrays and paths
- HEALTHCHECK: Extract all health check options and command
- LABEL: Parse key-value label pairs
- ONBUILD/STOPSIGNAL/MAINTAINER: Extract instruction details

Key implementation details:
- Raw text parsing for robust handling of tree-sitter variations
- Helper functions for parameter/path/JSON array extraction
- 100% test coverage with comprehensive test suite
- All converters now return void instead of error

Part of PR #3: Instruction Converters
Stacked on: docker/02-tree-sitter-integration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants