feat(callgraph): Phase 2 - Complete Type Inference with Inter-Procedural Propagation

[shivasurya]

Summary

This PR completes Phase 2 of Python Type Inference, building on Phase 1 (#334) to add inter-procedural type propagation, return type inference, and critical bug fixes. Phase 2 achieves 63.6% overall call resolution on label-studio (up from 56.3%), with 920 type-inferred resolutions and 68.4% class types.

What's Changed

🎯 Core Features

1. Return Type Inference (Tasks 6-7)

• Extracts return statements from all Python functions
• Infers types from return expressions (literals, class instantiations, function calls)
• Merges multiple return types per function with highest confidence
• Foundation for inter-procedural propagation

2. Variable Assignment Tracking (Task 8)

• Tracks variable assignments in function and module scopes
• Infers types from assignment RHS (literals, calls, instantiations)
• Creates placeholders for function call results (call:funcName)
• Resolves placeholders with actual return types in second pass

3. Inter-Procedural Type Propagation (Task 9)

• Propagates return types through function call chains
• Example: user = create_user() → resolve create_user() return type → type user
• Enables user.save() resolution via variable type lookup
• Confidence decay for propagation (0.95x multiplier)

4. Attribute Chain Resolution (Task 10)

• Resolves variable.method() calls using inferred variable types
• Checks function scope first, then falls back to module scope
• Validates methods exist in code graph or builtin registry
• Python-specific: Strips class names for module-level method lookup

🐛 Critical Bug Fixes

Bug 1: Class Instantiation Not Detected

Problem: response = HttpResponse() created unresolvable placeholders instead of class types.

Fix: Call ResolveClassInstantiation() before creating placeholders in inferTypeFromExpression().

Impact: Test cases 60%→75% class types, label-studio 0%→67.3% class types (3x improvement).

Bug 2: Qualified Function Names

Problem: logging.getLogger created invalid FQNs like module.logging.getLogger.

Fix: Check if function name contains dots before qualifying with scope.

Bug 3: Module-Level Variable Accessibility

Problem: Module-level logger = logging.getLogger(__name__) inaccessible from functions.

Fix: Implemented scope fallback - check function scope THEN module scope.

Impact: ~400 additional resolutions.

📊 Results

Test Cases:

• ✅ 94.1% resolution (16/17 calls)
• ✅ 75% class types (6/8 type-inferred)
• ✅ 50% of resolutions via type inference

Label-Studio (27k+ methods):

• Overall: 63.6% resolution (12,186 / 19,167 calls) - up from 56.3%
• Type Inference: 920 resolutions (7.5% of total)
• Class Types: 68.4% (629/920)
• Confidence: 0.85 average
• High Confidence: 31.4% (0.9-1.0)

By Inference Source:

• class_instantiation_local: 59.3% (546)
• literal: 31.4% (289)
• class_instantiation_heuristic: 9.0% (83)
• function_call_propagation: 0.2% (2)

Remaining Failures:

• attribute_chain: 2,926 (15.3%) - mostly external stdlib functions
• not_in_imports: 1,788 (9.3%)
• orm_pattern: 1,124 (5.9%)

📝 Implementation Details

Three-Pass Algorithm:

1. Pass 1: Extract return types from all functions
2. Pass 2: Extract variable assignments, resolve placeholders with return types
3. Pass 3: Resolve call sites using type inference + traditional methods

Module-Level Variable Support:

// Check function scope first
if functionScope != nil {
    if b, exists := functionScope.Variables[base]; exists {
        binding = b
    }
}
// Fall back to module scope
if binding == nil {
    moduleScope := typeEngine.GetScope(currentModule)
    if moduleScope != nil {
        if b, exists := moduleScope.Variables[base]; exists {
            binding = b
        }
    }
}

Python Method Resolution:

// Python stores class methods at module level (test.save, not test.User.save)
// Strip class name and lookup module.method
modulePart := typeFQN[:lastDot]  // "test"
pythonMethodFQN := modulePart + "." + rest  // "test.save"

📁 Files Changed

• graph/callgraph/builder.go: Three-pass algorithm, module scope fallback, Python method resolution
• graph/callgraph/type_inference.go: Return type merging, placeholder resolution, qualified names fix
• graph/callgraph/variable_extraction.go: Variable tracking, class instantiation detection, module-level support
• graph/callgraph/return_type.go: Return statement extraction, type inference
• cmd/resolution_report.go: Enhanced reporting with type inference statistics

🔍 Testing

All existing tests pass + new integration tests:

• ✅ Return type inference tests
• ✅ Variable assignment tests
• ✅ Inter-procedural propagation tests
• ✅ Module-level variable tests

📈 Performance

• Memory: ~2.18 GB for label-studio (27k methods)
• Processing: ~5 seconds for graph building with type inference
• Three-pass overhead: Minimal (<500ms for large projects)

🎯 Next Steps (Phase 3)

Phase 2 revealed the next major blocker: external function return types

Top Unresolved Placeholders:

1. logging.getLogger - 530 failures (stdlib with unknown return type)
2. business_client.get/post - 76 failures (API client methods)
3. Django ORM patterns - ~100 failures (.objects.filter(), .objects.get())

Phase 3 Goals:

• Add return type mappings for common stdlib functions (logging, datetime, uuid, etc.)
• Enhanced Django ORM pattern recognition
• Import tracking for external modules
• Function parameter type inference

Breaking Changes

None - all changes are additive and backward compatible.

Related Issues

• Builds on feat(callgraph): Python Type Inference for Improved Call Resolution #334 (Phase 1: Basic Type Inference)
• Addresses attribute chain resolution failures
• Improves Python callgraph accuracy for security analysis

Checklist

• All tests passing
• No linting errors
• Performance validated on large codebase (label-studio)
• Comprehensive commit messages
• Documentation in code comments

---

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

[safedep]

SafeDep Report Summary

No dependency changes detected. Nothing to scan.

_{This report is generated by SafeDep Github App}

[codecov]

Codecov Report

❌ Patch coverage is 70.14218% with 126 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.76%. Comparing base (8b53631) to head (1810807).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
sourcecode-parser/cmd/resolution_report.go	9.63%	74 Missing and 1 partial ⚠️
sourcecode-parser/graph/callgraph/builder.go	69.51%	18 Missing and 7 partials ⚠️
sourcecode-parser/graph/callgraph/return_type.go	88.29%	17 Missing and 7 partials ⚠️
...code-parser/graph/callgraph/variable_extraction.go	91.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #335      +/-   ##
==========================================
- Coverage   76.03%   75.76%   -0.28%     
==========================================
  Files          38       39       +1     
  Lines        4102     4485     +383     
==========================================
+ Hits         3119     3398     +279     
- Misses        877      969      +92     
- Partials      106      118      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.