Skip to content

Conversation

@shivasurya
Copy link
Owner

Summary

Fixes stdlib resolution issue where calls like os.path.join() were not being resolved when modules were imported directly with import os.path.

Depends on: #338 (PR #2 - Stdlib loader)

Problem

When Python code imports stdlib modules directly (e.g., import os.path), the resolution logic was failing:

import os.path  # Import os.path directly, not just os

result = os.path.join(a, b)  # Call os.path.join()

Why it failed:

  1. importMap has "os.path""os.path" mapping
  2. When resolving os.path.join, we split into base="os" and rest="path.join"
  3. importMap.Resolve("os") returns false (because import is "os.path", not "os")
  4. Never reaches stdlib validation → marked as unresolved

Result: 310+ stdlib calls marked as external_framework failures

Solution

Added fallback stdlib check before giving up on resolution:

// builder.go:738-744
// PR #2: Last resort - check if target is a stdlib call (e.g., os.path.join)
// This handles cases where stdlib modules are imported directly (import os.path)
if typeEngine != nil && typeEngine.StdlibRegistry != nil {
    if validateStdlibFQN(target, typeEngine.StdlibRegistry) {
        return target, true, nil
    }
}

This checks the full target string (e.g., os.path.join) against the stdlib registry with platform-specific aliasing (os.pathposixpath).

Impact

Resolution Improvement

Before this fix (PR #2 only):

  • Resolution: 66.3% (3,561/5,367 calls)
  • Unresolved: 1,806 calls
  • os.path.join in top unresolved: 107 occurrences
  • external_framework failures: 311 (including os.*: 311)

After this fix:

  • Resolution: 72.1% (3,871/5,367 calls)
  • Unresolved: 1,496 calls
  • os.path.join: FULLY RESOLVED (not in top 20 unresolved)
  • external_framework failures: 1 (down from 311)
    • os.*: 1 (99.7% reduction)

Net improvement:

  • +310 resolutions (5.8 percentage points)
  • Resolved 310 out of 311 os. stdlib calls*

Top 20 Unresolved (After Fix)

Now mostly pytest fixtures and third-party libraries:

  1. tmpdir.join (86) - pytest fixture
  2. tmp_path.joinpath (71) - pytest fixture
  3. in_git_dir.join (42) - pytest fixture
  4. cap_out.get (35) - pytest fixture
  5. f.read (32) - stdlib, needs type inference
  6. f.write (30) - stdlib, needs type inference
  7. cfgv.validate (27) - third-party
  8. parser.add_argument (22) - stdlib, needs type inference
  9. logger.warning (18) - stdlib, needs type inference

Remaining Stdlib Calls Analysis

~135 stdlib calls still unresolved (all require type inference):

  • f.read/f.write (62) - need to infer f is a file object
  • parser.add_argument (22) - need to infer from type annotation
  • logger.warning (18) - need to infer from assignment
  • cfg.read/write (33) - need to infer configparser type

These require Phase 3 type inference enhancements (out of scope):

  • Variable assignment type inference: logger = logging.getLogger()
  • Function parameter annotations: parser: argparse.ArgumentParser
  • Context manager types: with open() as f

~361 non-stdlib calls (cannot be resolved without external sources):

  • pytest fixtures (214) - dynamically generated
  • Third-party libs (68) - cfgv, re_assert
  • Custom objects (79) - project-specific

Test Results

Build & Tests

gradle buildGo  # ✅ SUCCESS
gradle testGo   # ✅ ALL TESTS PASS
gradle lintGo   # ✅ 0 ISSUES

Validation (pre-commit codebase - 5,367 calls)

Total improvement from baseline:

Files Changed

  • graph/callgraph/builder.go - Added fallback stdlib check (8 lines)

Part of Stdlib Registry System

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

Resolves issue where stdlib calls like os.path.join were not being
resolved when modules were imported directly (import os.path).

**Problem:**
- Code like `import os.path; os.path.join()` was not resolving
- importMap.Resolve("os") fails because import is "os.path", not "os"
- Target "os.path.join" never reached stdlib validation logic
- Result: 310+ stdlib calls marked as unresolved "external_framework"

**Solution:**
- Add fallback stdlib check before giving up on resolution
- Check if target itself is a stdlib call (e.g., os.path.join)
- Uses existing validateStdlibFQN() with os.path -> posixpath aliasing

**Impact:**
- Resolution improved: 66.3% -> 72.1% (+310 calls, +5.8%)
- os.* unresolved: 311 -> 1 (99.7% reduction)
- os.path.join no longer in top 20 unresolved patterns

**Test Results:**
- All tests passing (gradle testGo)
- Zero lint issues (gradle lintGo)
- Validated on pre-commit codebase (5,367 calls)

**Remaining Unresolved Stdlib Calls:**
Analysis shows ~135 stdlib calls still unresolved, all requiring
type inference:
- f.read/f.write (62 calls) - need to infer file object type
- parser.add_argument (22 calls) - need function param annotations
- logger.warning (18 calls) - need assignment type inference
- cfg.read/write (33 calls) - need configparser type inference

These require Phase 3 type inference enhancements (not in PR #2 scope).

Related: PR #2 (stdlib registry loader)
Next: PR #3 (remote registry hosting)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@safedep
Copy link

safedep bot commented Nov 2, 2025

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

This report is generated by SafeDep Github App

@shivasurya shivasurya merged commit 5026a10 into main Nov 2, 2025
3 checks passed
@shivasurya shivasurya deleted the feat/stdlib-resolution-fix branch November 2, 2025 01:01
@codecov
Copy link

codecov bot commented Nov 2, 2025

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.16%. Comparing base (982129c) to head (11c7dbb).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
sourcecode-parser/graph/callgraph/builder.go 0.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #339      +/-   ##
==========================================
- Coverage   74.20%   74.16%   -0.05%     
==========================================
  Files          45       45              
  Lines        5288     5291       +3     
==========================================
  Hits         3924     3924              
- Misses       1199     1201       +2     
- Partials      165      166       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@shivasurya shivasurya self-assigned this Nov 2, 2025
@shivasurya shivasurya added enhancement New feature or request go Pull requests that update go code labels Nov 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants