Skip to content

Claude filtering silently disabled due to deprecated Haiku model in validate_api_access #69

@rsharma-figma

Description

@rsharma-figma

Summary

The validate_api_access() method in claude_api_client.py hardcodes claude-3-5-haiku-20241022, which was retired today (2026-02-19). When this validation call fails, FindingsFilter.__init__ silently disables Claude-based false positive filtering, causing all findings to pass through with "justification": "Claude filtering disabled" and a default confidence_score of 10.0.

Impact

  • The second-pass Claude filtering (which checks whether findings are false positives) is silently skipped for all findings
  • No visible error in the job logs — the failure is logged to claudecode-error.log (stderr) which is only surfaced on full scan failure
  • Findings still appear with _filter_metadata.confidence_score: 10.0 and justification: "Claude filtering disabled", which is easy to miss

Root Cause

In claude_api_client.py:61:

def validate_api_access(self) -> Tuple[bool, str]:
    try:
        self.client.messages.create(
            model="claude-3-5-haiku-20241022",  # <-- hardcoded deprecated model
            max_tokens=10,
            messages=[{"role": "user", "content": "Hello"}],
            timeout=10
        )
        return True, ""
    except Exception as e:
        return False, f"API validation failed: {error_msg}"

And in findings_filter.py, when validation fails:

valid, error = self.claude_client.validate_api_access()
if not valid:
    self.claude_client = None
    self.use_claude_filtering = False  # silently disabled

Observed In

  • GitHub Actions run
  • ENABLE_CLAUDE_FILTERING was true, ANTHROPIC_API_KEY was set, but filtering was still disabled

Suggested Fix

Use the same model that's already configured for the scan (passed via CLAUDE_MODEL env var / the model parameter) instead of hardcoding a specific Haiku model for validation. For example:

def validate_api_access(self) -> Tuple[bool, str]:
    try:
        self.client.messages.create(
            model=self.model,  # use the configured model
            max_tokens=10,
            messages=[{"role": "user", "content": "Hello"}],
            timeout=10
        )
        return True, ""
    except Exception as e:
        return False, f"API validation failed: {str(e)}"

Alternatively, update the hardcoded model to claude-haiku-4-5-20251001 (the current Haiku).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions