Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Sep 2, 2025

This PR enhances the terminal environment to provide source code context for algorithmic errors (assertion failures), bringing them to parity with runtime and syntax errors which already show helpful context.

Problem

Previously, when an assertion like assert_equal(call("add", 1, 2), 5) failed, students would only see:

✗ Failed Instructor Test
Student code failed instructor test.
I ran the code:
    add(1, 2)
The value of the result was:
    3
But I expected the result to be equal to:
    5

While runtime errors already provided much better context:

✗ Type Error
I ran your code.
A TypeError occurred:
    Unsupported operand type(s) for +: 'int' and 'str'

Line 3 of file runtime.py
    json.loads("1") + ""
    ^^^^^^^^^^^^^^^^^^^^

Solution

This PR adds source code context to assertion failures by:

  1. Creating source context utilities (pedal/utilities/source_context.py) that can locate function definitions in the student's AST and extract relevant source code lines.

  2. Enhancing RuntimeAssertionFeedback to detect function calls in assertion contexts and automatically include the function definition location and source code.

  3. Maintaining backward compatibility - assertions without function calls work exactly as before.

Result

Now when the same assertion fails, students see:

✗ Failed Instructor Test
Student code failed instructor test.
I ran the code:
    add(1, 2)
The value of the result was:
    3
But I expected the result to be equal to:
    5
In your function add on line 1:
    def add(x: int, y: int) -> int:

This provides students with immediate context about WHERE in their code the problem is occurring, making debugging significantly easier.

Implementation Details

  • Added find_function_definition() to locate functions in student AST by name
  • Added get_source_line_context() to extract source code around a given location
  • Modified RuntimeAssertionFeedback message template to include {source_context} field
  • Enhanced assertion feedback to detect function calls and provide location context
  • Handles edge cases gracefully (missing functions, multiple contexts, complex signatures)

The enhancement is minimal and surgical - it only activates when an assertion fails and a function call is detected in the context, with no performance impact on passing tests.

Fixes #118.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Always provide source code context Add source code context to assertion failures in terminal environment Sep 2, 2025
Copilot AI requested a review from acbart September 2, 2025 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Always provide source code context

2 participants