How to use COM812 in linter but avoid conflict with formatter?

[jagerber48]

I would like to achieve the following things:

• Use the ruff linter on my code
• Use the ruff formatter on my code
• Enable rule COM812 for the linter

I like the linter, I like the formatter, and I like rule COM812. However, when I have COM812 as an allowed rule I get a warning when I run the formatter that rule COM812 may cause conflicts and that it should be disabled to avoid unexpected behavior. Is there, then, a recommended way to achieve my three goals?

I see a number of issues discussing rule COM812 but I don't see any addressing this exact question.

[charliermarsh]

I don't think it's currently possible based on our definition of "incompatible". In short, we want to avoid cases in which running the formatter introduces new lint rule violations. In the case of COM812, you could have a function like:

def args(aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh):
    pass

Which the formatter may reformat as:

def args(
    aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh
):
    pass

Which would violate COM812. After adding the trailing comma to fix COM812, the formatter would then reformat again as:

def args(
    aaaaaaaa,
    bbbbbbbbb,
    cccccccccc,
    ddddddddd,
    eeeeeeee,
    ffffff,
    gggggggggggg,
    hhhh,
):
    pass

We would need a new trailing-comma setting in the formatter, like "force".

\cc @MichaReiser

[jagerber48]

Thanks for the response! I would be interested in any sort of solution to get compatibility, something like the "force" flag you mention seems fine to me. I would be curious to know if it's decided that this is something ruff will support at some point.

I think I've already experienced the two step process you described I think for now, for my codebase, I will bear the annoyance of the incompatibility message and implement the format - lint (fix) - format routine to make the formatting stable.

[bittner]

I'm refactoring large code bases, and the case you quote above is particularly common as an outcome of reformatting.

def args(
    aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh
):
    ...

What the formatter does is indeed very natural and intuitive: When the line gets too long it first tries to put everything inside braces onto a line of its own. Shouldn't COM812 accept this as a good case, in general? Kind of, "there are several arguments on a single line", that's a signal that it's okay not to have a trailing comma.

The trailing-comma = "force" is probably just that. - Maybe "strict" vs. "forgiving" would reflect the intention even better, with "forgiving" as the default, probably.

[MichaReiser]

I understand that having COM812 and the formatter not agreeing on the formatting here is annoying.

What's the motivation for using both the formatter and COM812. Wouldn't it be sufficient to only rely on the formatter that also fixes the issue automatically rather than having the lint rule enabled too? Or do you need COM812 enabled because you dislike the "all on one line" formatting that the formatter produces?

[jagerber48]

What's the motivation for using both the formatter and COM812. Wouldn't it be sufficient to only rely on the formatter that also fixes the issue automatically rather than having the lint rule enabled too? Or do you need COM812 enabled because you dislike the "all on one line" formatting that the formatter produces?

There are a few things I want to satisfy. The motivation for COM812 from the docs is

The presence of a trailing comma can reduce diff size when parameters or elements are added or removed from function calls, function definitions, literals, etc.

This is true in cases where there is one argument per line. So in those cases (one argument per line) I do want COM812 enforced.

Another constraint is I want arguments to be wrapped in some way when a single line like

def args(aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh):
    pass

is too long.
But I would be happy with

def args(
    aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh
):
    pass

As a resolution to a line too long in the case that the arguments are too long to stand on one like with the function definition (def args( ... )) but short enough to stand on their own line. That is, I don't think this case needs a trailing comma because I don't think it shortens the diff.

So yes, I think I agree with @bittner that I don't understand why this form violates COM812. Maybe COM812 should only be enforced when there is one argument per line. That way the trailing comma won't be added by the linter after the re-formatting, thus inducing the formatter to again want to do more.

This wouldn't even require a new option I don't think. BUT if a new option is the way to go: Is it possible for LINTING rules to have optional configuration (other than inclusion or exclusion)? Or can only formatting rules be controlled by optional configuration?

[MichaReiser]

I'm currently looking into what it takes to make COM812 compatible with the formatter. It sounds like a simple change, but it's a very expensive token-based rule. I need to figure out how to test if all arguments (or list elements etc) are on their own line without having to track too much state.

I don't intend to add a configuration option that gives users the old behavior because I don't see the formatter's behavior to be in contradiction to the rule's goals.

[MichaReiser]

Interesting, the existing rule isn't able to catch all cases where I would suspect a trailing comma to be required because it only tests if there's a newline between the last token and the closing parentheses:

E.g. the following is okay

def call(a, b,
    c): pass

[jagerber48]

Interesting. As for my desired behavior independent of any existing functionality: Assuming all arguments are simple parameter names, the three types of formats I would ever want (in this preference) are

• One line, params/args appear on the same line as def foo(...)
• Three lines. One line for the function definition, one line for params and args and one line for the closing parentheses, colon, and return type annotation.
• n lines. One line for the function definition, one line for each param/arg pair, and one line for the closing parentheses, colon, and return type annotation.

I would prefer no trailing comma for the first two but not be upset by the existence of a trailing comma. The third one must have a trailing comma.

Anything other than these three options, like what was in the last post, would not be acceptable to me.

I don't have a simple way to say what rules I expect if there is nesting of complex expressions

def foo(
    param_1=3
    param_2=[
        first_element,
        bar(
            "a deeply nested input",        
        ),
    ],
    param_3="top_layer_input",                  
) -> None:

I think my preference would be for the above three rules to be applied in every scope. I have no idea about the feasibility of this. Thank you for looking into it!

[bluetech]

For me, I do want the lint to flag the following

def args(
    aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh
):
    pass

because it's missing the trailing comma, and that means adding a new param will change two lines instead of one, and avoiding this is the main purpose of COM812.

I realize it's conflicting with the formatter, and compat with the formatter is probably more important. But if a solution can be found which keeps preventing the code above, when not using the formatter, or even when are using the formatter, that would be nice.

[MichaReiser]

because it's missing the trailing comma, and that means adding a new param will change two lines instead of one, and avoiding this is the main purpose of COM812.

Can you explain how you add the parameter that it changes two lines? The formatting I'm thinking of either change one line or all parameter lines.

I would expect this to either become:

def args(
    aaaaaaaa, bbbbbbbbb, cccccccccc, ddddddddd, eeeeeeee, ffffff, gggggggggggg, hhhh, iiii
):
    pass

in which case it only changes one line or

def args(
    aaaaaaaa, 
	bbbbbbbbb, 
	cccccccccc, 
	ddddddddd, 
	eeeeeeee, 
	ffffff, 
	gggggggggggg, 
	hhhh,
):
    pass

In which case it changes all lines.

[bluetech]

I'm currently looking into what it takes to make COM812 compatible with the formatter. It sounds like a simple change, but it's a very expensive token-based rule.

Curious, in my mental model token-based rules were supposed to be faster than AST based rules, but I guess I need to update :)

BTW, while flake8-commas (on which the ruff implementation is based) is token-based, there is also https://github.com/asottile/add-trailing-comma which IIRC is AST based, maybe it's a better base for this rule.