[Styling] stylify using ruff
#27144
[Styling] stylify using ruff
#27144
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
ArthurZucker
force-pushed
the
remove-black
branch
from
5335f89 to
0d678ad
Compare
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/deebert/src/modeling_highway_bert.py
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
| "rjieba", | ||
| "rouge-score!=0.0.7,!=0.0.8,!=0.1,!=0.1.1", | ||
| "ruff>=0.0.241,<=0.0.259", | ||
| "ruff>=0.1.5,<=0.2", |
There was a problem hiding this comment.
Could you elaborate a bit on this choice?
There was a problem hiding this comment.
Latest version for better performance and unified api I'd say. While <=2 to have enough rolling in case they change things
There was a problem hiding this comment.
Oh, I misread the numbers. Got it!
utils/check_copies.py
Outdated
| async def run_ruff(code): | ||
| command = ["ruff", "format", "-", "--config", "pyproject.toml", "--silent"] | ||
| process = await asyncio.create_subprocess_exec( | ||
| *command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE | ||
| ) | ||
| stdout, _ = await process.communicate(input=code.encode()) | ||
| return stdout.decode() |
There was a problem hiding this comment.
would love to hear the motivation to make is async 🙏
There was a problem hiding this comment.
Followed what was done in huggingface hub, nor real reasons apart trying to make it faster. Though it won't be run in parallel. Did not benchmark both
There was a problem hiding this comment.
Well, as here we don't really use the concurrency (which is the advantage of using coroutine), it looks somehow overwhelming to use async stuff here.
But my real concern is: here we spawn a new process whenever there is a # Copied from statement, which is an expensive op. Previously, blackify uses black.format_str which is inside the same python process, despite black itself is slow.
I checked the run (check_copies) with this PR and on main: 44 sec. on this PR and 32 sec. on main .
See run with this PR and run with main.
I am afraid, at least for check_copies.py, we lose the advantage of using ruff format due to the many new processes being created each time.
There was a problem hiding this comment.
Well I can do not async, no worries. Black is just too complicated, and with ruff you don't need to use doc-builder on top of it.
Now differences can also come from modified files as well but let's remove black because if you take the make style time into account it's already worth it
ydshieh
left a comment
ydshieh
left a comment
There was a problem hiding this comment.
Thanks for this awesome work! Having a few question, but sure LGTM!
Remove asynch
* try to stylify using ruff * might need to remove these changes? * use ruf format andruff check * use isinstance instead of type comparision * use # fmt: skip * use # fmt: skip * nits * soem styling changes * update ci job * nits isinstance * more files update * nits * more nits * small nits * check and format * revert wrong changes * actually use formatter instead of checker * nits * well docbuilder is overwriting this commit * revert notebook changes * try to nuke docbuilder * style * fix feature exrtaction test * remve `indent-width = 4` * fixup * more nits * update the ruff version that we use * style * nuke docbuilder styling * leve the print for detected changes * nits * Remove file I/O Co-authored-by: charliermarsh <[email protected]> * style * nits * revert notebook changes * Add # fmt skip when possible * Add # fmt skip when possible * Fix * More ` # fmt: skip` usage * More ` # fmt: skip` usage * More ` # fmt: skip` usage * NIts * more fixes * fix tapas * Another way to skip * Recommended way * Fix two more fiels * Remove asynch Remove asynch --------- Co-authored-by: charliermarsh <[email protected]>
What does this PR do?
Removes our dependency on
blackin favor of rust:make styleI'll try to get numbers but that one is pretty obvious