ruby_parser.rb - add void_stmt to insert_comments loop by MSP-Greg · Pull Request #1290 · lsegal/yard

MSP-Greg · 2019-11-19T15:59:13Z

Description

While running YARD with Ruby 2.7, the following comment is not added to class B:

module A
  # comment
  # ...
  # end comment
  class B

but, if a blank line is added between module A and the comment, it parses fine.

Tests passed in my fork, but 2.7/trunk is not tested against. I checked the issue locally.

I could add Actions testing on Windows (Ruby 2.4 thru trunk), which I assume would show the issue.

Completed Tasks

I have read the Contributing Guide.
The pull request is complete (implemented / written).
Git commits have been cleaned up (squash WIP / revert commits).
I wrote tests and ran bundle exec rake locally (if code is attached to PR).

coveralls · 2019-11-19T16:03:58Z

Coverage decreased (-1.8%) to 91.933% when pulling 10a2e5b on MSP-Greg:void-stmt into 4b5c875 on lsegal:master.

koic · 2019-11-25T14:23:22Z

Superfantastic!

koic · 2019-11-25T14:32:09Z

@lsegal This PR will resolve the following repositories CI failures related to RuboCop when using Ruby 2.7.0-dev.

Could you release a new version of YARD gem? I'm happy when it is released :-)

MSP-Greg · 2019-11-25T15:25:19Z

@koic

Thanks for testing. Over the weekend some changes were made to irb (see ruby/ruby@efbca15), and it broke YARD, specifically the code here:

yard/lib/yard/parser/ruby/legacy/ruby_lex.rb

Lines 1 to 7 in 4b5c875

    
           require "e2mmap" 
        
           require "irb/slex" 
        
           module YARD 
        
             module Parser::Ruby::Legacy 
        
               # Legacy lexical tokenizer module. 
        
               module RubyToken

A simple fix is to add gem 'irb', '1.1.0' somewhere, but I think a more permanent fix is needed...

aycabta · 2019-11-26T12:17:57Z

The ruby-token.rb was also removed at ruby/ruby@45bb6f2 but the class, RubyToken was copied and pasted to lib/yard/parser/ruby/legacy/ruby_lex.rb of YARD.

RubyLex in lib/irb/ruby-lex.rb was big-bang changed at ruby/ruby@94b740b#diff-80fbffd8d3dfc72fb539e0c03a8f0581 and also copied and pasted to the same file (lib/yard/parser/ruby/legacy/ruby_lex.rb).

I think SLex should be also copied from lib/irb/slex.rb and be pasted to the same file (lib/yard/parser/ruby/legacy/ruby_lex.rb).

lsegal · 2019-11-26T20:28:19Z

Another option is to remove legacy given that YARD has officially dropped support for the 1.8 parser a while ago. If Ruby is going in the same direction, I don't see a reason to keep such old outdated code.

MSP-Greg · 2019-11-28T01:23:33Z

@lsegal

Another option is to remove legacy

I briefly looked at this. The following overload statement parses with the legacy parser, but a Ripper based parser may not allow it, since the parameters aren't valid Ruby code:

yard/lib/yard/handlers/ruby/decorator_handler_methods.rb

Lines 3 to 4 in ebd6269

    
           module YARD::Handlers::Ruby::DecoratorHandlerMethods 
        
             # @overload process_decorator(*nodes, opts = {}, &block)

Thoughts?

This PR solves the following "The manual directory is out of sync" error. ```console % ruby -v ruby 2.7.0dev (2019-12-03T05:51:14Z master e42d9d8df8) [x86_64-darwin17] % CI=true bundle exec rake (snip) The manual directory is out of sync. Run `rake generate_cops_documentation` and commit the results. ``` https://circleci.com/gh/rubocop-hq/rubocop-rails/2180 This issue will be resolved with the release of YARD 0.9.21 or higher, which includes the following changes: lsegal/yard#1290

This PR solves the following "The manual directory is out of sync" error when using Ruby 2.7.0-dev. ```console % ruby -v ruby 2.7.0dev (2019-12-03T05:51:14Z master e42d9d8df8) [x86_64-darwin17] % CI=true bundle exec rake (snip) The manual directory is out of sync. Run `rake generate_cops_documentation` and commit the results. ``` https://circleci.com/gh/rubocop-hq/rubocop-performance/869 This issue will be resolved with the release of YARD 0.9.21 or higher, which includes the following changes: lsegal/yard#1290

This PR solves the following "The manual directory is out of sync" error when using Ruby 2.7.0-dev. ```console % ruby -v ruby 2.7.0dev (2019-12-03T05:51:14Z master e42d9d8df8) [x86_64-darwin17] % CI=true bundle exec rake (snip) The manual directory is out of sync. Run `rake generate_cops_documentation` and commit the results. ``` https://circleci.com/gh/rubocop-hq/rubocop-minitest/968 This issue will be resolved with the release of YARD 0.9.21 or higher, which includes the following changes: lsegal/yard#1290

This PR solves a broken document as follows: ```console % ruby -v ruby 2.7.0dev (2019-12-08T15:13:07Z master 0e71fbc18e) [x86_64-darwin17] % bundle exec rake generate_cops_documentation Files: 473 Modules: 83 ( 76 undocumented) Classes: 437 ( 411 undocumented) Constants: 628 ( 620 undocumented) Attributes: 31 ( 0 undocumented) Methods: 1080 ( 961 undocumented) 8.46% documented * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_bundler.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_gemspec.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_layout.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_lint.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_metrics.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_migration.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_naming.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_security.md * generated /Users/koic/src/github.com/rubocop-hq/rubocop/manual/cops_style.md ``` The document will be empty as follows: ```diff % git diff diff --git a/manual/cops_bundler.md b/manual/cops_bundler.md index b369413..3dc102845 100644 --- a/manual/cops_bundler.md +++ b/manual/cops_bundler.md @@ -6,32 +6,7 @@ Enabled by default | Safe | Supports autocorrection | VersionAdded | VersionChan --- | --- | --- | --- | --- Enabled | Yes | No | 0.46 | - -A Gem's requirements should be listed only once in a Gemfile. - -### Examples - -```ruby -# bad -gem 'rubocop' -gem 'rubocop' - -# bad -group :development do - gem 'rubocop' -end - -group :test do - gem 'rubocop' -end - -# good -group :development, :test do - gem 'rubocop' -end - -# good -gem 'rubocop', groups: [:development, :test] -``` +No documentation (snip) ``` This issue will be resolved with the release of YARD 0.9.21 or higher, which includes the following changes: lsegal/yard#1290

z64 · 2019-12-10T18:03:15Z

@lsegal @MSP-Greg Sorry to bump a PR - could we get a tracking issue for the irb/slex etc. issue above for visibility?

Ran into it myself and had quite a difficult time debugging to find this conversation among other irb/slex issues over the ages / trying to deduce if my environment was just broken. 😺

This commit solves the following "The manual directory is out of sync" error. ```console % ruby -v ruby 2.7.0dev (2019-12-11T07:52:06Z master 3098798044) [x86_64-darwin17] % CI=true bundle exec rake (snip) The manual directory is out of sync. Run `rake generate_cops_documentation` and commit the results. ``` https://circleci.com/gh/rubocop-hq/rubocop-rails/2180 This issue will be resolved with the release of YARD 0.9.21 or higher, which includes the following changes: lsegal/yard#1290

Follow up of lsegal#1290 (comment). This is a workaround path for Ruby 2.7.0. `irb/slex` internal API has been removed in the following commits. ruby/ruby@45bb6f2 This PR copies the above file to lib/yard/parser/ruby/legacy/irb/slex.rb. Hopefully this workaround patch will be dropped along with legacy Ruby code in the future. And E2MM (e2mmap gem) is no longer bundled gems. https://www.ruby-lang.org/en/news/2019/12/25/ruby-2-7-0-released/ ```console % cd path/to/repo/yard % ruby -v ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-darwin17] % bundle exec rake (snip) LoadError: cannot load such file -- e2mmap # ./lib/yard/parser/ruby/legacy/ruby_lex.rb:1:in `require' # ./lib/yard/parser/ruby/legacy/ruby_lex.rb:1:in `<top (required)>' ``` This dependency on `e2mmap` could be removed when `irb/slex` is removed in the future.

koic · 2019-12-26T11:24:48Z

I opened a workaround patch #1296 for using Ruby 2.7.0.

Follow up of lsegal#1290 (comment). This is a workaround path for Ruby 2.7.0. `irb/slex` internal API has been removed in the following commits. ruby/ruby@45bb6f2 This PR copies the above file to lib/yard/parser/ruby/legacy/irb/slex.rb. Hopefully this workaround patch will be dropped along with legacy Ruby code in the future. And E2MM (e2mmap gem) is no longer bundled gems. https://www.ruby-lang.org/en/news/2019/12/25/ruby-2-7-0-released/ ```console % cd path/to/repo/yard % ruby -v ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-darwin17] % bundle exec rake (snip) LoadError: cannot load such file -- e2mmap # ./lib/yard/parser/ruby/legacy/ruby_lex.rb:1:in `require' # ./lib/yard/parser/ruby/legacy/ruby_lex.rb:1:in `<top (required)>' ``` This PR adds `e2mmap` gem to the gemspec. And this dependency on `e2mmap` could be removed when `irb/slex` is removed in the future.

References: #1196, #1220, #1242, #1244, #1245, #1258, #1260, #1262, #1263, #1269, #1274, #1276, #1284, #1288, #1290, #1294, #1297

* CI against Ruby 2.7 * Ruby 2.7.0 Released https://www.ruby-lang.org/en/news/2019/12/25/ruby-2-7-0-released/ * e2mmap gem is no longer bundled with Ruby 2.7 ``` /home/travis/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- e2mmap (LoadError) ``` https://www.ruby-lang.org/en/news/2019/12/25/ruby-2-7-0-released/ ``` The following libraries are no longer bundled gems. Install corresponding gems to use these features. * CMath (cmath gem) * Scanf (scanf gem) * Shell (shell gem) * Synchronizer (sync gem) * ThreadsWait (thwait gem) * E2MM (e2mmap gem) ``` * `lib/irb/slex.rb` has been removed from Ruby 2.7 ``` /home/travis/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- irb/slex (LoadError) ``` Refer ruby/ruby@efbca15 lsegal/yard#1290 (comment) * Allow lower irb versions * Add benchmark gem explicitly

This PR solves the following "The manual directory is out of sync" error when using Ruby 2.7.0-dev. ```console % ruby -v ruby 2.7.0dev (2019-12-03T05:51:14Z master e42d9d8df8) [x86_64-darwin17] % CI=true bundle exec rake (snip) The manual directory is out of sync. Run `rake generate_cops_documentation` and commit the results. ``` https://circleci.com/gh/rubocop-hq/rubocop-performance/869 This issue will be resolved with the release of YARD 0.9.21 or higher, which includes the following changes: lsegal/yard#1290

ruby_parser.rb - add void_stmt to insert_comments loop

10a2e5b

This was referenced Nov 20, 2019

Add GitHub Actions Windows CI - Ruby 2.4 thru master/trunk. #1291

Merged

CI against Ruby 2.7.0-dev #1285

Closed

lsegal approved these changes Nov 26, 2019

View reviewed changes

lsegal merged commit 27874ee into lsegal:master Nov 26, 2019

MSP-Greg deleted the void-stmt branch November 26, 2019 20:56

koic mentioned this pull request Dec 6, 2019

Workaround for broke CI with YARD 0.9.20 rubocop/rubocop-rails#169

Merged

8 tasks

This was referenced Dec 6, 2019

Workaround for broke CI with YARD 0.9.20 when using Ruby 2.7.0-dev rubocop/rubocop-performance#84

Merged

Workaround for broke CI with YARD 0.9.20 when using Ruby 2.7.0-dev rubocop/rubocop-minitest#41

Merged

koic mentioned this pull request Dec 9, 2019

Workaround for broke docs with YARD 0.9.20 when using Ruby 2.7.0-dev rubocop/rubocop#7558

Merged

8 tasks

koic mentioned this pull request Dec 26, 2019

Support Ruby 2.7.0 #1296

Merged

4 tasks

lsegal added a commit that referenced this pull request Dec 31, 2019

Tag release v0.9.21

f13462d

References: #1196, #1220, #1242, #1244, #1245, #1258, #1260, #1262, #1263, #1269, #1274, #1276, #1284, #1288, #1290, #1294, #1297

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ruby_parser.rb - add void_stmt to insert_comments loop#1290

ruby_parser.rb - add void_stmt to insert_comments loop#1290
lsegal merged 1 commit intolsegal:masterfrom
MSP-Greg:void-stmt

MSP-Greg commented Nov 19, 2019

Uh oh!

coveralls commented Nov 19, 2019 •

edited

Loading

Uh oh!

koic commented Nov 25, 2019

Uh oh!

koic commented Nov 25, 2019 •

edited

Loading

Uh oh!

MSP-Greg commented Nov 25, 2019

Uh oh!

aycabta commented Nov 26, 2019

Uh oh!

lsegal commented Nov 26, 2019 •

edited

Loading

Uh oh!

MSP-Greg commented Nov 28, 2019

Uh oh!

z64 commented Dec 10, 2019

Uh oh!

koic commented Dec 26, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

Uh oh!

Conversation

MSP-Greg commented Nov 19, 2019

Description

Completed Tasks

Uh oh!

coveralls commented Nov 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koic commented Nov 25, 2019

Uh oh!

koic commented Nov 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MSP-Greg commented Nov 25, 2019

Uh oh!

aycabta commented Nov 26, 2019

Uh oh!

lsegal commented Nov 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MSP-Greg commented Nov 28, 2019

Uh oh!

z64 commented Dec 10, 2019

Uh oh!

koic commented Dec 26, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

coveralls commented Nov 19, 2019 •

edited

Loading

koic commented Nov 25, 2019 •

edited

Loading

lsegal commented Nov 26, 2019 •

edited

Loading