Create Neovim plugin with Tree-sitter grammar for LUMOS

[rz1989s]

Goal

Build a Neovim plugin for LUMOS with Tree-sitter grammar for syntax highlighting and LSP integration for IDE features.

Phase: 5.2 IDE Integration
Depends On: #45 (LSP implementation)

---

Problem

Neovim users have no LUMOS support:

• ❌ No syntax highlighting
• ❌ No auto-completion
• ❌ No diagnostics
• ❌ .lumos files treated as plain text

---

Solution

Two-part implementation:

1. Tree-sitter grammar - Syntax highlighting (fast, accurate)
2. LSP configuration - IDE features (completion, diagnostics)

---

Part 1: Tree-sitter Grammar

What is Tree-sitter?

Tree-sitter provides fast, incremental parsing for syntax highlighting.

• Used by Neovim, GitHub, Atom
• Better than regex-based highlighting
• Generates parse tree from grammar

Grammar Implementation

Repository: tree-sitter-lumos (new repo)

File: grammar.js

module.exports = grammar({
  name: 'lumos',
  
  rules: {
    source_file: $ => repeat($._definition),
    
    _definition: $ => choice(
      $.struct_definition,
      $.enum_definition,
      $.comment,
    ),
    
    struct_definition: $ => seq(
      repeat($.attribute),
      'struct',
      field('name', $.identifier),
      field('body', $.struct_body),
    ),
    
    struct_body: $ => seq(
      '{',
      repeat($.field),
      '}',
    ),
    
    field: $ => seq(
      field('name', $.identifier),
      ':',
      field('type', $.type),
      ',',
    ),
    
    attribute: $ => seq(
      '#[',
      $.identifier,
      optional(seq('=', $.string_literal)),
      ']',
    ),
    
    type: $ => choice(
      $.primitive_type,
      $.solana_type,
      $.array_type,
      $.option_type,
      $.vec_type,
      $.identifier,
    ),
    
    primitive_type: $ => choice(
      'u8', 'u16', 'u32', 'u64', 'u128',
      'i8', 'i16', 'i32', 'i64', 'i128',
      'bool', 'String',
    ),
    
    solana_type: $ => choice(
      'PublicKey',
      'Signature',
    ),
    
    vec_type: $ => seq(
      'Vec',
      '<',
      field('element', $.type),
      '>',
    ),
    
    option_type: $ => seq(
      'Option',
      '<',
      field('inner', $.type),
      '>',
    ),
    
    array_type: $ => seq(
      '[',
      field('element', $.type),
      ']',
    ),
    
    identifier: $ => /[a-zA-Z_][a-zA-Z0-9_]*/,
    
    string_literal: $ => /"[^"]*"/,
    
    comment: $ => choice(
      $.line_comment,
      $.block_comment,
    ),
    
    line_comment: $ => /\/\/[^\n]*/,
    block_comment: $ => /\/\*[\s\S]*?\*\//,
  }
});

Syntax Highlighting

File: queries/highlights.scm

; Keywords
["struct" "enum" "pub"] @keyword

; Attributes
(attribute) @attribute

; Types
(primitive_type) @type.builtin
(solana_type) @type.builtin
(identifier) @type

; Fields
(field name: (identifier) @variable.member)

; Comments
(line_comment) @comment
(block_comment) @comment

; Strings
(string_literal) @string

; Punctuation
["{" "}" "[" "]" "<" ">"] @punctuation.bracket
[":" ","] @punctuation.delimiter

Build Tree-sitter Parser

npm install
npx tree-sitter generate
npx tree-sitter test

---

Part 2: Neovim Plugin

Plugin Structure

nvim-lumos/
├── ftdetect/
│   └── lumos.lua          # File type detection
├── lua/
│   └── lumos/
│       ├── init.lua       # Plugin entry point
│       └── lsp.lua        # LSP configuration
├── queries/
│   └── lumos/
│       ├── highlights.scm # Syntax highlighting
│       └── injections.scm # Code injections
└── README.md

File Type Detection

File: ftdetect/lumos.lua

vim.api.nvim_create_autocmd({"BufRead", "BufNewFile"}, {
  pattern = "*.lumos",
  callback = function()
    vim.bo.filetype = "lumos"
  end,
})

LSP Setup

File: lua/lumos/lsp.lua

local M = {}

function M.setup()
  local lspconfig = require('lspconfig')
  local configs = require('lspconfig.configs')
  
  -- Register LUMOS LSP server
  if not configs.lumos then
    configs.lumos = {
      default_config = {
        cmd = {'lumos-lsp'},
        filetypes = {'lumos'},
        root_dir = lspconfig.util.root_pattern('.git', 'Cargo.toml'),
        settings = {},
      },
    }
  end
  
  -- Setup LSP
  lspconfig.lumos.setup({
    on_attach = function(client, bufnr)
      -- Enable completion
      vim.api.nvim_buf_set_option(bufnr, 'omnifunc', 'v:lua.vim.lsp.omnifunc')
      
      -- Keybindings
      local opts = { noremap=true, silent=true, buffer=bufnr }
      vim.keymap.set('n', 'gd', vim.lsp.buf.definition, opts)
      vim.keymap.set('n', 'K', vim.lsp.buf.hover, opts)
      vim.keymap.set('n', '<leader>rn', vim.lsp.buf.rename, opts)
      vim.keymap.set('n', '<leader>ca', vim.lsp.buf.code_action, opts)
    end,
    capabilities = require('cmp_nvim_lsp').default_capabilities(),
  })
end

return M

Plugin Entry Point

File: lua/lumos/init.lua

local M = {}

function M.setup(opts)
  opts = opts or {}
  
  -- Setup LSP
  require('lumos.lsp').setup()
  
  -- Setup Tree-sitter
  require('nvim-treesitter.configs').setup({
    ensure_installed = { 'lumos' },
    highlight = { enable = true },
  })
end

return M

---

Installation

Using lazy.nvim

{
  'getlumos/nvim-lumos',
  dependencies = {
    'neovim/nvim-lspconfig',
    'nvim-treesitter/nvim-treesitter',
  },
  config = function()
    require('lumos').setup()
  end,
}

Using packer.nvim

use {
  'getlumos/nvim-lumos',
  requires = {
    'neovim/nvim-lspconfig',
    'nvim-treesitter/nvim-treesitter',
  },
  config = function()
    require('lumos').setup()
  end,
}

Manual Installation

# Clone plugin
git clone https://github.com/getlumos/nvim-lumos \
  ~/.local/share/nvim/site/pack/plugins/start/nvim-lumos

# Install Tree-sitter parser
git clone https://github.com/getlumos/tree-sitter-lumos \
  ~/.local/share/nvim/site/pack/parsers/start/tree-sitter-lumos

# Add to init.lua
require('lumos').setup()

---

Features

Syntax Highlighting (Tree-sitter)

• Keywords: struct, enum, pub
• Attributes: #[solana], #[account]
• Types: Different colors for primitives vs Solana types
• Comments: Line and block comments

LSP Features

• Auto-completion (Ctrl+N / Ctrl+P)
• Go to definition (gd)
• Hover documentation (K)
• Diagnostics (inline errors)
• Code actions (<leader>ca)
• Rename symbol (<leader>rn)

File Type Support

• Automatic detection of .lumos files
• Proper indentation
• Comment strings set correctly

---

Testing

Test Tree-sitter Grammar

cd tree-sitter-lumos
npm test

File: test/corpus/struct.txt

================
Simple struct
================

#[solana]
struct Player {
    wallet: PublicKey,
    level: u16,
}

---

(source_file
  (struct_definition
    (attribute (identifier))
    name: (identifier)
    body: (struct_body
      (field name: (identifier) type: (solana_type))
      (field name: (identifier) type: (primitive_type)))))

Manual Testing

1. Open Neovim
2. Create test.lumos
3. Verify:
   • Syntax highlighting
   • Auto-completion works
   • Errors show inline
   • gd jumps to definition
   • K shows hover info

---

Success Criteria

• Tree-sitter grammar parses all LUMOS syntax
• Syntax highlighting works in Neovim
• LSP integration provides completion
• Diagnostics show errors in real-time
• All LSP features functional (goto, hover, rename)
• Plugin installable via package managers
• Published to GitHub
• Documentation complete
• Tests passing

---

Documentation

Create New Files

• tree-sitter-lumos/README.md - Tree-sitter grammar docs
• nvim-lumos/README.md - Plugin installation and usage
• docs/editors/neovim.md - Complete Neovim setup guide

Update Files

• README.md - Add Neovim installation instructions
• CHANGELOG.md - Document plugin release
• ROADMAP.md - Mark Phase 5.2 item as complete ✅

---

Repositories

Create two new repositories:

1. getlumos/tree-sitter-lumos - Tree-sitter grammar
2. getlumos/nvim-lumos - Neovim plugin

---

Related

• Depends on: Implement Language Server Protocol (LSP) for LUMOS #45 (LSP server)
• ROADMAP.md Phase 5.2 - IDE Integration

---

Priority: Medium
Complexity: Medium
Timeline: 5-7 days
Depends On: #45

---

📌 Remember: Update ROADMAP.md after completing this issue!

[rz1989s]

✅ Issue Complete: Neovim Plugin with Tree-sitter Grammar

Successfully implemented comprehensive Neovim support for LUMOS with Tree-sitter syntax highlighting and LSP integration.

Deliverables

1. Tree-sitter Grammar Repository

Repository: https://github.com/getlumos/tree-sitter-lumos

Features:

• Complete grammar for LUMOS syntax (structs, enums, attributes, all types)
• Syntax highlighting queries for all major elements
• 6 comprehensive test cases (all passing)
• Support for unit, tuple, and struct enum variants
• Hidden type nodes for clean parse trees
• Flexible comma handling

2. Neovim Plugin Repository

Repository: https://github.com/getlumos/nvim-lumos

Features:

• File type detection for .lumos files
• Full LSP integration with lumos-lsp server
• Tree-sitter syntax highlighting
• Pre-configured keybindings:
  • gd: Go to definition
  • K: Hover documentation
  • gr: Find references
  • rn: Rename symbol
  • ca: Code actions
  • f: Format document

Installation Support:

• lazy.nvim configuration
• packer.nvim configuration
• Manual installation guide
• Comprehensive troubleshooting section

Testing

• 6/6 Tree-sitter grammar tests passing
• Parse tree validation for structs and enums
• Syntax highlighting verified
• LSP integration tested

Documentation

• Complete README for tree-sitter-lumos with grammar coverage
• Complete README for nvim-lumos with installation guides
• Troubleshooting guides for common issues
• Example configurations for popular plugin managers

Impact

• Phase 5.2 IDE Integration: 60% complete (3/5 editors)
• Success Metric ACHIEVED: LSP used by 3+ editors (VS Code, IntelliJ IDEA/Rust Rover, Neovim)
• LUMOS now has professional Neovim support matching industry standards

Related Work

• tree-sitter-lumos: Complete syntax highlighting
• nvim-lumos: Full LSP client integration
• ROADMAP.md updated with completion details

Bismillah, this is complete! Neovim users can now enjoy first-class LUMOS support. 🎉