Testing adapters - My approach

[olimorris]

Wanted to share with the community how I'm testing adapters.

For my RSpec adapter, one of the challenges I've had as contributors add new test types and new treesitter queries is that functionality can break quite easily. Not a slight on Neotest, simply how the adapter had been structured from the start. To get around that, I had some example specs (which are "tests" in RSpec) that I would manually run alongside some basic Plenary.nvim tests for checking Treesitter query output.

My approach now is to use those specs and run them against some specified input commands and verify against specified outputs. From the example command file, you can see it tells the test to go to line 2 before calling the neotest api. This is actioned via a Makefile and Plenary.nvim. The output is generated from a crude consumer I've created.

I need to tidy it up some more but it works well...except for in GitHub Actions. Still need to figure out how to get that to work...

Anyway thought I'd share with community in case someone else finds it useful.

[rcarriga]

Thanks for sharing! I quite like this approach actually. I ironically enough don't actually have tests for my adapters, mainly because I'm using them constantly so I'm not too worried about breakages, but it's definitely something I'd to change in case I don't use them as much or they get more complex. Handling the inputs/outputs of the adapter for more unit-style tests did seem like it'd be a pain, I much prefer this approach of seeing the end results from neotest and it means that you get proper integration tests as well

except for in GitHub Actions. Still need to figure out how to get that to work...

I'm curious about the issues and if you've managed to crack them?

[olimorris]

Yeah no luck unfortunately. I can recreate the issue locally, with act but I keep hitting this error:

Fail	||	Testing blockless its (file: /home/runner/work/neotest-rspec/neotest-rspec/spec/blockless_it/blockless_it_spec_05)	
            ./tests/template.lua:54: Expected objects to be the same.
            Passed in:
            (table: 0x7f1b02ce04e8) { }
            Expected:
            (table: 0x7f1b031a19c8) {
             *[1] = './SPEC/BLOCKLESS_IT/BLOCKLESS_IT_SPEC.RB@@SKIPPED@@this is a pending test' }
            
            stack traceback:
            	./tests/template.lua:54: in function <./tests/template.lua:41>

I wonder if maybe there is some Treesitter setup error in my ci file which means Neotest can't generate any output. I'll let you know what I come up with.