Add advanced language features for Csound via custom Tree-sitter–based LSP

[PasqualeMainolfi]

Csound vscode

This PR introduces a new Csound extension for Visual Studio Code built on top of a custom Csound Language Server (LSP) and a Csound Tree-sitter grammar specifically designed for Csound.

The extension aims to provide a modern, semantically aware editing experience for Csound users, fully compatible with Csound7.

Key Features

LSP

• Advanced diagnostics
• Detection of unused variables
• Detection of undefined variables
• Opcode error reporting
• Advanced scoping logic, aware of instruments, UDOs, score blocks, and global contexts
• Precise error localization based on the Tree-sitter syntax tree

Editor

• Format on type
• CodeLens for:
  • Running a script
  • Saving output to an audio file
  • Opening the Csound manual
• Code Actions for quick fixes and navigation

Completion & Hover

• Opcode completion
• Option/flag completion
• Hover support for:
  • Built-in opcodes
  • User-defined opcodes (UDOs)
  • User-defined types

Commands exposed by the Language Server

The server exposes the following commands:

• csound-lsp.run_file — run the current Csound script
• csound-lsp.to_audio_file — render and save output to an audio file
• csound-lsp.open_manual — open the Csound reference manual

Offline Manual (Web Preview)

• The Csound HTML reference manual is bundled with the language server
• A lightweight local HTTP server is started by the server
• The manual is displayed inside a VS Code Webview
• Fully functional offline browsing, including links, searching and assets

Language Injections

• Python and HTML language injections are supported inside Csound files
• Enables proper highlighting and tooling for embedded code

Technical Notes

• Syntax parsing is handled via Tree-sitter
• Semantic analysis is built directly on the syntax tree
• The LSP architecture is modular and designed for future extensions

Future Work

Planned next steps include:

• Support for Csound plugins (not yet tested)
• Deeper support for Cabbage blocks
• Additional semantic checks, completions, and editor actions

This PR lays the groundwork for a complete and modern Csound development environment in VS Code, aligned with current language-server–based tooling.
Feedback and testing are highly appreciated.

[kunstmusik]

Just wanted to note that I saw this PR but it came in while I was traveling for holidays. I'm on another trip now so will look at this when I return home in a couple days. Apologies @PasqualeMainolfi for the delay in review!

[kunstmusik]

@PasqualeMainolfi One thing I did want to ask is if you tested this as a Web Extension (i.e., when installed and running in github.dev). I didn't see code to support that but may have just missed it.

[PasqualeMainolfi]

No worries.
In the meantime, I’m working on fixing a few bugs and improving the parsing of .udo files when they are used as imports. I also need to add support for the additional syntax introduced for raw strings (R{}R).
I haven’t tested the extension as a Web Extension on github.dev.
This extension relies on a language server process and a Tree-sitter grammar, which currently require a Node / native (or WASM) environment and are not compatible with VS Code for the Web out of the box.
For this reason, the extension is intended to run in the desktop VS Code environment only.
If web support becomes a goal in the future, it would likely require a web-based LSP and additional work for Tree-sitter compatibility.

[rorywalsh]

Thank you @PasqualeMainolfi, it's going to be very nice to have this feature. I am going to leave the technical review to @kunstmusik as he is more familiar with these things, but I am concerned that this PR overwrites the project's readme file. This removes all the historical context for the existing project as well as removing valuable information about live-coding and using the UDP server. Would you mind reverting the README to the original master branch version. We can then update accordingly after any merge.

[PasqualeMainolfi]

I’ve restored the original README to preserve the historical context and existing information. Additionally, I’ve added release notes to the PR, without referencing any version numbers.

[rorywalsh]

Thanks @PasqualeMainolfi - I know Steven is extremely busy right now, but hopefully we can get this merged and into the extension in time for the first release of Csound 7 👍

[PasqualeMainolfi]

I was wondering if there is any approximate timeline available for the release of Csound7. Having a rough idea of the release date would help me plan the remaining work and properly schedule the effort needed to deliver a stable version of the LSP.

I am already quite close, and only a few final adjustments are still missing.

Thanks in advance for any information you can share.

[rorywalsh]

Good question. I think most of the tasks have been completed at this stage. So it could be in the next few weeks but I don't think any hard deadline has been set.

[kunstmusik]

This should be in English to match the rest of the plugin. (I haven't done I18N for vscode plugins before; we could investigate that in a separate PR)

[kunstmusik]

@PasqualeMainolfi I ran this in the extension development host. It requested installing the LSP and then reported success in installing the LSP. I checked before and after and CSD and ORC code looks no difference with little syntax highlighting. I did not get any manual entries, autocomplete, etc. I am on MacOS. Unsure if something's gone wrong in installation or what but it's not operating as (I assume) expected.

[PasqualeMainolfi]

It seems very strange. Check the lsp-bin folder and verify the downloaded executable, it’s probably a permissions issue. Also, check /var/folders/hp/.../T/ (temporary files) to see if the csound-lsp-temp-folder exists. This is where it downloads the external resources it uses. I’ve run several tests and everything seems fine. Additionally, I made a modification to the source code to avoid issues with the local manual server, especially on Windows (though I don’t think this is the cause of the problem).

One more thing: there is a check in the extension that disables the LSP in web environments:

  if (vscode.env.uiKind === vscode.UIKind.Web) {
    console.warn("LSP disabled in web");
    return;
  }

[kunstmusik]

I updated to latest and am getting further. Parser seems to have problems with floating point numbers that start with decimal:

I think because of the decimal issue it reports unclosed blocks:

Also multiline comments seem to throw things off:

[PasqualeMainolfi]

I think I have fixed the bugs you showed me.
To give priority to the LSP for handling multi-line comments, I had to disable the comments section in language.
Note that the LSP checks installed plugins on each startup. Could you please verify if everything is working as expected by checking the Developer Tools / Show Dev Logs for the Csound Language Server?
If anything was missed or there are additional things to implement, we can go over the details next.

[kunstmusik]

@PasqualeMainolfi I tried the latest and when I start up the extension host it asks to download a newer version of LSP. I says yet and then it errors:

[PasqualeMainolfi]

Thanks for the report!
The issue was caused by using a for...in loop instead of for...of when iterating over the files returned by fs.readdirSync().
for...in iterates over array indices (e.g. 0, 1, …), which caused the extension to try to unlink non-existent paths like lsp-bin/0.
This has now been fixed by switching to for...of, which correctly iterates over the actual file names.
The installer should work correctly now.

[kunstmusik]

One other note, I asked Opus about using tree-sitter in Javascript so that we can use it in the web extension version of this plugin. Looks like compiling to WASM would be an option, which is promising.

Looking at this PR, your concern is valid. The PR introduces a Csound Language Server built on a custom Tree-sitter grammar, which typically compiles to native binaries. This would indeed be incompatible with running as a VS Code web extension.

The Good News

Yes, Tree-sitter parsers can run in JavaScript/WebAssembly. Tree-sitter supports compiling grammars to WASM, which can then be loaded and run in JavaScript environments (including browsers and VS Code web extensions).

How to Make It Work for Web Extensions

The tree-sitter-csound grammar would need to be compiled to WASM instead of (or in addition to) native binaries. Here's what's typically required:

1. Compile the grammar to WASM:

   tree-sitter build --wasm

2. Use web-tree-sitter - the official JavaScript/WASM binding:

   npm install web-tree-sitter

3. Load the parser in JS:

   const Parser = require('web-tree-sitter');
   await Parser.init();
   const parser = new Parser();
   const Csound = await Parser.Language.load('tree-sitter-csound.wasm');
   parser.setLanguage(Csound);

Recommendations for the PR

1. Ask the PR author if the LSP server (csound-lsp) supports running in a browser/Node.js environment with the WASM-compiled parser, or if it's strictly a native binary.

2. Consider a hybrid approach:

   • Native LSP for desktop (full features, better performance)
   • WASM-based parsing for web extension (potentially reduced features)

3. Check the LSP implementation language - if it's written in a language that can compile to WASM (Rust, TypeScript, etc.), the entire LSP could potentially run in the browser.

[PasqualeMainolfi]

Yes, I can.
I think, the best approach would be to consider a hybrid version: a full-featured implementation for desktop, and a slightly limited version for the web. Certain features, like file watchers and other system-level integrations, simply cannot be ported to a web environment. Rust (and tree-sitter) is not an issue... the main challenge is deciding which parts of the functionality to bring to the web and which to leave out.

[kunstmusik]

I think we're on the same page, offer what we can in each environment. I think this will work out nicely and users will be happy. :)

[PasqualeMainolfi]

At the moment, I’m working on parsing UDOs, specifically their signatures, including argument validation and output checking.
Another aspect I think would be interesting to explore is introducing a docstring/annotation model for UDOs. The templates used in the official manual don’t seem to be a good fit for inline annotations inside scripts, so I’m not convinced they work well in this context.
Do you have any thoughts on what kind of docstring or annotation model would be more appropriate for use directly in code? Somethings like

/**
 * @opcode value_from_udo_file
 * @brief Testing opcode info
 *
 * @param value1 First input value
 * @param value2 Second input value
 *
 * @return i, i
 *
 * @example
 *   first:i, second:i = value_from_udo_file(x, y)
 */

or

/*************************************
@opcode: value_from_udo_file
@description: testing opcode info
@inputs:
    value1
    value
@outputs:
    i, i
@example:
    first:i, second:i = value_from_udo_file(x, y)
****************************************/

or

/**
value_from_udo_file
-------------------
Inputs:
  i value1
  i value2

Outputs:
  i, i

Example:
  first:i, second:i = value_from_udo_file(x, y)
*/

and

/****************************************************************************
ift BufCt1 ilen [, inum]
creates a function table of ilen seconds for recording

creates an "empty" function table (filled with zeros) of ilen seconds, using GEN02, for recording sound
written by joachim heintz

ilen - length in seconds
inum - if zero (which is also the default), the number of the function table is given by Csound. Any other positive integer will represent the function table, but the user must take care of not using a number twice
ift - table number as output
****************************************************************************/

...

[rorywalsh]

I think the first one is good as it can be used with doxygen to document .csd files. But I think opinions will be mixed on this one.

[rorywalsh]

I just had time to finally test this PR. The LSP installed without a problem. And I think all the issues Steven mentioned have been resolved. I'm still seeing a few little things, such as:

and

Also, the Cabbage JSON syntax is no longer highlighting correctly?

With the old extension it looked like this:

Any idea why this is happening? Btw, the error reporting is really great. Having this feedback, in realtime, without needing to compile seems like some kind magic :)

[PasqualeMainolfi]

@rorywalsh thanks! I’ve fixed the strange behaviors you pointed out.
Regarding the Cabbage block, for now I’ve added JSON injection, but to manage it properly (opcode documentation, widgets, etc.) I’ll need your help.
I’d like to have access to the raw documentation so it can be used as a resource that the LSP can query. I also need to better understand what is actually possible to implement and what isn’t.
Ideally, I’d like to reach a more stable version of the LSP in time for the Cs7 release.

[rorywalsh]

Great, I'm seeing no more squiggly lines 👏 Here is how the JSON looks now:

It's definitely better, but still not very accessible in terms of highlighting. keys and values often have the same colouring. Btw, those "//" comments are decorated by the cabbage extension, so they can be ignored. The documentation is here:
https://rorywalsh.github.io/cabbage3docs/docs/intro
But it will move to a permanent location once the Cabbage 3 site is ready. I am happy to leave any kind of deeper Cabbage implementation until after the release of Csound 7. Proper JSON highlighting would be nice to have by then though.

[PasqualeMainolfi]

Fantastic! This way, we can guide the compilation of a Cabbage block and maybe display the documentation separately, just like it’s done for the Csound manual. This could make everything much smoother. Let me know what you would like to have readily available. Keep in mind that Cabbage opcodes might currently show an error because I don’t think they are present in the Csound opcodes (repo).

Check json now. The official tree-sitter-json query for keys uses the type @string.special.key. It should be all set now… I hope.

[kunstmusik]

@PasqualeMainolfi The documentation style is interesting. I've used javadocs, jsdoc/tsdoc, and doxygen, and their all similar. I checked and jsdoc and doxygen don't appear to work without knowing the underlying language. I also realized I had done a small script in my live-code system to generate docs from UDOs and instruments.

I think it would be nice to create a csdoc project that can generate docs from orc code. If we do that, we can specify exact tags and format and the LSP can share a standard. Most likely something like the first example would be my preference.

I'll see if I can use my csdoc.py file from csound-live-code as a starting point and convert into a standalone project that has documentation for standards and things. I think I could release it on pypi so that it could be used with uv/uvx. I'll reply here shortly on how that goes.

[rorywalsh]

Ah, I thought doxygen was one of the few tools that would work without knowing the underlying language. My bad.

[kunstmusik]

FYI: I created an initial version of csdoc here:

https://github.com/kunstmusik/csdoc

Generates documentation site like the following:

Probably a few more things to add but usable already. Only a few tags supported, LMK what you think about the format and tags.

[PasqualeMainolfi]

I think your proposal is very efficient. From my experience, less is often better.
Maybe just a few additional elements, in a style closer to Csound’s documentation: an initial section to specify @signature, @description, and @author, followed by the parameters section as you proposed, and a final @example section. Optionally, an @deprecated tag could also be useful.
The same structure could apply to typedef struct, using @signature, @description, and @params.
Clear semantics, simple line-based parsing, and zero ambiguity. It is clean, readable, and easy to serialize (e.g., JSON / LSP cache).
I also think that using a Python-based parser is a perfect fit here. We already have the grammar, which means we always have access to the AST and its nodes. In Python, parsing the structure of a Csound file with Tree-sitter is very straightforward. The nodes involved are unambiguous, well-defined, and come with plenty of references, so there are no conflicts to deal with.

[PasqualeMainolfi]

If you’d like, I can add a small .py module to your project that exposes helper functions to iterate over the syntax tree and extract the relevant information, keeping the integration with the rest of the system clean and easy to maintain.

[rorywalsh]

With regard to the JSON stuff, it’s all working well now. I’m going to start testing with Cabbage 3 next. I’ll be very happy to remove all of my ad-hoc error-reporting code in lieu of this. 🙂

[kunstmusik]

@PasqualeMainolfi Not sure I understand the syntax tree idea exactly but happy to accept PR's. For tags, not sure we need @description as the body of the comment can be that. @Signature I'm also wary of as it's something to keep in sync with @params and @return, so it might be good to just generate. @author, @deprecated, and @example all make sense to me.

[rorywalsh]

One thing I noticed @PasqualeMainolfi, when I add something like this above my Cabbage section it breaks the parsing:

<!--⚠️ Warning: Any manual formatting (indentation, spacing, or comments) may
be lost when working with the UI editor. -->

The question is whether this is legal or not. I can't include comments in the JSON, but felt the need to warn people about this. Anything outside of a matched tags should be ignored, but is this feasible, or does it complicate things? I don't want you to waste time on this if it's not straightforward.

[rorywalsh]

Another issue I noticed with some of the new things in Csound7:

In this instance vco is the type of opcode I'm trying to create an array of. Are we still managing one long list of opcode names for syntax highlighting, or is this being handled elsewhere now?

[rorywalsh]

@PasqualeMainolfi How difficult would it be to tap into the existing vscode-json-languageservice for Cabbage sections of a csd file? I have no idea. I know that in .json files I have this out of the box with vscode, but not when JSON is embedded into another language. I could try to handle this myslef in my Cabbage vscode extension, but given your level of knowledge on these matters, I thought I would throw it out there for discussion :)

[PasqualeMainolfi]

Regarding the issues you showed me, I think a relatively simple fix should resolve everything.
As for the second point, I believe json-languageservice works on files, not strings (though I could be mistaken). For that reason, we can try the following approach: through the LSP, I can extract the Cabbage block and create a virtual in-memory file that you can pass to the JSON language service. The service will then treat it as a regular JSON file. The language service doesn’t need to know anything about the .csd file, it only sees the JSON itself.
In other words, I can just pass you a list of URIs and positions. Simple and fast.

[rorywalsh]

For now the main concern is Csound :) Your appraisal of the JSON seems to match what my brief ressearch shows. In this case I might as well handle this myself from within my own extension. It probably makes more sense this way too. Let me know when you have a fi for the other issue and I will test. I've a lot of Csound7 orchestras to throw at it!

[rorywalsh]

Some more issues:

[PasqualeMainolfi]

Check now...
For the run opcode, I think it’s because it hasn’t been added to the manual yet.

[rorywalsh]

Great, looking good now. Did something change with the Cabbage section though? The custom highlighting seems broken again.

It might be the SVG path string that preceeds this 🤔 I've attached the file, I had to rename it to allow uploading.
SuperSaw.txt

[PasqualeMainolfi]

Try it now. I initially thought I could handle it with a generic block, but that wasn’t possible, so I rewrote the JSON block specifically for the Cabbage section. It’s an isolated block and shouldn’t introduce any conflicts. Please let me know if you notice any side effects, but it shouldn’t... I’ve tested it with your file.

...the different color of the curly braces appears to be a VS Code issue. In other environments, the coloring is correct.