Support Query / Feature Request: AI Assistant Workflow for Databricks Remote Files in VS Code

[SoulEvill]

I'm trying to integrate the use of AI coding assistants (like GitHub Copilot, etc.) with my Databricks notebooks and files accessed via the extension. My current workflow for doing this feels very manual and inefficient, and I wanted to reach out to understand if there is a recommended best practice or a more integrated way to achieve this.

Here is the issue I'm encountering:

• When I access a file from Databricks through the extension, it appears in my VS Code workspace.
• However, I cannot seem to directly use my AI assistant on this 'remote' representation of the file. The AI tools don't interact with it as they would with a standard local file.
• Instead, I find I have to manually save/download the file locally.
• I then use the AI assistant on this local copy.
• Finally, I have to manually sync/upload the modified local file back to Databricks before I can run the code.

This manual download-edit-upload cycle is cumbersome, time-consuming, and significantly disrupts the flow of development, especially when using AI assistants which are designed for rapid iteration.

I wanted to ask:

Is there currently a better or recommended way to use AI coding assistants seamlessly with remote Databricks files accessed via the VS Code extension?

If not, is this something that the development team is aware of or considering for a future release? A feature that allows direct AI interaction with the remote file representation, or automatic handling of local copies and syncing in the background for AI tools, would be incredibly valuable.

With AI coding assistants becoming increasingly mainstream, I believe improving this workflow would significantly enhance the productivity and user experience for those working with Databricks notebooks and files through VS Code.

Any guidance on the current capabilities or information regarding potential future improvements in this area would be greatly appreciated.

Best,
Wendao

[gbrueckl]

in general, AI assistants should just work fine, at least on the level of a single file - e.g. use GitHub CoPilot to do some changes for you
working with multiple files (adding them to your context) I think does not work with the virtual file system the extension uses. Not sure there is anything we can do about this at the moment - this would need to be fixed by the AI assistants extensions

alternatively you can use the official Databricks Extensions and use the sync feature there which will sync a local folder automatically with a folder on Databricks. So you could work on your local folder and the extension would take care of keeping your files in sync