How can I best use AI to co-develop my plugin with JUCE?

Development
1

Hi,

I’m an amateur developer and I’m currently working in my free time on developing a VST/AU plugin using JUCE. It’s a MIDI editor for the Oberheim Matrix-1000 synthesizer. Since my C++/JUCE knowledge is limited, I’m relying on AI assistance to develop my plugin more quickly.

I’ve subscribed to a GitHub Copilot plan and I’ve tried almost all the available models (GPT-4.1, GPT-5, Claude Sonnet 4.5, Claude Haiku 4.5, Grok Code Fast 1), with a clear preference for the Claude Haiku 4.5 model.

After a few weeks of rather interesting testing, during which I was able to establish a fairly solid set of specifications, write my main use cases, define a first version of my software architecture, test some JUCE concepts and define my first custom software components with Xcode, etc., I feel like I’m not using the full potential of Copilot and the tools at my disposal. Therefore, I would like to get some feedback and suggestions for improving my workflow.

I’m developing on macOS Tahoe / Xcode 26. I’m currently using the Copilot extension for Xcode, which already seems much more convenient than formulating my requests in the Copilot chat on github.com. For the moment, I ask my questions in the GitHub extension chat in the sidebar window, then I copy and paste the necessary code into Xcode to test the solutions proposed by the AI.

I was wondering if it’s possible to speed things up by letting Copilot directly modify my source files, instead of doing all this copying and pasting? Similarly, it would be very convenient to let it handle correcting compilation errors, rather than having to show them to it via copy and paste…

Furthermore, the models I’ve tested so far tend to occasionally suggest JUCE code that doesn’t compile because the code is either deprecated, incorrect, or not compliant with the online JUCE documentation. Is it possible to fix this? Is it possible to teach Copilot to stop systematically making the same mistakes, for example by having it read or memorize the JUCE online documentation? (My question is probably naive…)

I recently heard about cursor.sh. It seems to better match my expectations in terms of how I want to use AI assistance. Unfortunately, this solution doesn’t seem to be (yet) compatible with Xcode at the moment…

That said, I also have access to a PC running Windows 11 (actually, my old Intel MacBook Pro that I repurposed as a PC), on which I installed and tested Visual Studio 2022. If there are more relevant solutions for Windows/VS, I’m open to those as well!

Thank you in advance for your feedback. :slight_smile:

All the best from France.

Guillaume

2

Hi, I’m a hobby programmer by myself. I guess the best advice I can give you is to understand the code that is given you from LLMs. If you don’t understand, what the code does or how the code works, you can not be sure, that the code is trash. Copy code from chatGPT and paste it to your IDE is okay, if you understand, what it does. If you want to test agentic coding I would recommend to have a look at Gemini CLI. You can simply install it in your terminal and navigate to the root of your project and start it. You have to auth with your google account and should have a free contingent every day. It runs in your terminal and can read all Project files. I would start with “analyse this project”. Now it will analyse the project and read a lot of files. You can ask for weaknesses or for better structure and so on.

Now, if you want to implent a feature, ask for a plan to do this. It will elaborate a plan how to realize this. If this feature is small, you have a good chance that it can implement it. But warning: Very often you will find you in a situation where it simply messed up your files and tries to repair it. Start with a clean git folder so you don’t lose any code!

Before you commit, you need to do a complete code review! I know, I repeate myself, but: You need to know, whats going on. Sometimes it deletes stuff and everthing builds just fine. If you don’t look on your “git diff” you never know. Sometimes it adds stuff, which is nonsense, but it builds fine and works fine. Until…

3

Hi,

Thanks for your message and your very sensible advice, it’s much appreciated. :slight_smile:

It’s true that I didn’t really specify, but my knowledge of C++, its mechanisms, and how JUCE works is advanced enough for me to understand the source code generated by the AI ​​and appreciate its relevance.

I’m also able to critically assess the quality of the generated code or the proposed solution, and to ask the AI ​​to rewrite certain parts or even start over if the result isn’t satisfactory.

For me, the problem of writing the source code myself is a bit like the problem I have with English: I understand and read it fairly well, but I have more trouble speaking and writing it, probably due to a lack of practice.

I should mention that I’m currently writing this message in French, and then using Google Translate to translate it into English and post it on this forum. :slight_smile:

In the meantime, I discovered the Agent mode of the GitHub Copilot application for Xcode. I was able to interact more easily with the AI ​​by letting it directly manipulate my source files in Xcode. I was able to successfully test some JUCE concepts simply by prompting Copilot and letting it do the rest, including fixing compilation errors.

This Agent mode is really convenient for me because I can more easily focus on describing what I want to achieve, rather than on writing the code itself, the correct syntax to use, the intricacies of the C++ language, etc. I think I have good writing skills and am precise in my technical descriptions, and this seems to be working well with the small test projects I’ve created with JUCE, Projucer, and Xcode.

To name a few, I’ve so far been able to create small test plugins that perform the following tasks:

  • Opening, closing, and selecting MIDI ports in my plugin, to bypass the problem of DAWs filtering SysEx messages at MIDI track input (as is the case with Ableton Live, which I use)
  • Receiving MIDI bytes from the Matrix-1000, then generating and displaying valid SysEx messages
  • Sending SysEx messages to the Matrix-1000 from a GUI equipped with a few basic sliders and using an APVTS object, and controlling a few parameters in the synthesizer
  • Creating a custom slider by inheriting from the juce::Button class
  • Creating custom buttons using my own LookAndFeel class
  • Creating a resizable GUI (Zoom combo box with presets + resizer) with automatic widget scaling
  • Implementing UI logic using an object APVTS

Using the GitHub Copilot application for Xcode isn’t entirely satisfactory for me at the moment. Because of how Projucer/Xcode works together, the source files end up external to the Xcode project. As a result, GitHub Copilot constantly asks for permission to access or modify specific source files whenever I request changes, and I end up constantly clicking the “Allow” button to grant the necessary permissions.

After explaining this problem to the AI, it suggested I create an Xcode project using CMake instead of Projucer. I tried this successfully, with its help and using the templates provided with JUCE, but unfortunately, the result is the same: the source files remain external to the Xcode project, and therefore it doesn’t solve my problem…

If anyone has a solution to this, I’d be very grateful!

Guillaume

4

What do you mean by that?

5

Hi Ben,

Because when you create the Xcode project for your JUCE plugin via the Projucer application, this is typically what you get :

Xcode
Xcode1084×986 150 KB

6

That’s completely normal & expected. The JUCE sources aren’t being copied into your build directory, they remain where they were on the file system and are referenced from there. This is how 99.99% of C++ projects will work, JUCE or not.

7

I know that, it’s understood. The problem I’m pointing out is that JUCE’s unique way of managing source files prevents the GitHub Copilot application for Xcode from functioning at full capacity when using Agent mode, because it can’t spontaneously take control of these sources and requires granting permissions repeatedly, making the workflow quite inefficient.

So I was asking here in this forum if anyone knew of a workaround or alternative method that would allow for full use of Agent mode.

8

I use Claude code here and it works pretty fine. Very happy with it.

9

It’s not unique to JUCE. I don’t know of any C++ build system that would copy all sources into the build tree. Such a system would be very weird to work with, because then you’d need to copy the changed sources back to the “real” copies to commit the changes.

I don’t know how Claude works, but if it’s based on directory permissions, maybe just give it permissions to the repository root directory instead of only the build directory.

10

Why would you need to modify the JUCE source? Your ai assistant is probably wrong in this case and wants to modify the juce source for some likely trivial reason.

11

AI coding assistants’ c++ coding abilities are nowhere near their usefulness in web technologies. You can’t really vibe-code in c++ yet, without understanding what the language does or why it fails. It doesn’t mean you can’t get successful results in some cases, but when the AI fails it fails badly. You can end up in an endless loop asking it to fix some apparently minor issue, while it is breaking even already working code. I’ve been working with AI assistants for about a year now and without understanding the code it is a waste of time, tokens and money.
On the positive side, since even the premium models are making dumb mistakes with C++, you can almost as effectively use cheap or even free models for general assistance. Gpt-5 mini is the one I’m currently using, and so far it hasn’t been worse one bit than Claude 4.5, maybe a bit slower.

2 Likes
12

to give everyone an honest answer. Me personally i use Codex and Claude. Sometimes Gemini pro is great for analyzing certain things i miss but if you really want proper coding, try codex. Seems to understand the relations between classes and where it could fall. It doesn’t get it 100% right and i usually have to do some manual patching but so far id bet on codex for better c++ coding. Claude comes in close second but often misses the intent. It codes well but sometimes pertaining to the project it may give the wrong definitions.

13

I’ll take a look, thank you !

14

I’ll check if it’s possible with Claude, thanks for the suggestion. Ideally, I would have liked to continue with my GitHub Copilot subscription, but if it’s worth upgrading to a Claude Pro subscription, I will.

15

Sorry, I must have expressed myself poorly. I wasn’t talking about the source files of the JUCE framework itself; I meant the source files of my JUCE project, created with Projucer and then managed in Xcode.

16

Thanks for sharing your experience. In my case it’s different because I understand the C++ code generated by the AI, and I have the ability to assess its relevance to my project.

17

I’m very interested in the solution you suggested with OpenAI Codex. The slightly annoying thing is that integration with Xcode doesn’t seem to be planned at the moment… Do you know if there’s a workaround?

18

It is in GitHub Copilot where you get unlimited free “GPT-5 mini” model usage, and it works very nicely with agentic workflows and plan workflows too. And so far it hasn’t generated me one bit of code that didn’t compile for the first time, while with Claude Sonnet 4.5 I kept getting compiler errors after a task. Ok, I’m lying, once I had a linker error, but that’s it.

19

Thanks for the info about GitHub Copilot and GPT-5 mini, I didn’t know that. I’ll do some testing. Thanks!