[SOLVED] Illegal Instruction on user machines

I’m running a beta and some users are reporting a crash in a certain situation. I cannot reproduce it. It only happens on Windows machines and seems tied to a set of hosts (mostly Reaper).

All the error messages which I have seen report an “illegal instruction” or “invalid operation”. However I am pretty sure this is not about AVX: I am compiling with /mavx in Visual Studio and made sure all the users have CPUs which are actually able to execute AVX.

I sent them debug builds with static runtimes, and then the crash dissappeared.
Any idea what kind of “illegal instruction” could be meant here?

I’m happy for any hints!

Did you get a stack trace from your beta testers? This will show you where in your program the “illegal instruction” was executed. At least it will narrow it down :slight_smile:

That’s what I sent them the Debug build for. AFAIK, you need debug symbols to have a readable stack-trace. But then the Debug build didn’t crash, so no report there.

I even tried to send them a Release build with static runtime and debug symbols, but that one crashed in the PluginFactory for some users (illegal instruction) and didn’t crash at all like the Debug build for at least one user.

Are you really sure that all your users machines Support AVX? Did you try disabling AVX?

Have you also checked other cpu extensions like FMA?

I had them report me their CPU-models and they do support AVX. Also on the machines which actually lacked AVX, the plugin would not register at all, not crash in a certain situation.

No, I will look into that!

I am not sure if compilers will automatically use FMA instructions, as they don’t have the same precision as their equivalent operations. I think you have to manually set the fast math flag to actually produce FMA instructions.

It’s normally obvious if you have illegal instructions (as in, unsupported instructions) because the very first time through your process loop things will end badly, so the user will see an immediate problem on their machine.

If it’s the other sort of illegal instruction (say jumping to an invalid memory location and hence executing rubbish as instructions) then the plugin will likely work for a little bit then go wrong, so the nature of the crash the users experience will help you track whether this is a build problem or something more sinister

1 Like

“Illegal instruction” is something I sometimes end up with when I write code in haste and leave a non-void function without an appropriate “return” statement at the end.
But the IDE should give you warnings about that (as it does in my case, so I easily spot the culprit)

Being a couple of years into plugin development this feels like a stupid question : how do you disable AVX?

VS won’t compile with standard settings in this case (unlike gcc…)

You can add an “Extra Compiler Flag” /arch:SSE2 for anything up to SSE2 (i.e. no AVX)
But please read the following for my doubts on whether this is working as I say:

I think the culprit code is this:

for (int voice = 0; voice < VOICES; ++voice) {
    p_source_pointers[voice] = m_sources->global_lfo;

Now combining the words “Illegal instruction” with the fact that this portion of code could be a prime example for vectorization, I might still be havin a problem with AVX / AVX2.

Could it be that that using /arch:AVX does not disable AVX2 for some reason?
I will try to get more information about user CPUs (the ones I’ve seen have AVX but no AVX2) and try to avoid optimization / vectorization on these particular lines.

1 Like

To see if your compiler really does vectorization there, you could look at the Assembly code generated by the compiler. I am working with Xcode, there you simply create an assistance editor and select Assembly in the upper left.

1 Like

Yes I did exactly that and surely enough I found a nasty vpbroadcastq instruction in there. This belongs to AVX2 according to this list of AVX2 instructions

I will now list the steps to find bad instructions for future lost souls in search of answers:

  1. Compile your project in Release with debug symbols (Projucer->Exporters->VisualStudio2019->Release->Debug mode “Enabled”)
  2. Build the project and set a breakpoint where you expect the evil code part
  3. Run your code and hit the breakpoint
  4. Right-click on the code line and “Go to dissasembly”
  5. You’ll find each line of code and underneath the instructions used to execute it!

The question remains, why VS would compile in AVX2, when I set the flag /arch:AVX. I could manually fix the lines in question, but I’d rather have a future-proof solution.

The problem arose a while back, when I realised that my linux build was using -march=native, which enables everything that is present on the build machine. This default value should be changed immediately IMHO btw.

Anyway, I settled for AVX, which seemed a reasonable choice and set the according flags on all exporters. But right now I’m considering dropping it again. I never measured if there’s an actual performance increase, and I get 3 emails a day of people who can’t open the plugin because they don’t have AVX.

As people don’t seem to have problem with AVX (as I take it from @richie’s answer), maybe putting no flag in there at all just leaves all AVX out on win and mac? But still uses SSE/SSE2?

What are you guys using?

I use SSE2 only for compatibility reasons (with cheap “Pentium” and “Celeron” CPUS). On win I use /arch:SSE2 and with XCode I think setting the Deployment Target automagically selects the right SIMD architectures for the machines that are able to run the set version of OSX. I still use 10.7 as deployment target and AFAIK this means just SSE2.
64 bit builds are always SSE2 anyway on all platforms. Unfortunately VS2019 complains about the /arch:SSE2 flag when doing a 64 bit build, but Projucer forces me to define it per exporter and not per config.


The problem arose a while back, when I realised that my linux build was using -march , which enables everything that is present on the build machine. This default value should be changed immediately IMHO btw.

We just got burnt by this. We were building pluginval on Azure build machines. Then grabbing the binary to run tests on our local build machines. I guess Azure just upgraded their build hardware, since all of a sudden the binaries would no longer run on our machines and fail with Illegal Instruction

1 Like

Using whatever Xcode sets as default, which I assume is SSE4.2 or below.
If you want to have performance and compatibility you can simply (haha) compile twice (or more often) with different settings, and decide at runtime which versions you call. Fabian gave me that advice at ACD couple of years ago, right aftert that I started the following thread, maybe it helps:

1 Like

It was changed between JUCE 5.4.6 and 5.4.7:

1 Like

Now it defaults to None for architecture. Should have have to select something?

right, this could explain why one of my customers complained that a previous version of my plugins worked fine for him under Linux and then suddenly stopped working. Thankfully the changes were just about stability fixes for Windows and so he didn’t care, but it was probably a change in the Azure Pipelines hardware meant all of a sudden there were AVX instructions in the build that weren’t there before and I was clueless as to why.