Proactive Measures To Prevent Inno Setup Installer From Getting Flagged By Windows Defender

Windows
1

I’m a bit of a Windows dev newbie. I recently built my JUCE Plugin on Windows 11, and used Inno Setup to create an installer, using a very basic .iss configuration to distribute the plugin in VST3, CLAP, and AAX.

It worked fine, and I signed the installer (using Azure Trusted Signing) and uploaded the .exe to github, thinking everything was sorted.

However, when I tried to download the installer from github, Windows Defender intercepted the download and instantly removed it from my machine. Windows Defender reported to me that it had classed the exe as a trojan detected: trojan:win32/wacatac.B!ml

I looked into it and it seems this type of false positive is not uncommon, especially when using inno setup.

I tried multiple things to rectify the issue, and eventually found I was able to get Windows Defender to allow downloads of my installer by rolling back the version of Inno Setup.

However, I have also seen that Windows Defender is still liable to start false flagging installers even after they’ve been out in the wild for a while.

So I’m hoping to hear from others who’ve been building for Windows for longer and have experience working with this issue. Is there anything that can be done proactively once you cut an installer to help prevent Windows Defender from throwing these False Positives? Similarly, what’s the best pathway to remediation when it instantly flags the installer and you are unable to alter the inno setup config?

2 Likes
2

Did you sign the plugins themselves?

It may be Windows defender sees it as a Trojan because a signed app has unsigned binaries within it.

3

This exact issue made me move away from InnoSetup and get into WiX - which is much harder to learn, but produces proper .msi installer packages that are less likely to get flagged.

My guess is that InnoSetup is used to create installers to distribute Malware and the “intelligent” virus scanners mistake parts of the InnoSetup code for malware.

3 Likes
4

Yes all the individual plugins were signed. It stopped getting flagged when i changed the inno setup version from the most recent, to the second most recent, without changing the config or plugin contents :person_shrugging:

1 Like
5

I remember looking at WiX early on, but ended up going with inno b/c it seemed popular and there are a lot of good usage examples on Github.

Anyway, I just did a google search though and your blog post on it popped up. After reading, I might move to this myself. Looking through your repo, it doesn’t seem too crazy to get a hang of, especially for simpler use-cases.

6

you could say that again!

So far every tutorial resource is along the lines of, “just do this” without any explanation of why. GUIDs everywhere with no explanation of what they do and why you should set them. The WiX website is an inpenetrable mess in my current state of mind! :joy:

But does seem like it might be the best solution when I can manage to decipher it all, as I’ve run into this annoying “your installer is malware” false positives a few times with InnoSetup.

7

Does anyone know how WiX compares to NSIS, out of interest?

I believe NSIS is the default for CPack which I’ve always been tempted to use but don’t fancy migrating all our Inno scripts!

8

We’ve made a WiX installer.
Ironically it’s an over complicated abstraction of Windows Installer.

You have other software companies that made simplified wrappers around WiX / Windows Installer but they’re costly.

About CPack,
There’s a discussion here. I’ve tried using CPack but it also had many gotchas and lacked some flows I’ve needed ironically in each platform / Installer.

2 Likes
9

I switched to innoSetup from WIX and it took not long to get the first virus scanner malware report by user.

What WIX version are you using? I wonder if it is still possible to install WIX. Only found nuget packages of the latest versions.

Edit: I’m using version 6 now. Finally was able to make it work with the extensions. Docs could be better. Also, a lot has changed with the version 5 update. This makes things even more complicated when searching for solutions and examples.

10

I’m experiencing the same issue with InstallBuilder! I’ve contacted both InstallBuilder support and Windows support, but neither has been able to help me.

11

What is the problem with nuget packages?

But there is also this:

12

I figured it out. Just needed to create a wix project file and use dotnet build to create the msi. This way it downloads the nuget packages automatically.

For others:

Here is an example:
https://docs.firegiant.com/quick-start/

I added dependencies in the project file:

<Project Sdk="WixToolset.Sdk/6.0.0-rc.1">
<ItemGroup>
  <PackageReference Include="WixToolset.Util.wixext" Version="6.0.0-rc.1" />
</ItemGroup>
<PropertyGroup>
  <OutputPath>.\</OutputPath>
  <OutputName>installer_vst3_64</OutputName>
</PropertyGroup>
</Project>
13

Anyone on WiX want to share their config or tips?

Thinking about going down the rabbit hole and creating a template for the community (unless there’s a better way to create .msi installers)

5 Likes
14

Our signed installers using InnoSetup have never been flagged by Windows Defender.

Make sure your payloads are signed and make sure you installer is signed.

1 Like
15

Also had no problems with Windows Defender so far, but we had a report from a user with a 3rd party virus scanner. This was reason enough for me to switch back to WIX and msi’s.

16

I’m happy to help. This probably also helps to improve our script. PM me if you start.

2 Likes
17

I shared my WiX v5 scripts here (+ on GitHub) - criticism would be welcome.
I would have loved to have some better examples online at the time.

https://apulsoft.ch/blog/windows-msi-plugin-installer-wix5/

5 Likes
18

@pflugshaupt Awesome! This is great, thanks!

19

Btw. I’m looking for a solution to make a .msi installer overwrite a single .vst3 file with a .vst3 folder of the same name to switch to the new(ish) vst3 folder structure. Windows Installer refuses to do that and tells me it can’t overwrite a file with a folder of the same name :roll_eyes:. There doesn’t seem to be flag to make it happen.

20

I also run into this. Also have no direct solution, but probably you manage to remove the folder during installation with

<RemoveFolder Id="Remove" On="both"/>

Unfortunately I wasn’t able to delete folders so far, only files.