[PROJUCER] Shared Code > 2GB Lib


#1

So I’ve reached the limit of 2GB for a static library in Visual Studio… any ideas to let PJ let us create more than a single static library so the solution can be built?

I did try to use the 64 bit VS tools in the IDE:

https://docs.microsoft.com/en-us/cpp/build/walkthrough-using-msbuild-to-create-a-visual-cpp-project#using-msbuild-to-build-your-project

unfortunately it’s not compatible with some external tools I use.

Cheers,

Rail


#2

I guess you didn’t write 2 GB of code, so there might be some resources (bitmaps etc.) in the code as well? I would recommend moving them out into an external file (like a zip-file) and then loading them at run-time.


#3

The only resources are two fonts… and the Lib includes all of JUCE

If they could make a separate Lib for JUCE that’ll probably be good.

Rail


#4

So it’s really two gigs of pure code? I guess the fonts are less than 1 MB each? That’s impressive. Just out of curiosity, how long does a “rebuild all” take?


#5

A final build which adds external code for protection takes about an hour to build each version on Windows… using Shared Code reduces it from a 4 hour ordeal to just over an hour. On MacOS a build takes about 20 minutes.

On Windows before I switched to using an SSD drive for the Parallels VM and excluded everything from Windows Defender… it’d take about 8 hours to build the Windows versions on a 3.4GHz i7 with 32GB of RAM.

Rail


#6

You mentioned protection, I assume it’s nothing like 2GB for a release build (without protection), I think I can hazard a guess as the to the protection your using and I would guess it’s a bit OTT on the amount of injected code. Is this for a plugin or an Application?


#7

It’s for a plug-in… but the plug-in has a full Mixer with internal effects as well… so not your typical plug-in.

Yes, the Protect build is the only config where the library is > 2GB. I’m trying different methods to reduce the size, but ultimately the solution will be to split the shared code into more than one library.

And to answer the next question… the final VST DLL size using JUCE 4.2 is around 30MB on Windows.

Rail


#8

That’s crazy! Surely it must be stuffed with debug symbols…? Even if a fully compiled instance of every class in JUCE was included in the library I can’t believe it’d be that big…


#9

Bytes aren’t worth what they used to be when I was a kid.


#10

More like injected/obfuscated code. I’m trying to build the Lib with Link Time Code Generation disabled to see if that makes any difference.

Even if a fully compiled instance of every class in JUCE was included in the library I can’t believe it’d be that big…

I checked the sizes of the OBJ files for the JUCE code zipped in the lib and they’re not the issue.

Cheers.

Rail


#11

So with “Link Time Code Generation” disabled the lib is much smaller… but the Standalone won’t link to it… will keep trying :slight_smile:

Rail


#12

Did you remove the flag on both? What’s the error message (as default LTCG is quite new, there shouldn’t be a problem linking to a “usual” library).
One of the reasons LTCG libraries are bigger is that all the intermediate code has to be added to the library on top of the compiled code. Without it, the library will be smaller and the linker will have less opportunities to crash (especially with obfuscation).


#13

I set the code generation in both the Shared and the Standalone to /MT – which is normally fine… but with LTCG disabled (in both) when linking the Standalone it’s complaining that the file was compiled with a mismatch /MD

Rail


#14

That’s odd. Are you sure that you are linking against the proper library and that it was generated properly? You have a mismatch, clearly, perhaps even you need to rebuild the solution to be sure that you have /MT everywhere? (shouldn’t happen, but one never knows).


#15

Yeah, everything seems right… but I’m going over everything again to make sure.

Thanks,

Rail


#16

Let us know what your conclusion is.

TBH this would be really hard to fix in JUCE.


#17

Disabling LTGC I’ve reduced the lib to 229,304 KB and the Standalone builds. I’ll have to do testing to see if I lose any optimizations with LTGC disabled and using a static lib… I may have lost some inline code.

Cheers,

Rail


#18

There really should be a way where VS will link the static shared code lib with LTGC enabled but then remove all the extra LTGC stuff from the resulting static lib. This way you would still benefit from LTGC for all the code inside the shared code lib.

EDIT: This is not possible. See @Matthieu_Brucher comment below!


#19

I think to do that normally you’d just use the x64 tools… so I’ll work on getting the external tools I use to work with the VS x64 tools… It’s probably a syntax I have to figure out.

Thanks,

Rail


#20

A static library has no link stage, the real link is done after, so the LTGC is only done when you create the plugin. I don’t see how it would be possible otherwise.