Microsoft compiler running out of heap space

I’m having trouble with my windows build (linux works fine). The compiler says

C1060: compiler is out of heap space

and refers me to an array

float m_wavetables[160][35][512] = {0};

This array should be just under 11Mb in size. I’ve heard somewhere that there’s a 10Mb size limit. Is this the case and is there a way to circumvent this? I’m already using the 64Bit compile tools.

How about creating that memory dynamically instead of statically? At least that way you don’t have to deal with the compiler complaining.

1 Like

For me, Visual Studio 2017 doesn’t even give an error for this, the compilation just seems to get stuck and never finishes. It looks like a compiler bug, but that said, it isn’t really the best of ideas to have huge raw arrays like that as member variables anyway…

class Big
{
public:
	float m_wavetables[160][35][512] = { 0 };
};

void test_big()
{
	Big x;
}

I’m always trying to avoid dynamic memory allocation, for convenience as well as leaky reasons…
In this situation I should give it a try though. However doing this will be a major operation across the entire project, since it’s at the very heart of the engine.

Hm okay thanks for the test. My project is far bigger though, that’s just the point where the compiler is complaining.

For that reason there is MemoryBlock (void*) or HeapBlock (templated)

2 Likes

ooooookk, is there a thing JUCE doesn’t do? :smiley:
Will give it a try thanks :slight_smile:

1 Like

Using those is complicated by the poster wanting to use the convenient 3D indexing into the memory…

Does something like this work, also with the 3D indexing?

HeapBlock<HeapBlock<HeapBlock<float>>> tables;

Oh yeah there’s certainly no way around that!
However allocating and deleting a single array should be within my abilities for a single array even with normal dynamic allocation :slightly_smiling_face:

Well, creating an inlineMethod is not too hard:

size_t width = 512;
size_t height = 35;
size_t depth = 160;
HeapBlock<float> m_wavetables (width * height * depth);

size_t indexFromPosition (int x, int y, int z)
{
   return x + (y + z * height) * width;
}

m_wavetables [indexFromPosition (5, 10, 15)] = 5.0f;
auto foo = m_wavetables [indexFromPosition (5, 10, 15)];
1 Like

Could also have one of the array dimensions be dynamically allocated, this seems to be enough to allow Visual Studio’s compiler to compile the code. No guarantees this code is 100% correct… :wink: (I did some basic tests which indicated it should be fine.)

class Big
{
public:
	float* m_wavetables[160][35];
	Big()
	{
		for (int i = 0; i < 160; ++i)
		{
			for (int j = 0; j < 35; ++j)
			{
				m_wavetables[i][j] = new float[512];
				for (int k = 0; k < 512; ++k)
				{
					m_wavetables[i][j][k] = 0.0f;
				}
			}
		}
	}
	~Big()
	{
		for (int i = 0; i < 160; ++i)
			for (int j = 0; j < 35; ++j)
				delete[] m_wavetables[i][j];
	}
	// don't really want to be dealing with the copy and move stuff for this class, so just disable copying and moving...
	Big(const Big&) = delete;
	Big(Big&&) = delete;
	Big& operator=(const Big&) = delete;
	Big& operator=(Big&&) = delete;
};

I probably wouldn’t do anything like that in my own code and would rather opt for something like what Daniel posted above. (Even if it meant extensive rewrites of existing code.)

Thanks to all for your replies and suggestions! I got it working now (not linking though :sob: ) with a (not really) simple 3D-memory allocation.

I’d rather not use the Memory- or HeapBlock methods, since I won’t have access to the multidimensional bracket operator [x][y][z] and that means going through half the codebase and fixing stuff.

// declare a structure containing your wavetable.
struct wavetable {
	float samples[160][35][512];
};

// allocate it on the heap, with automatic memory management (no need to remember to delete it).
auto m_wavetables = std::make_unique<wavetable>();

// Example of accessing wavetable using multidimensional bracket syntax.
m_wavetables->samples[3][3][3] = 1.0f;

// Example of accessing wavetable via a reference to make it even more convinient.
auto& wavetable = m_wavetables->samples;
wavetable[3][3][3] = 1.0f;
1 Like

Allocating it on the heap blew up the VS2017 compiler just the same for me.

class Big
{
public:
	float m_wavetables[160][35][512];
};

void test_big()
{
	Big* x = new Big;
}

It looks like there is something about float m_wavetables[160][35][512]; as a member variable, that the Visual Studio compiler can’t handle.

More like:

class Whatever 
{ 
public:

     Whatever()
     {
     m_pWavetables = new float[160*35*512];

     }

     ~Whatever()
     {
     delete m_pWavetables;
     }

private: 

     float* m_pWavetables = nullptr; 
}; 

Rail

That isn’t what the original poster wants to use. You are allocating a one dimensional array with that and it can’t be indexed as a 3D array. (Without changing the source code the poster already has.)

Can’t you define it as a float***, and allocate and de-allocate in nested loops in the constructor/destructor? Won’t that allow accessing via m_wavetables[a][b][c]? (Not sure, but it seems legit to me.)

That would be a pointer to a pointer to a pointer.
If you use the analogy of float** in the AudioBuffer.getArrayOfWritePointers(), it returns an array of actual pointers. So in your case float*** would be an array of pointers, that point to all the float** pointers, that point to float*, that eventually point to the actual data.
Not what you expected, and a lot of values to create behind the scenes.

I don’t see anything wrong with my idea. Here is a complete working example (with sizes [7,8,9] instead of those huge numbers, just to make it easier to see in the debugger output:


const int asz = 7;
const int bsz = 8;
const int csz = 9;

struct mystruct
{
	mystruct()
	{
		pMyFloats = new float**[asz];
		for (int a = 0; a < asz; ++a)
		{
			pMyFloats[a] = new float*[bsz];
			for (int b = 0; b < bsz; ++b)
			{
				pMyFloats[a][b] = new float[csz];
				for (int c = 0; c < csz; ++c)
					pMyFloats[a][b][c] = 100 * a + 10 * b + c;
			}
		}
	}
	
	~mystruct()
	{
		for (int a = 0; a < asz; ++a)
		{
			for (int b = 0; b < bsz; ++b)
			{
				delete [] pMyFloats[a][b];
			}
			delete [] pMyFloats[a];
		}
		delete [] pMyFloats;
	}
	
	float*** pMyFloats;
};


//==============================================================================
int main (int argc, char* argv[])
{
	mystruct x;
	
	for (int a = 0; a < asz; ++a)
	{
		for (int b = 0; b < bsz; ++b)
		{
			for (int c = 0; c < csz; ++c)
				fprintf( stdout, "Value at [%d][%d][%d] is: %4.0f\n\n", a, b, c, x.pMyFloats[a][b][c] );
		}
	}

	return 0;
}
1 Like

Alright that works, somehow I got a knot in my head :wink:
But I wouldn’t use that, call it personal preference…
Cheers

1 Like

A more object-oriented approach would be for there to be several classes, say A, B and C. You have an array (myArray) of size aSize of objects of class A. Class A has an array of size bSize of objects of class B. Class B has an array of objects of size cSize of floats. All three support the [index] operator. The [index] operator for each class returns a reference to the specified item from its internal array. Thus, myArray[a][b][c] returns a reference to a float, as desired.

2 Likes