Bifrost overflow on file size error...

[arthur]

When i am caching out a bifrost sim, if my file size per frame goes above about 4.2gb. when i try to load the cache back in i get this error.

"K:/cache/bifrost/Bifrost_test_06/bifrostLiquidContainer1_p.0047.bif (Overflow on file size)" in the console window and the cache wont load. 

As long as i keep my files per frame below this size its fine.

is this a problem with bifrost?

thanks

Arthur

[Anifex09]

This sounds like a 32bit limitation or a File table formate issue. I assume bifrost engine is running in 64 bit. I have yet to receive my upgrade so I can't test this. If you are working on a windows platform open up the task manager while you are doing a sim and see if the bifrost app is running in 64 bit mode.

[arthur]

when calculating a sim, bifrost doesnt show up as a seperate process in the task manager, it shows maya using the cpu process.

Arthur

[agraham]

Which file exactly is going above 4.2Gb? Your user cache? Your meshed sim?
A 4.2Gb user cache is pretty huge, how many particles/voxels are in your sim?

[arthur]

it is the particle cache that is going over 4.2gb and yes this is a large file, but i am siming a car splash that is pretty close to camera for film and yes i can get around it by layering which is what i will do .

but it is not unheard of to have extremly large cache files when dealing with film res assets on, so i am surprised that maya can write out this size file but cant read it back in

Arthur

[agraham]

You're saying that each .bif file written to disk is 4.2Gb?

I've dealt with some pretty large sims, but I haven't come across a 4.2Gb file representing a single frame of sim data.

What kind of numbers are you generating? How many particles/voxels are in your sim? What is your scene scale?

[arthur]

my sim size per frame ranges from 1.2gb to 4.7 gb per frame.

its about 80mil particles and 42mil voxels.

my scene is set to cm's and i am working on 12 x 12 units for the actual bifrost emiter.

my voxel size is down to .012

This sim is a 4k shot with a car landing into the ocean and is full frame, so far i have 3 layers of different bifrost sim's and particle for the effect, and yes i can reduce to get the file under 4.2gb.

But it would be good to know if this 4.2gb is a limitation.

and i have worked with larger sims in softimage and houdini, and for a feature film effect i dont think it's unheard of having large sim's

Arthur

[agraham]

Just out of curiosity, you're not on a 32-bit OS, are you? I assume you're on Windows 64 bit?

[agraham]

Y'know, I apologize for not believing you at first 🙂

This is a bug, and I'm calling this a showstopper. The ID for this issue is:

    MAYA-38451 Large user cache files not readable

I was easily able to reproduce this using Maya 2015 SP2 (which was released this week).

We're working on compression for .bif files at the moment, which will fix the problem a bit -- until you hit that limit again. The underlying issue is: why can't Bifrost read files over a certain size?

[arthur]

the compression will help, look forward to seeing a fix for this, as bifrost is looking pretty sweet, but i am constantly hitting the 4.2gb limit.
cheers
arthur

[agraham]

We found the issue. Turns out to be a compiler limitation, which you learn more about here.

We're working on a fix, but for the record, with compression a .bif cache file with 260 million particles took up only around 2.3Gb of space. The voxel .bif took 850Mb. Compression is making a huge difference, but we're still going to fix the issue.