Sub-Folders Per Folder Before Performance Drops?

[DanSingleton]

Using Vault Basic 2016, is there a maximum or suggested number of sub-folders that can be contained in a parent folder before performance degrades?

We've recently started moving our older/archival CAD files from our communal network drive location into our existing vault. As we're populating the folders in Vault we're seeing a massive degradation in performance once we get above around 1000 subfolders in a parent folder? Considering Windows can have like 4,000,000,000+ sub-folders/files per parent folder 2000+ doesn't seem like it should be such a big deal in Vault? 

For example, we currently have a parent folder containing 1597 sub-folders (and growing). When clicking on the parent folder the view pane freezes, displaying a "Please wait for the grid to populate" message and doesn't populate for roughly 50 seconds (timed it on my phone). The same lag is now observed when right-clicking the parent folder and trying to create a new sub-folder or when dragging and dropping a directory from the local harddrive into that parent folder to automatically create sub-folders and check-in the files.

Obviously, 50+ seconds to create each sub-folder is horrible, unacceptable, performance. Is anyone else seeing this? Autodesk, what's the scoop? 

Any help would be greatly appreciated.

-Dan

[Neil_Cross]

I've worked on a lot of big Vaults and although (obviously) I've never counted the number of sub-folders, there's been some big Vaults, arguably every folder in Vault is a subfolder to some degree as they're usually always under a parent 'Designs' folder of some sort, but I've never seen it perform that bad.

I'd start by checking over your environment first.  If you're using Vault Basic, is it accurate to assume you're using the SQL Express that came with Vault?

If so, SQL Express is handicapped as to how much server resource it can use.  The big flagship banner limit which is associated with SQL Express is the database size limit, but it's other limitations might be hitting you here.  Again I'm assuming you're using Express but if you're not, none of this applies, but if you are, it's important to note that SQL Express can only utilise 1GB of RAM per instance.  If you're smashing it with that much info, it'll undoubtably be hitting it's RAM cap and bogging out on you.  There is also a CPU core limit on Express but I doubt that would be the issue here.

If you are using Express, I'd recommend downloading a trial of SQL Standard Edition and restoring your backup into a test system with that on and compare the performance.

If you're not using Express, can you list what your server spec is, what kind of network you're on 100mbps/1gpbs, wi-fi/cabled, are these folders all empty or populated with files?

[Anonymous]

Not going into the back end with SQL at all, I'll leave that for the professionals.

Recommendations to us from Autodesk and resellers has always been to flatten out the folder structure as much as possible, and rely on the fact that it is a database, and that search results are returned rather quickly. Customising views with filters and using the search tool speeds up the process remarkably, particularly if you're doing the same search regularly.

We had many sub folders and found that you would spend more time drilling down into sub-folders than the time taken to search at a higher or highest level.

The limit from the top level down for us was around 6-7 sub levels/folder.

The paradigm shift is realising that the folders themselves are just displayed on the left as a hangover from the windows system.

[Neil_Cross]

I have a backup of one of the larger Vaults I've worked with, there is a parent folder here with a total of 94,000 sub folders under it, running that Vault in a VM with full SQL Standard Edition is pretty quick, it takes less that 1 seconds to click the parent folder and have it return the sub folders.  The 94,000 sub folders are not all in a flat list however there are around 1000 sub folders per level.

It sounds to me like you're hitting a hardware or resource limitation, or you have an isolated database issue, I would stick my neck out and say this isn't a Vault problem as such or related in any way to how you've structured your folders.

Edit: If you want to see proof, I can attach a video of me clicking through parent folders which have 999+ subfolders underneath so you can see how quick it should be.

[Henry.Huang]

Hi, Dan 

What version SQL are you using? Also what' your server's hardware specification?

Henry 

[DanSingleton]

Vault Basic Server (2016)

Vault Basic Server 21.0.50.0

Build 21.0.50.0

SQL Server 2012 SP2, Standard Edition

(We were reaching SQL Server Express limits on our previous server, so we upgraded to full blown SQL server when we moved to this newer server).

Vault server (Virtualized with VMware)

Windows Server 2012 R2 Standard

CPU: E5-2620 @ 2.00GHz (2 processors)

RAM: 16 GB

HD: 300GB (230 used, 70 free)

System type: 64-bit OS

CPU utilization averages about 10%

Memory utilization averages around 30%

Here's a clip of our folder structure... The highlighted "4" folder could have upto 9999 folders, currently at 1598. It's not a horrendously "deep" folder structure. The 00 sub-folder located under the "4" would be the deepest we go.

 Edit: Added Vault Server and build info.

[DanSingleton]

The issue seems to be related to the "Default View". If I change to "Paged View" the grid populates almost instantly.  Which makes sense because then it's only displaying 100 folders and their related info, not having to propagate 1598 folders information into the full grid of the default view.

Could be related?

https://forums.autodesk.com/t5/vault-general-discussion/vault-2012-slow-waiting-for-grid-to-display/...

https://forums.autodesk.com/t5/vault-general-discussion/vault-client-performance-slow-2013/td-p/3646...

https://forums.autodesk.com/t5/vault-general-discussion/vault-client-2013-basic-files-not-showing/td...

[DanSingleton]

"If you're not using Express, can you list what your server spec is, what kind of network you're on 100mbps/1gpbs, wi-fi/cabled, are these folders all empty or populated with files?"

Sorry... Missed this question earlier. Server specs posted above. 1GB ethernet (doesn't seem to make any difference if I'm at 100mbps. Just tested at both speeds. Plugged into the 100mps network we use for our phone lines...) The folders are populated primarily with Inventor IPT/IAM/DWGs and AutoCAD DWG files. We also have a smattering of STEP, PDF, XLSX/M, and Word docs here and there.

[DanSingleton]

Still having the issue...

[Anonymous]

I'm guessing you're still having the issue post changing the view?

I'd suggest sending your vault log files to your reseller and discussing the issue with them, or another Vault specialist you may have locally as their may be something else happening that we cannot diagnose over the forum.