Libs and Catalog Databases performance over network

[EE1337]

We are running Autocad Electrical 2015 with a central copy of Libs and catalog databases on the network drive, and performance is very slow for instance when generating a terminal strip.  It takes maybe 30 seconds to load a terminal strip with 8 terminal blocks, but on a machine with all the Libs and .mdb files on the local SSD, it is very fast.  The network usage is very high during this time, and I believe it takes a long time to read a large number of small files off the network.  

We are going to move everything from the network drive onto Vault, so that users can work off a local working copy of these files, and gain the performance advantages of working off the SSD.  

The problem is with the Libs folder, Vault doesn't want to let me load in all the .dwg files outside of Autocad Electrical.

Is this even a good idea?  Is there a way to load our component libraries into vault so that users can work off their local copy?  

[drathak]

---

@baberb1984 wrote:

We are running Autocad Electrical 2015 with a central copy of Libs and catalog databases on the network drive, and performance is very slow for instance when generating a terminal strip.  It takes maybe 30 seconds to load a terminal strip with 8 terminal blocks, but on a machine with all the Libs and .mdb files on the local SSD, it is very fast.  The network usage is very high during this time, and I believe it takes a long time to read a large number of small files off the network.  

We are going to move everything from the network drive onto Vault, so that users can work off a local working copy of these files, and gain the performance advantages of working off the SSD.  

The problem is with the Libs folder, Vault doesn't want to let me load in all the .dwg files outside of Autocad Electrical.

Is this even a good idea?  Is there a way to load our component libraries into vault so that users can work off their local copy?  

---

We are dealing with a similar problem.  Our original reseller suggested putting everything on the network like that for shared access.  And It SUCKS for performance.

We've kicked around the idea of having all the symbols, catalogs and other support files in Vault but settled on having a two part system.  There will still be a common network location for these files (which will be vaulted), but the local ACADE startup is going to be handled by a script that checks for any updates to the central file store and updates the needed files before launching ACADE.  This way people will always have the latest version of symbols, catalogs, menus, etc.  Content editors like myself will be able to push files in the other direction and set the update flag.  

A bit crude, but at least at this point looks simpler than trying to keep everything in Vault and accessing that as needed.  That would be even slower.  (Or at least as slow if still pulling from a shared location.)

I am certainly open to better solutions as this is just on the horizon for us as we upgrade to 2015.

[mathalekar]

Hello Joe & Brad,

We would like to understand your environment for analysing the root cause.

1. OS you used

2. Network deployed or standalone shared ?

3. All (Libs, Catalog data, Databases) shared ?

4. WAN or LAN ?

5. How many users conencting the shared databases ?

There could be various factors that might affect accessing data over network. Help us to understand your work environment so that we can see if there is a possible hint for us to find root cause.

[drathak]

Thanks Mathalekar,

1. Windows 7 Enterprise 64bit SP1

2. From my IT person: FAT client install with shared network workspace.

3. Yes.  Libs, catalog DB, other databases, menu files, plot related files (PC3 & CTB) as well as the working folder.

4. LAN  More details were requested from IT.

5. Around 15 at any given time.

Pretty sure our IT folks have been over this with Autodesk before.  And everyone at AU recoiled in horror over the fact that we are using a network folder for the working folder.  So that's our #1 thing to change with Vault during this upgrade cycle.  Moving everything else back local is just making things simpler and reducing network traffic as much as possible.

[EE1337]

I believe it's taking time to read through the .mdb files, as well as all the .dwg files in the Libs folder.  This is not a problem with a single block, but the terminal strip can be slow because it has to read so many small files (I imagine).  

It may also be possible that I don't have my Support File Search Paths setup correctly, and it's having to search through all the directories in order to find what it's looking for?

Here's what we're running:

1. Windows 8.1 x64 Pro

2. We haven't setup a deployment, and we are not sharing files from any one computer to another.  We have a network drive for shared files.

3. The files shared on the network drive include the wd.env, the Catalogs and PLC folders (with the .mdb files), Libs folder, and Templates and Reports settings.  

4. 10 Gbit LAN

5. I think we have 6 seats, but currently I am the only one using Electrical

I attached a screenshot of my Support File Search Paths, this seemed to be the only way I could get it to work through much trial and error.  Maybe I need to move things around?

[RBDIII]

---

@drathak wrote:

Thanks Mathalekar,

1. Windows 7 Enterprise 64bit SP1

2. From my IT person: FAT client install with shared network workspace.

3. Yes.  Libs, catalog DB, other databases, menu files, plot related files (PC3 & CTB) as well as the working folder.

4. LAN  More details were requested from IT.

5. Around 15 at any given time.

Pretty sure our IT folks have been over this with Autodesk before.  And everyone at AU recoiled in horror over the fact that we are using a network folder for the working folder.  So that's our #1 thing to change with Vault during this upgrade cycle.  Moving everything else back local is just making things simpler and reducing network traffic as much as possible.

---

We also had the same problems with network lag doing everything from the network.  We now work locally on our machines (using Vault).  We still have all of our databases for catalogs, components, and footprints shared on a network server.  Read/write access is only given to those that are permitted to change or add new components, footprints to the libraries.  You just have to modify a couple autocade config files to point to the new server locations. This made a huge difference in performance using ACADE. Hope this helps.

[EE1337]

I tried Vaulting the Catalog and PLC .mdb databases, but see no noticeable difference in performance using those off the local SSD.  

So either my settings have acade searching all over the place to find the files it needs, or there is true network latency accessing a large number of small .dwg files over the network.  

I did try to Vault the Libs files, but I wasn't able to easily copy all the .dwg Libs files into Vault because it wants you to bring them in through the ACADE application.  Maybe I should create a project for my Libs files so that I can easily Check-In All?

[dougmcalexander]

Are you using a 1GB router switch and NIC?  I always recommend upgradng the network infrastructure for customers who will share project data over a network.  The drawings themselves are not so much of an issue.  I have been using the program since 1997 over a network and sharing projects is not much of a problem.  

Keep your default_cat.mdb file trimmed down to favorite vendors/part numbers.  Make a copy of the full version and name it complete_cat.mdb.  Set your project properties to use the complete database as a secondary.  This serves two purposes.  You get the speed increase of a smaller database, but you also created a preferred vendor catalog, while still maintaining access to the full catalog.

Sharing menus and library folders can create a bottleneck.  Some companies have their IT department create a login script that copies the menus and libraries to the local drive path during login. You could also keep the stock Autodesk libraries local (i.e. JIC, IEC, etc.) and only maintain the custom symbol on the network.

[drathak]

---

@dougmcalexander wrote:

Are you using a 1GB router switch and NIC?  I always recommend upgradng the network infrastructure for customers who will share project data over a network.  The drawings themselves are not so much of an issue.  I have been using the program since 1997 over a network and sharing projects is not much of a problem.  

Keep your default_cat.mdb file trimmed down to favorite vendors/part numbers.  Make a copy of the full version and name it complete_cat.mdb.  Set your project properties to use the complete database as a secondary.  This serves two purposes.  You get the speed increase of a smaller database, but you also created a preferred vendor catalog, while still maintaining access to the full catalog.

Sharing menus and library folders can create a bottleneck.  Some companies have their IT department create a login script that copies the menus and libraries to the local drive path during login. You could also keep the stock Autodesk libraries local (i.e. JIC, IEC, etc.) and only maintain the custom symbol on the network.

---

Here is what I just recieved from our IT contact:

   Currently we have 100MB from Access Layer Switches to Desktops.

   2GB Uplinks from Access Layer to a 10GB Core Network with 10GB connectivity to servers.

   We do utilize QoS to prioritize Voice traffic for our phone system. 

Our default_cat.mdb is very stripped down, with the complete catalog as the secondary.  Both are on the network share.

And I must admit that lately things have been going pretty smooth with the exception of the Terminal Strip Editor.  I just timed it and from Right-Clicking and choosing "Edit Terminal Strip" until the edit window showed up and I had control again takes 1m 12.6s.  Launching the TSE from the ribbon takes about the same length of time before I can choose which termninal strip to edit.

To me, that is way too long when I have lots of these to deal with. This first project I just issued had 109 Terminal Blocks.  (So nice to be able to get that information quickly!!)

[dougmcalexander]

TSE should be reading the project scratch database on your local when it compiles the list of terminal strips, so that shouldn't take long at all. It's when the TSE must access the footprint lookup and load the footprints for the layout view that you should see a slight bit of network latency. Makes me wonder if someone tried to configure you to store your project scratch databases on the server. People have tried that but, it isn't necessary, and it slows things to a crawl.

100MB between your workstation and the router switch is also a bottleneck. 1GB speed has been available for many years. That's 10X the throughput.

[drathak]

Scratch DB is local.  And once I finish any adjustments and switch to the graphical layout tab it pulls the footprint fairly quickly.  It is only on the initial launch, when it is reading the scratch DB I presume.  Or maybe rebuilding/freshening it first.

Not my call on the link to the switch or the switch speeds. (I'm GB+ at home.)  That upgrade is in the works though.  Not a cheap undertaking when you have several hundred PC's on the network.  I told them they could start with our switch and do the rest later but their sense of humer didn't extend that far.

[dougmcalexander]

I feel your pain. I once asked my boss if I could work from home because my computer at home was twice as fast as the ones at the office.

Have you tried deleting the scratch database and restarting AutoCAD Electrical?

[drathak]

---

@dougmcalexander wrote:

I feel your pain. I once asked my boss if I could work from home because my computer at home was twice as fast as the ones at the office.

Have you tried deleting the scratch database and restarting AutoCAD Electrical?

---

Yep.  That and AEREBUILDDB from within ACADE.

It is also related to the number of terminals in the project.  Just tried a simple project and it is snappy.  TSE was open in a second or so, but there were only three terminal strips.

[dougmcalexander]

I have projects with lots of terminals as well but with no problem. You may have to send to Autodesk for evaluation. It makes no sense at this point.

[sonny3g]

Are these custom terminals, ie, did you create new terminal symbols?  If so, did you put wipeouts on them, extra attributes or other "extras"? 

[drathak]

---

@sonny3g wrote:

Are these custom terminals, ie, did you create new terminal symbols?  If so, did you put wipeouts on them, extra attributes or other "extras"? 

---

Schematic symbols, no.  But the panel symbols are custom.  No wipeouts and only approved attributes.

EDIT: The schematic symbols were edited to change where certain attributes show up by default.  No other changes.

I have attached one of the panel symbols.  The only "extra" I can think of is that the WIRENO attributes have been made multiline capable.

[EE1337]

Yeah I think it must have been something strange, either in my AutoCAD program installation, or in my project scratch database.  I was getting unhandled exception errors when running automatic wire numbers, say after editing a terminal jumper, with autocad referencing some .dll.  I reinstalled Autocad, and rebuilt the scratch database, and it's much faster now.  About 5 seconds to draw up a typical terminal strip, which is acceptable - it doesn't seem to matter much how many terminals there are, mostly it's the number of connections across all the drawings, which makes sense to me.

I moved the catalog databases into vault, and running those off of the SSD doesn't make any noticeable difference in performance for us.  The bottleneck appears to be in processing the connections across all the drawings.  I've decided to simply run those directly off the network drive.

Symbol libraries would be nice to have Vaulted, but there's no good way to get them in there.  I could start a 'Symbols' project in Autocad Electrical, and just do it that way, but then every time I add a symbol I'd have to add it to that project and manually check it in to vault.