This is often an indication that other memory is corrupt.

[mdellunto]

Hi Everyone,

So I enabled Content indexing service and a few days later My engineers are telling me they can't always check in files and they get error messages saying that vault can't check the file in. Next time they try to check the file in it says the file is already there. Also vault is performing slowly and freezes sometimes. I will get a screenshot of the error and post it here next time it happens.

I checked the server logs and i see the following. The exception highlighted in red makes me worrie.

Any ideas what is going on here? Do i just need to reindex all the files or maybe defrag the database?

Also SQL(sqlservr.exe) is taking 4GB of memonry. Only started after turning on indexing.

I am also about to update to vault SP1 but want to clear these problems before doing so. Maybe Sp1 will fix this?

20/03/2013 9:52:09 AM *******************************************************************
Error: Soap Exception ( mesg-id = 634993699293103464 )
Exception: UnspecifiedSystemException [0] 
Stacktrace:
Exception(Inner): Exception has been thrown by the target of an invocation.
Stacktrace(Inner): 
Server stack trace: 
at System.Threading.Thread.InternalCrossContextCallback(Context ctx, IntPtr ctxID, Int32 appDomainID, InternalCrossContextDelegate ftnToCall, Object[] args)
at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(IMessage reqMsg)
Exception rethrown at [0]: 
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Connectivity.Document.Services.FolderService.AddFile(Int64 folderId, String filename, String comment, DateTime lastWrite, FileAssociationParam[] associations, BOM bom, Stream file, FileClassification fileClass, Boolean hidden)
at Connectivity.Web.Services.v17.DocumentService.AddFile(Int64 folderId, String fileName, String comment, DateTime lastWrite, FileAssociationParam[] associations, BOM bom, FileClassification fileClassification, Boolean hidden, ByteArray`1 fileContents)
Exception(Inner): Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Stacktrace(Inner): at Connectivity.ContentSourcePropertyProviders.Utils.DwgDatabase.InitializeHost()
at Connectivity.ContentSourcePropertyProviders.Utils.DwgDatabase.InitializeDatabase(String fileName, FileShare fileShare, Int32& errorStatus)
at Connectivity.ContentSourcePropertyProviders.Utils.DwgDatabase.InitializeDatabase(String fileName, FileShare fileShare)
at Connectivity.ContentSourcePropertyProviders.Providers.AutoCADProviderImpl.IsSupportedFile(String filePath)
at Connectivity.ContentSourceProperties.ContentSourcePropertyProviderMgr.GetProviderForFile(String filePath, Boolean requiresWriteAbility, List`1 defaultExts)
at Connectivity.Core.BusinessLogic.ContentSourceCache.GetContentSourcePropertyReadProvider(String[] providerSources, String sourceFile, Boolean& badFile)
at Connectivity.Core.BusinessLogic.EntityPropertyData.a()
at Connectivity.Core.BusinessLogic.PropertiesBL.AddProperties(EntityPropertyData entityPropData, Boolean allowWriteToSystemPdef, Boolean catchValueConversionExceptions)
at Connectivity.Core.Behaviors.BusinessLogic.AddPropertyEditEntities.Edit(EntityData[] entitiesToEdit, List`1 entPropDatas)
at Connectivity.Core.Behaviors.BusinessLogic.UserDefinedPropertyBehaviorClass.SetEntityBehaviors(EntityData[] entityDatas, List`1 behaviorIds, Boolean areNewEntities, Boolean adhereToBehCfg, List`1 entPropDatas)
at am.a(Boolean A_0, EditResults& A_1)
at Connectivity.Core.Behaviors.BusinessLogic.CategoryEditEntities.Edit(EntityData[] entitiesToEdit, List`1 entPropDatas)
at b3.a(IEditEntities A_0, EntityData[] A_1, List`1 A_2)
at Connectivity.Core.Behaviors.BusinessLogic.CategoryBehaviorClass.SetEntityBehaviors(EntityData[] entityDatas, List`1 behaviorIds, Boolean areNewEntities, Boolean adhereToBehCfg, List`1 entPropDatas)
at Connectivity.Core.Behaviors.BusinessLogic.BehaviorBL.SetEntityDefaultBehaviors(Entity entity, Boolean isNewEntity, EntityPropertyData entPropData)
at Connectivity.VaultPro.Document.BusinessLogic.Vault_FolderExtension._Post_AddFile(File& newFile, BOM bom)
20/03/2013 9:52:25 AM Acquired license(215350144) for username(Mike), userId(8), host(192.168.1.63).
20/03/2013 9:52:26 AM Acquired license(215350144) for username(Mike), userId(8), host(192.168.1.63).
20/03/2013 9:52:27 AM Begin executing event Autodesk.ECO.EmailNotification at 20/03/2013 9:52:27 AM
20/03/2013 9:52:27 AM End executing event Autodesk.ECO.EmailNotification at 20/03/2013 9:52:27 AM
20/03/2013 9:52:41 AM *******************************************************************

[mdellunto]

I would like to add this only happens with DWG file.

[minkd]

I'm pretty sure that problem was fixed in 2013 SP1 and 2012 SP2.

It had to do with the DWG iFilter not being thread-safe so problems just like what you are seeing in the log could happen when multiple files were being added by different clients at the same time.

-Dave

[mdellunto]

i installed SP 1 last night.

Everything seemed to work ok for the first 2 hours now everything is falling apart. I am going to try and disable content indexing. Problem only start happening once it was enabled. 

Right now users can't even navigate through vault with out it frezzing.

Also SQL is still taking over 3 GB of ram. 

And now i see errors like this.

Error: Soap Exception ( mesg-id = 634994579710151742 )
Exception: DatabaseDeadlock [231] 
Stacktrace: 
Server stack trace: 
   at Connectivity.Core.Database.TransactionContext.OnSqlException(SqlException e)
   at Connectivity.Core.Database.SqlAccess.ExecuteNonQueryInternal(SqlCommand cmd)
   at Connectivity.Core.Database.SqlAccess.ExecuteNonQuery(CommandType commandType, String commandText, Int32 commandTimeout, SqlParameter[] commandParameters)
   at Connectivity.Core.Behaviors.DataAccess.PropertyConstraints.UpdatePropertyCompliance(String entityClassName, Boolean historyManaged, List`1 entityIds, List`1 updateTypes, List`1 constraintFailures, List`1 equivFailures, Int64 propDefIdForUpdateComplVerProp)
   at Connectivity.Core.Behaviors.BusinessLogic.PropertyConstraintBL.CheckCompliance(List`1 compEntityDatas)
   at Connectivity.Core.Behaviors.BusinessLogic.PropertyConstraintBL.ExecutePropertyComplianceCheck(EntityData[] entities, Boolean updatePropComplianceVerValue)
   at Connectivity.VaultPro.Document.BusinessLogic.Vault_FolderExtension._Post_AddFile(File& newFile, BOM bom)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Connectivity.Document.Services.FolderService.AddFile(Int64 folderId, String filename, String comment, DateTime lastWrite, FileAssociationParam[] associations, BOM bom, Stream file, FileClassification fileClass, Boolean hidden)
   at Connectivity.Web.Services.v17.DocumentService.AddFile(Int64 folderId, String fileName, String comment, DateTime lastWrite, FileAssociationParam[] associations, BOM bom, FileClassification fileClassification, Boolean hidden, ByteArray`1 fileContents)

Exception(Inner): Transaction (Process ID 61) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Stacktrace(Inner):    at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Connectivity.Core.Database.SqlAccess.ExecuteNonQueryInternal(SqlCommand cmd)

[mdellunto]

So I have disabed content indxing and everything seems to be back to normal.

Did i miss something about enabling indxing?

Are there any steps that i need to follow before turing it on so it doesn't shut down my production and engineering department?

[minkd]

While database deadlocks can never completely be eliminated, we do actively try to minimize where they can occur. If you are seeing a lot of deadlocks, then I recommend opening a product support case.  To reproduce it they will likely need a dataset and an understanding of what users are trying to do when they occur.

-Dave

[mdellunto]

I am not so worred with the deadlocks. What worries me is the fact the turning on content indexing service crippled our vault.

Content indexing is something we must have but it can't impact the server to the point where it is not usesable. 

Our sql database is no long taking 3 gb of Ram and vault is acting like this never happened.

[minkd]

Based on what you have told me, it would appear that when you turn on full content indexing in your dataset, users adding files begin experiencing horrible performance or application freezes which coincide with deadlock errors.

IMHO, the best way to get a problem like that resolved is to open a product support case.

-Dave

[John_Kavusak]

Hi

I found a solution to this problem.  Please email me at john.kavusak@synergis.com and I can help you get this fixed.  So far my fix has worked for 6 customers.

Thanks

jk

[mariepl]

Hi

Would you care sharing your solution with the public? I'm curious to see it your solution is one I have already evaluated or not...

Thanks