Not able to delete Extensible Storage schema

[snajjar]

I'm trying to put together a command to erase Extensible Storage (ES) data and the corresponding Schemas for our addin in case if unwanted by the users or corrupt. I had no luck so far. 

After the ES data has been written to the file it won't go away. I am being able to delete the DataStorage elements with no troubles, but the schemas keep popping up after erasing them.

The procedure I'm trying is:

1. Open the Revit file with the saved ES data.
2. Delete all the DataStorage elements holding the data.
3. Save and close Revit, reopen the file again.
4. Erase all schemas created by our addin.
5. Save and close Revit to make sure I clear the schemas in memory.
6. Start Revit again.
7. Use the Schema.ListSchemas() function with no documents open to make sure our custom schemas are not loaded, this clears out.
8. Open the file and use the Schema.ListSchemas() function, now all the schemas that I have erased previously reappear after opening the file!

I'm using the same procedure found in the "ExtensibleStorageUtility" code example in the Revit SDK in a macro (code below).

One observation I have that if I close Revit without saving after step 4 I get no warning to save the file as if no changes happened to the file, although I'm doing the erase inside a transaction.

This article has some info on ES and Schema behaviors that I tried to utilize but didn't work.

Below is the macro function I'm using:

		public void DeleteSchemas() {
			
			var uiDoc = this.ActiveUIDocument;
			var document = uiDoc.Document;
			
			var message = "";				
			IList<Schema> schemas = Schema.ListSchemas();
			int deleted = 0;
			int count = schemas.Count;
			
			using(Transaction tErase = new Transaction(document, "Erase EStorage")){
				
				tErase.Start();
				
				foreach (Schema schema in schemas.Where(s=> s.VendorId == "MYAD"))
				{
					//Note-this will delete storage of this schema in *all* open documents.
					try{
						document.EraseSchemaAndAllEntities(schema);
						Schema.EraseSchemaAndAllEntities(schema, true);
						deleted++;
					} catch {
						
					}
				}
				tErase.Commit();
			}
			
			message =string.Format( "{0} storage elements out of {1} were deleted.", deleted.ToString(), count.ToString());
			
			TaskDialog.Show("ExtensibleStorageUtility", message);
		}

Any light on this would be much appreciated.

Thanks!

[guillain.jolivet]

How is your schema created? Maybe your add-in create it when it's loaded?

[jeremy_tammik]

I would avoid the try/catch with an empty catch-all. Using that, you will never notice if anything goes wrong. Never catch all exceptions:

https://thebuildingcoder.typepad.com/blog/2017/05/prompt-cancel-throws-exception-in-revit-2018.html#...

Here is an old article by The Building Coder on erasing extensible storage:

https://thebuildingcoder.typepad.com/blog/2013/11/erasing-extensible-storage-with-linked-files.html

If the problem persists, I would suggest putting together a minimal reproducible case for deeper analysis:

https://thebuildingcoder.typepad.com/blog/about-the-author.html#1b

[snajjar]

No, the schema is created on demand.

[snajjar]

Thank you for the swift reply @jeremy_tammik 

I have already looked at all the Schema related posts in the building coder blog, none helped solving the issue in hand.

After further investigation, it turned out that the Schema.EraseSchemaAndAllEntities(...) function is working fine in Revit 2019, it is in Revit 2022 that it is failing. I haven't tried the in-between versions but I suspect that this is the case for 2021 as well as it is in that version that the API change happened of deprecating this function from the Schema class and providing one in the Document class.

I have attached 2 Revit files with embedded macros for versions 2019 and 2022, both files have the same code except that the 2022 file is using the new Document.EraseSchemaAndAllEntities(...) function along with the deprecated one.

To replicate the issue:

1. Open file es test-V22.rvt in Revit 2022
2. Run the embedded macro ListSchemas to examine the OOTB existing schemas
3. Run the embedded macro WriteESData to write sample extensible storage data
4. repeat step #2, you should see a new entry of MYID:  SomeData
5. Save the file
6. Run the embedded macro DeleteSchemas
7. Repeat step #2, you should see that the new entry is gone
8. Save the file
9. Close Revit 2022 (Just to make sure that the schemas are cleared from memory)
10. Open Revit and the file again
11. Repeat step #2, the MYID:  SomeData schema reappears again!

If you follow the same steps on the file es test-V19.rvt in Revit 2019 the deletion will be successful.

I hope I am doing something silly, but I really think that it is an issue with the API update.
Please let me know if you need further info on the test case.

Thanks,

Sam

[jeremy_tammik]

Dear Sam, 

Sorry for not following up immediately and thank you for pointing out this again.

@RPTHOMAS108  just discussed and resolved a similar issue in another thread here in the forum, and I am pretty sure that addresses the same root cause. His explanations will hopefully help resolve your problem as well:

https://thebuildingcoder.typepad.com/blog/2021/11/new-analytical-model-api.html#6

I very much hope this helps.

Cheers,

Jeremy

[snajjar]

Thank you for your reply @jeremy_tammik, I much appreciate all the help you provide.

I read through the new blog post and the solution by @RPTHOMAS108, and I'm sorry to say that it does not address the issue I'm having.

The solution and guidelines are addressing DataStorag and Schema creation and retrieval, I don't have issues there, and I can say that we are following the proposed best practices mentioned in the blog post for the creation procedures:

• Schemas are being created on-demand
• Using new Entity(Schema) instead of new Entity(GUID)
• Not passing entities ByRef
• Utilizing ExtensibleStorageFilter and Schema.Lookup(SchemaGuid).

My issue is with deleting the schema, which is working fine in 2019 but not in 2022.

I am testing this in freshly created Revit files which I have included in my previous response with embedded macro code. The same code is being used in both files, but the 2022 one is not working.

The reason we are seeking Schema deletion is that when a Revit file that contains a newer version of the schema (adding more fields for example) and it has a linked Revit file with an older version, Revit will display a warning when the file is opened. This warning dialog is hindering our client's automation software that opens Revit files and process them automatically. We don't have control over that software nor want to suppress this warning because, while we are sure that this will not affect the tools we are developing and the integrity of the file, we can't guarantee that for all other vendors and the suppression will hide all such warnings for data coming from all the installed addins. Our goal is to provide a tool to delete our schema from the linked files to resolve the issue.

I really hope this can be looked at by the development team to see if it is a bug caused by the API update that took place in Revit 2021.

Thank you again for all your help!

Sam

[RPTHOMAS108]

I've not looked into this it should be possible to delete the schema if it is not used regardless, so that does seem wrong (especially if it previously worked). Did you have schema conflicts in the 2019 version though? 

However one thing that occurred to me as I read your latest message is that you should only get that warning in the first place if you've done something wrong in terms of managing the schemas. When you create a schema with a GUID that is that version forever. To add additional members you should create a new version and transfer the data (leave the old one alone not reuse it's GUID).

[jeremy_tammik]

Oh wow, well spotted!

Very sorry I missed that, Sam. You say:

> a newer version of the schema (adding more fields for example) 

No such thing exists!

If you have created such a situation, you are in serious trouble.

Please refer to numerous articles by The Building Coder pointing out that an existing schema cannot be modified:

• https://thebuildingcoder.typepad.com/blog/2013/08/deleting-and-updating-extensible-storage-schema.ht...
• https://thebuildingcoder.typepad.com/blog/about-the-author.html#5.23

[RPTHOMAS108]

Hello @jeremy_tammik 

I followed the procedure above (2022 version only) and was also not able to remove the schema, so regardless of misuse something seemingly warranting some further investigation with the EraseSchemaAndAllEntities methods perhaps. i.e. this is a clean file to start with presumably.

Note that the macro doesn't delete the DataStorage element but I deleted this manually prior to step 6. However I still wasn't able to permanently remove the schema.

I note that the old method (Schema.EraseSchemaAndAllEntities) has a boolean overrideWriteAccessWithUserPermission which is not present in the new Document method of the same name.

This may be a conceptual choice going forward but may be part of the issue (changes to underlying method that serves both). Also the test was only conducted within the macro environment (add-in id is explicitly set in the attribute).

I made the below minor change so any exceptions would show up but still no exceptions were reported:

[snajjar]

Thank you for the follow-up @RPTHOMAS108 and @jeremy_tammik,

Your observation is spot-on, my usual procedure is to change the guid if I ever change something in the schema so a new schema would be created, but it looks like I missed doing that in one class, and this is why I am seeking the deletion functionality. This should not be an issue for future updates, but I need to provide a resolution for the models edited by the current version.

And again, the issue with not being able to delete the schema is irrelevant to what I did wrong.

Appreciate the help!

[jeremy_tammik]

OK, so it seems I have to pass this on to the development team for further analysis. Are the two sample files that you shared above in a fit state to be passed on to them, or is there anything that can be simplified or cleaned up before I go ahead? Thank you!

[snajjar]

The files are in their simplest forms, they are freshly made out of the architectural template and contain no elements, just the embedded macro scripts.

[jeremy_tammik]

Happy New Year!

Thank you for your confirmation and patience. Thanks also to Richard for pointing out this existing thread in the new one raising a similar question:

https://forums.autodesk.com/t5/revit-api-forum/error-with-new-api-schema-revit/m-p/10858918

I have passed on both issues to the development team for further analysis and hopefully some helpful advice.

[jeremy_tammik]

The devteam replied:

After running DeleteSchemas macro, open Manage > Purge Unused. In the tree, select Extensible Storage Schema.
Check the schema a9dc2b48 and click OK to purge it. Run ListSchemas - the schema is purged. So, please use Purge Unused to delete schemas without entities.

I pointed out that this question comes from add-in developers automating processes and asked how they should run the purge command programmatically. We'll see what they come up with next.

[jeremy_tammik]

A similar issue came up again with a new customer, and the devteam underline:

I would strongly recommend to any developer not to reuse a GUID for a new or modified schema for any reason, even if you think it has been purged from one document it may not be purged from other documents and you are setting yourselves up for a potential in-memory conflict.

We have seen this many times from developers who copied and did not modify the GUID in our samples.

[2537747694]

Hi, have you found a solution to your problem? I probably had the same problem

[Songohu_85]

Very recently I faced the same issue, that for some projects I got InternalException while trying to remove ExtensibleStorage schemas. The exception does not occur after I unloaded IFC links. 

[jeremy_tammik]

Wow. Thank you for a very interesting observation. It would be nice (and very useful) if the other parties concerned could test and hopefully verify this solution.

[Songohu_85]

I created a very small example with IFC link (link with a few walls created in Revit2020).
In the file you will find 4 macros:
1. To create example schema

2. To remove example schema

3. To list all schemas

4. To remove all schemas
In Revit2020 I am not able to remove any schemas when the IFC link is loaded.

Update: InternalException when removing schemas occurs also when there is more than one project file opened (Revit 2020)

Kind regards
Marek