Not able to delete Extensible Storage schema

[snajjar]

I'm trying to put together a command to erase Extensible Storage (ES) data and the corresponding Schemas for our addin in case if unwanted by the users or corrupt. I had no luck so far. 

After the ES data has been written to the file it won't go away. I am being able to delete the DataStorage elements with no troubles, but the schemas keep popping up after erasing them.

The procedure I'm trying is:

1. Open the Revit file with the saved ES data.
2. Delete all the DataStorage elements holding the data.
3. Save and close Revit, reopen the file again.
4. Erase all schemas created by our addin.
5. Save and close Revit to make sure I clear the schemas in memory.
6. Start Revit again.
7. Use the Schema.ListSchemas() function with no documents open to make sure our custom schemas are not loaded, this clears out.
8. Open the file and use the Schema.ListSchemas() function, now all the schemas that I have erased previously reappear after opening the file!

I'm using the same procedure found in the "ExtensibleStorageUtility" code example in the Revit SDK in a macro (code below).

One observation I have that if I close Revit without saving after step 4 I get no warning to save the file as if no changes happened to the file, although I'm doing the erase inside a transaction.

This article has some info on ES and Schema behaviors that I tried to utilize but didn't work.

Below is the macro function I'm using:

		public void DeleteSchemas() {
			
			var uiDoc = this.ActiveUIDocument;
			var document = uiDoc.Document;
			
			var message = "";				
			IList<Schema> schemas = Schema.ListSchemas();
			int deleted = 0;
			int count = schemas.Count;
			
			using(Transaction tErase = new Transaction(document, "Erase EStorage")){
				
				tErase.Start();
				
				foreach (Schema schema in schemas.Where(s=> s.VendorId == "MYAD"))
				{
					//Note-this will delete storage of this schema in *all* open documents.
					try{
						document.EraseSchemaAndAllEntities(schema);
						Schema.EraseSchemaAndAllEntities(schema, true);
						deleted++;
					} catch {
						
					}
				}
				tErase.Commit();
			}
			
			message =string.Format( "{0} storage elements out of {1} were deleted.", deleted.ToString(), count.ToString());
			
			TaskDialog.Show("ExtensibleStorageUtility", message);
		}

Any light on this would be much appreciated.

Thanks!

[josephsWYW3U]

Awesome news! Thanks for your hard work!

[c_hanschen]

Hello,

Sorry, I posted this Message, but missed some replied to this original post.

MESSAGE 40 OF 42: Good news! The development team completed work on REVIT-203545 [Unable to delete extensible storage] and scheduled it for inclusion in an update release of Revit 2024.

Will it be solved for earlier versions of Revit as well??

I think I can Assume my problem is the same and already 'under attention'.

----------------

Following up on this issue:

Before Revit 2021:

Extensible Storage could be read by Autodesk.Revit.DB.ExtensibleStorage.Schema  (.ListSchemas) and Deleted (.EraseSchemaAndAllEntities)

All issues we had with conflicting Schema Data, see attached image, could be resolved by deleting Schema Data.

(conflicting Data between open en linked models)

After Revit 2020 (2011 and up) .

Command .EraseSchemaAndAllEntities is moved from Autodesk.Revit.DB.ExtensibleStorage.Schema to Autodesk.Revit.DB.Document, while Reading still has to be done by Autodesk.Revit.DB.ExtensibleStorage.Schema (.ListSchemas).

So this data is loaded from document (by opening document) to Autodesk.Revit.DB.ExtensibleStorage, so the conflict can this remain.

I have different projects in Revit 2023, I thought this problem would we solved by the (API) changes of Revit 2011, did not had this problem for a while, but it can still occur (and it does).

All Linked are unloaded:

No problem, so the conflict is between our model and Linked model, but not listed in .ListSchemas when links are unloaded.

Just reloading the 'problem link': 

Message occurs, see attached image.

Erase Schema data FAILS: "No Write Access to this Schema"

Conclusion:

Before R2021 we had this message very frequent, but we we always able to solve it!

Now we still have this issue, but we can not solve it.

Does anybody has a solution to this problem?

Thanks in Advance,

Chris Hanschen

LKSVDD architecten

The Netherlands

[jeremy_tammik]

Dear Chris,

Right you are, that needs to be taken into consideration.

<joke____more_or_less>

Yes, sure. All you have to do is present a sufficiently convincing business case:

This issue needs to be assessed by our engineering team and prioritised against all other outstanding change requests. Any information that you can provide to influence this assessment will help. Please provide the following where possible:

• Impact on your application and/or your development.
• The number of users affected.
• The potential revenue impact to you.
• The potential revenue impact to Autodesk.
• Realistic timescale over which a fix would help you.
• In the case of a request for a new feature or a feature enhancement, please also provide detailed Use cases for the workflows that this change would address.

This information is crucial. Our engineering team has limited resources and must focus their efforts on the highest impact items. We do understand that this will cause you delays and affect your development planning, and we appreciate your cooperation and patience.

</joke____more_or_less>

Hmm, honestly, might be a bit tricky. The development team is currently working on the development release that will probably make its way out into the wild in one, two or three years' time. Consequently, they are pretty busy and fully scheduled.

Still, I have asked them for suggestions on how to handle the situation in previous releases for you, and they have passed it on to the relevant experts.

I think the best way to handle extensible storage is, always has been, and will remain forever, to treat it as read-only.

That means: no modification, no deletion, no changing it in any way whatsoever.

In order for it to evolve, you define a new schema, completely new, completely unrelated to all previous ones.

You also implement and maintain code to update from all your previous schemata to whichever current form you prefer.

Can't you handle all your needs that way?

Cheers

Jeremy

[c_hanschen]

@jeremy_tammik 

"Can't you handle all your needs that way?" - Yes, I can, for all my Addins I have no problem at all.

The problem for me is, not the problem as a developer, but as user.

Revit keeps showing this message when opening a model, all team members of the project.

This needs to be addressed, I wrote a simple 'Extensible Storage schema Deletion Tool' to solve this problem for all the project members, but now it fails because the deletion fails.

If you (or any other Autodesk member) needs a project with with message when opening the project, I can Add you to a BIM 360 project.

Thanks in Advance!

gr. Chris

[jeremy_tammik]

Dear Chris,

Thank you for the explanation and clarification. I passed it on to the development team. They say:

> After the period of bake, we can target the next Revit 2024 update release to include this fix. I will reopen this bug with nominate process as soon as the first Revit 2024 update release has been published.

That will probably not address the issue you are having with problematic RVT files in Revit release 2022 and 2023. I have asked them for advice on how to handle those as well.

Cheers

Jeremy

[swfaust]

Adding to this, I have been all through this thread and can not find any way to successfully purge a bad schema.  In our case we were making some text standardization changes and updated documentation not realizing that this would be seen as a change to schema (I don't think it really should be but that's another discussion).  All that we did was add a period at the end of the documentation and it caused this.  Anyway, we are now trying to get a way for users to remove the 'bad' schema.  Ironically we aren't even using the schema anymore so I would really like to just create something to get rid of it ultimately.  I have the following simply command:

    [Transaction(TransactionMode.Manual)]
    public class CmdRepairSchema : IExternalCommand
    {
        public Result Execute(ExternalCommandData commandData, ref string message, ElementSet elements)
        {
            var doc = commandData.Application.ActiveUIDocument.Document;
            var id = new Guid("**My Schema ID**");
            var schema = Schema.Lookup(id);

            if (schema == null)
            {
                TaskDialog.Show("Schema Not Found", "The old schema was not found, no update is necessary.");
                return Result.Cancelled;
            }

            using (var t = new Transaction(doc, @"Repair Extensible Storage"))
            {
                t.Start();

                doc.EraseSchemaAndAllEntities(schema);
#if r2021 || r2022
                Schema.EraseSchemaAndAllEntities(schema, true);
#endif

                t.Commit();
            }

            TaskDialog.Show("Success", "The schema has been updated");

            return Result.Succeeded;
        }
    }

I also have a really basic 'List Schemas' command that just uses `Schema.ListSchemas` and puts them in a `TaskDialog`.

My steps are as follows:

1. Open Revit
2. Open Project (don't open any other projects so this is the only project open).  Note also that this is a testing file I am using that is just started from the default template and schema added and saved.  There is nothing else in it including any links of any kind.
3. Run the list command to make sure the unwanted schema is there (it is)
4. Run the fix command
5. Run a 'Purge Unused' and remove the schema by ID
6. Save the model
7. Restart Revit
8. Run the list command

Here are different variations of the fix command and process I have tried:

• As you know, the Schema.EraseSchemaAndAllEntities method doesn't exist after 2022 so it's only run on 2021 and 2022.  I have also tried without that line at all (so that we only call the document method in all versions).
• I have tried doing a standard save on the model as well as a "save as" to save it to a new location.
• I even tried removing the schema and then re-creating it with new and different documentation (that should match the older 'correct' version).

Some observations:

• After running the fix command WITH the call to the static method, the list command can be run and no longer shows the schema in the list.  HOWEVER, the schema DOES show up in the PURGE command when I do that, leading me to believe it's still in there somewhere.
• After running the fix command WITHOUT the call to the static method, the list command will still show the schema AND it shows up in the purge command.  After running the purge command it no longer shows up in purge but DOES still show up in the list command.
•  With the above, I tried in 2021 with the static command in place which removed it from the list command, then purged which removed it from the purge command.  At this point I can see no evidence of the schema in any of the API calls nor in the Revit purge dialog (or Revit lookup for that matter) which leads me to believe that it's actually gone.  However, I save, close, and restart Revit, open the model, and run the list command and it's there in the list again.
• When I tried recreating with the different documentation I then ran the list command in the model.  It showed the schema as existing but with the updated documentation so the change had been made.  However, I saved the model, closed and restarted Revit, opened the model, and ran the list command to find that it listed the schema with the bad documentation still.

I can see no other way that I can get rid of this.  We also tried this in 2021.1 (the earliest we need to support) and 2024.1 (the latest we need to support currently) with no difference.  Of course the static method tests were only done in the versions that support them but other than that it's the same in each.

Help...

[RPTHOMAS108]

What did you attach the schema to in the document i.e.

Project information singleton

Data storage element

Multiple elements (data storages or others)

Nested families

Does the document have links that are loaded that could potentially load the schema?

I have a feeling those static calls only really work if the schemas are not in conflict to begin with i.e. once they are in conflict it perhaps can't identify to remove them anyway. Perhaps that is not correct but from shared experiences those methods either do not work at all or only work in that case. Out of interest have you tried deleting a schema that isn't noted as in conflict?

There is a specific issue with 2024 so I would await the conclusion of that elsewhere before expending effort on that. Otherwise regarding 2024 I would probably create a whole new schema with new id even if there are no changes to the structure required.

It is easy to make the mistake with the documentation field, I made it in the past but it is what it is (part of the structure).

[swfaust]

Schema is attached to the project info object (started before data storage objects were an option).

No there are no other elements.  Per the post, this is just a project started from a built in template and added a 'messed up' schema for testing.  It does not have any links or even any families loaded other than what comes with the built in Revit template (which obviously don't have the messed up schema in them.

It's also not currently listed as in conflict.  This is intended to solve that issue, but for development and testing I only create the new blank model, add the 'bad schema' and then try to fix it.  So the 'bad' one is the only one in the project and the project is the only one open, thus no conflict warning. 

[RPTHOMAS108]

So it sounds like the methods don't work at all then, if you started a clean session without any conflicts occurring (that schema or others) between when you tried to delete the schema and when you opened Revit.

They say that purging is the option but that doesn't work either it seems.

There is also Element.DeleteEntity since you know it is on project information element. For that the schema would still exist in memory until Revit is restarted (if that method works).

[josephsWYW3U]

Try:

1. Run the Fix

2. Purge

3. Save

4. Run Fix Again

5. Puge

6. Save

7. Confirm Schema is gone

[swfaust]

Well... That seems to have corrupted the file.  I followed those steps and then restarted Revit to try to confirm it's gone (step 7) and now when I try to open the file Revit says 'Some data in file **my file path** needs to be recovered' followed by 'A serious error has occurred.  The current action has been cancelled.'

**UPDATE**  So it seems to have kind of worked.  I opened the model with an audit and it did open, though it said that Project Information was missing and would be replaced.  I allowed it to do that and it did open and I did verify that the schema was gone.  So yes it seemed to have gotten rid of the schema but gotten rid of project info as well in the process.  It seems recoverable but not exactly a good user experience "follow these 7 steps, it will corrupt your model, then you can audit and repair it and it's all fixed!".  Is there any way to put the project info back programmatically?

[josephsWYW3U]

Well that's not ideal. The project Information deletion could be tied to your fix, or potentially tied to the schema that was purged? Might be worth testing this process on a different data schema, say something that's loaded in through a family and isn't referencing project Information.

Else i could send on the purge program from the TraceIt creators. It made the schema visible and capable of being purged (though we had to run through the process above to fully delete it as it came back into the project via save).

[swfaust]

I'm sure it was related to the purge somehow.  This is an old program that started using ES before data storage objects were available so it was stored on the PI object.  I would guess that it's deleting that object when I purge the schema out.  This may work on another data schema but unfortunately that doesn't do me any good as this is our errant one that we need to get rid of.

If you have the TraceIt one it may be worth looking at but if you had to do the same process I'd guess it would come out with the same result.

[josephsWYW3U]

Could be, but might be worth a try. This is the most recent plugin I received from the UNIFI people. It's worked with the TraceIt and Enigma schema conflicts we encountered. It installs and just runs in the the background.

1. Install

2. Boot Revit & Tell it to run the add-in

3. Purge, the plugin should make the Schema visible

4. Save

5. Purge again

6. Save

7. Check purge, it should now be gone

8. Shut down, re-open and confirm

No promises. It's not my plugin, but it's been super helpful on my end. The only issue you may run into is if the schema is loading in from a link, or a family, at which point you need to track it down and purge it at the source. Easier said than done. Use at your own risk and all that.

https://www.dropbox.com/scl/fo/7tpsryvrfacw1m6318avq/h?rlkey=7qkidxjuem71v6uenu270cz2b&dl=0

[swfaust]

Great thank you!  I will try it and see what it will do.

[ivo.lafeber]

Same problem over here in revit 2023. 

When the schema's are erased they have to be purged from the model (done this manualy) , but still remain in memory. 

purging elements is possible for family and types, but there is no adviserrule for schema's :(. 

Matt Taylor / RevitPurgeUnused · GitLab

[jeremy_tammik]

If you open one single file, erase the schema, purge it from memory and shut down Revit, it is gone forever. It will never reappear unless you open some other model the still contains it and thus reintroduces it on opening.

Is that statement correct for you? If not, in which way is it flawed, please?

[ivo.lafeber]

the manual part of purging is where it goes wrong. I don't want to manualy purge every time a schema is erased. 

I Built a schema update checker, which compares the schema in the model with the schema in the software. so I have a schemaModel and a schemaSoftware, do the comparing and erase one of the two because only one needs te be active. but it cannot be erased completely by the api, so everytime before I close the model I need to purge the not active schema's. 

[jeremy_tammik]

Dear Jeff,

One statement you make caught my colleague's eye; you say:

> - especially, in my case, they may change multiple times during a session.

That makes them VERY concerned!

An extensible storage schema CANNOT CHANGE. Never, ever.

Please take heed of this important and repeatedly highlighted note from another recent discussion:

https://forums.autodesk.com/t5/revit-api-forum/extensible-storage-undo/m-p/12232911

> Please consider an extensible storage schema something that you create once and for all for all eternity. You NEVER modify it in any way whatsoever. You can delete it, but I would be careful with that as well. This has caused many complications and misunderstandings, so best to consider it undeletable as well. If you wish, go ahead and delete the data entities. Just leave the schema alone.

Is that understood?

Cheers

Jeremy

[JeffStuy]

And that is the exact issue.  Since we cannot edit / change the schema, the process must be to erase the current schema and provide an updated replacement schema. However, if we cannot erase schema, then we can only create new schema.  This means, all of the prior schema hangs around polluting the model.  If the schema is a onetime thing, it is next to useless – as you know, data changes, that is the nature of data.  If this data can never change and is always fixed, there really is almost no reason to store information within extensible storage system. 

However, your comment: “delete the data entries.  Just leave the schema alone.” Is interesting.  My understanding is that this is not possible.  That is, once schema and data is created, the data cannot be changed.  My understanding is the only way to update the data is to erase the schema and data and provide a new schema with new data.  Is this incorrect?  Additionally, could the schema be modified – can additional fields be added (I think not)?  Because that would be needed also.

Thanks,

Jeff