using Ext. Storage inside DocumentSavingAs event handler causes elements missing

[kirill78zah]

Hello!
I encountered a very strange problem using Revit API 2018.

A short description of the problem is:
If the DocumentSavingAs event handler writes data to the Extensible Storage of the saving document

(the line of code of this type is "doc.ProjectInformation.SetEntity (entity)"),
then after that all the objects that will be created in this document will not be saved.

It also may cause Revit Error: Missing many elements.

More detailed description in the code below. It is code of entire testing project. There is only one class implementing IExternalApplication.

#region Namespaces
using System;
using System.Collections.Generic;
using Autodesk.Revit.ApplicationServices;
using Autodesk.Revit.Attributes;
using Autodesk.Revit.DB;
using Autodesk.Revit.DB.Events;
using Autodesk.Revit.DB.ExtensibleStorage;
using Autodesk.Revit.UI;
#endregion

namespace RevitTestAddin
{
    /// <summary>
    /// 2 use cases:
    /// 
    /// 1. Open existing document
    /// -> Create 2 or 3 random objects (I created beams)
    /// -> Save document
    /// -> Close document
    /// -> Open document
    /// -> Everything works fine
    /// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /// 
    /// 2. Create new document or open existing document
    /// -> Save document as to any directory
    /// -> Create 2 or 3 random objects
    /// -> Save document
    /// -> Close document
    /// -> Open document
    /// -> The created objects have disappeared !!!
    /// </summary>
    class App : IExternalApplication
    {
        public static Guid Guid { get; private set; }
        public static Schema Schema { get; private set; }

        /// <summary>
        /// Create Schema to store array of ElementId
        /// </summary>
        static App()
        {
            Guid = new Guid("d05893e6-c01e-44e9-8fbb-0d140a26015b");
            SchemaBuilder schemaBuilder = new SchemaBuilder(Guid);
            schemaBuilder.SetReadAccessLevel(AccessLevel.Public);
            schemaBuilder.SetWriteAccessLevel(AccessLevel.Public);

            schemaBuilder.AddArrayField("ObjectArr", typeof(ElementId));

            schemaBuilder.SetSchemaName("ObjectsSchema");

            Schema = schemaBuilder.Finish();
        }



        public Result OnStartup(UIControlledApplication a)
        {
            a.ControlledApplication.ApplicationInitialized += OnApplicationInitialized;
            return Result.Succeeded;
        }


        void OnApplicationInitialized(object sender, ApplicationInitializedEventArgs e)
        {
            Application app = sender as Application;


            //Add event handlers for DocumentSaving and DocumentSavingAs events
            //Both of event handlers are execute same method which create Entity and save it into ProjectInfo
            app.DocumentSaving += DocumentSaving_EventHandler;
            app.DocumentSavingAs += DocumentSavingAs_EventHandler;
        }



        public static void DocumentSaving_EventHandler(object sender, DocumentSavingEventArgs e)
        {
            Document doc = e.Document;
            SaveDocumentObjectIds(doc);
        }

        public static void DocumentSavingAs_EventHandler(object sender, DocumentSavingAsEventArgs e)
        {
            Document doc = e.Document;
            SaveDocumentObjectIds(doc);
        }




        /// <summary>
        /// Create Entity
        /// Save ids of all family instances in document into ProjectInfo
        /// </summary>
        /// <param name="doc"></param>
        private static void SaveDocumentObjectIds(Document doc)
        {
            if (!doc.IsFamilyDocument)
            {
                Entity entity = new Entity(Schema);
                IList<ElementId> objectArr = new List<ElementId>();
                FilteredElementCollector familyInstanceCollector = new FilteredElementCollector(doc);
                familyInstanceCollector.OfClass(typeof(FamilyInstance));
                foreach (Element elem in familyInstanceCollector)
                {
                    objectArr.Add(elem.Id);
                }
                entity.Set("ObjectArr", objectArr);

                if (entity.IsValid())
                {
                    using (Transaction trans = new Transaction(doc))
                    {
                        trans.Start("SaveDataIntoDocument");
                        doc.ProjectInformation.SetEntity(entity);
                        trans.Commit();
                    }
                }
            }

        }


        public Result OnShutdown(UIControlledApplication a)
        {
            return Result.Succeeded;
        }
    }
}

[jeremytammik]

Dear Kirill,

Thank you for your query.

That sounds bad.

Can you clarify a little bit more the difference between the two use cases listed in the comments?

You say it works in one case, but not the other.

Would it be clearer if you implement two macros or external commands, one that demonstrates the failure and one that does not?

Would it be clearer if you created the beams programmatically as well?

Otherwise, please describe the exact steps and the exact beam creation steps and absolutely every single step of any kind.

Can you please provide the exact steps and complete code to reproduce the problem, so I can pass it on to the development team for analysis?

http://thebuildingcoder.typepad.com/blog/about-the-author.html#1b

Thank you!

Cheers,

Jeremy

[kirill78zah]

The main difference between use cases is

in first one we are saving document without changing file name and everything works fine

and in second one we are saving document with a new file name and elements are missing.

This problem can be  reproduced manually but I coded two external command classes for each of two use cases. They contain detailed comments. Thus test project contain 3 classes: one external application (from first post) and two external commands (they are below). I attached test project to this post.

I work with Revit 2018 in Windows 10.

I hope everything is clear. Thank you, Jeremy.

#region Namespaces
using System;
using System.Collections.Generic;
using System.Diagnostics;
using Autodesk.Revit.ApplicationServices;
using Autodesk.Revit.Attributes;
using Autodesk.Revit.DB;
using Autodesk.Revit.DB.Structure;
using Autodesk.Revit.UI;
using Autodesk.Revit.UI.Selection;
using WinForms =  System.Windows.Forms;
#endregion

namespace RevitTestAddin
{
    /// <summary>
    /// Call this command in existing document, not new created document
    /// Document must have at least one beam family loaded (BuiltInCategory.OST_StructuralFraming)
    /// After command execution close document and open it another time
    /// 
    /// Everything looks fine
    /// </summary>
    [Transaction(TransactionMode.Manual)]
    public class Command1 : IExternalCommand
    {
        public Result Execute(
          ExternalCommandData commandData,
          ref string message,
          ElementSet elements)
        {
            UIApplication uiapp = commandData.Application;
            UIDocument uidoc = uiapp.ActiveUIDocument;
            Application app = uiapp.Application;
            Document doc = uidoc.Document;

            CreateBeamsAndSaveDocument(doc);

            return Result.Succeeded;
        }

        public static void CreateBeamsAndSaveDocument(Document doc)
        {
            //1. Create beams
            //Get any StructuralFraming FamilySymbol
            FilteredElementCollector beamSymbolsCollector = new FilteredElementCollector(doc);
            beamSymbolsCollector.OfCategory(BuiltInCategory.OST_StructuralFraming)
                .OfClass(typeof(FamilySymbol));
            FamilySymbol fs = beamSymbolsCollector.FirstElement() as FamilySymbol;
            //Get any Level
            FilteredElementCollector levelCollector = new FilteredElementCollector(doc);
            levelCollector.OfClass(typeof(Level));
            Level level = levelCollector.FirstElement() as Level;

            Line line1 = Line.CreateBound(XYZ.Zero, new XYZ(10, 10, 10));
            Line line2 = Line.CreateBound(new XYZ(10, 10, 10), new XYZ(10, 20, 30));
            Line line3 = Line.CreateBound(new XYZ(10, 20, 30), XYZ.Zero);
            using (Transaction tr = new Transaction(doc))
            {
                tr.Start("CreateBeams");

                //Activate symbol
                if (!fs.IsActive)
                {
                    fs.Activate();
                    doc.Regenerate();
                }

                doc.Create.NewFamilyInstance(line1, fs, level, StructuralType.Beam);
                doc.Create.NewFamilyInstance(line2, fs, level, StructuralType.Beam);
                doc.Create.NewFamilyInstance(line3, fs, level, StructuralType.Beam);
                tr.Commit();
            }

            //2. Save document
            doc.Save();
        }
    }
}

#region Namespaces
using System;
using System.Collections.Generic;
using System.Diagnostics;
using Autodesk.Revit.ApplicationServices;
using Autodesk.Revit.Attributes;
using Autodesk.Revit.DB;
using Autodesk.Revit.UI;
using Autodesk.Revit.UI.Selection;
#endregion

namespace RevitTestAddin
{
    /// <summary>
    /// Call this command in new created document which was never saved before.
    /// Document must have at least one beam family loaded (BuiltInCategory.OST_StructuralFraming)
    /// Document will be saved as "test.rvt" in MyDocuments folder
    /// After command execution close document and open it another time
    /// 
    /// Objects are missing
    /// </summary>
    [Transaction(TransactionMode.Manual)]
    public class Command2 : IExternalCommand
    {
        public Result Execute(
          ExternalCommandData commandData,
          ref string message,
          ElementSet elements)
        {
            UIApplication uiapp = commandData.Application;
            UIDocument uidoc = uiapp.ActiveUIDocument;
            Application app = uiapp.Application;
            Document doc = uidoc.Document;

            //Save document to MyDocuments
            string myDocumentsPath = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
            SaveAsOptions options = new SaveAsOptions() { OverwriteExistingFile = true };
            doc.SaveAs(myDocumentsPath+"\\test.rvt", options);

            Command1.CreateBeamsAndSaveDocument(doc);

            return Result.Succeeded;
        }
    }
}

[jeremytammik]

Dear Kirill,

Thank you for your report and reproducible case.

I logged the issue REVIT-127216 [adding Extensible Storage in DocumentSavingAs event handler causes missing elements -- 13917629] with our development team for this on your behalf as it requires further exploration and possibly a modification to our software. Please make a note of this number for future reference.

You are welcome to request an update on the status of this issue or to provide additional information on it at any time quoting this change request number.

This issue is important to me. What can I do to help?

This issue needs to be assessed by our engineering team and prioritised against all other outstanding change requests. Any information that you can provide to influence this assessment will help. Please provide the following where possible:

• Impact on your application and/or your development.
• The number of users affected.
• The potential revenue impact to you.
• The potential revenue impact to Autodesk.
• Realistic timescale over which a fix would help you.
• In the case of a request for a new feature or a feature enhancement, please also provide detailed Use cases for the workflows that this change would address.

This information is extremely important. Our engineering team have limited resources, and so must focus their efforts on the highest impact items. We do understand that this will cause you delays and affect your development planning, and we appreciate your cooperation and patience.

Best regards,

Jeremy

[kirill78zah]

Dear Jeremy,

Excuse me, could you clarify

How can I request status of this issue?

How can I provide information for prioritization?

I mean, should I send it by e-mail or somehow.

[jeremytammik]

Dear Kirill,

The status can only be queried by asking ADN technical services developer support, e.g., by adding a note to this thread.

You can provide a business case in the same manner.

If you wish to provide confidential information that you do not want to share in public, an email is fine.

Cheers,

Jeremy

[kirill78zah]

I noticed one more thing: the problem only occurs when we are saving document with a new name overwriting an existing file.
If before there was no file in this location, the problem does not arise.

[jeremytammik]

Dear Kirill,

Thank you for the important update.

I added it to REVIT-127216 [adding Extensible Storage in DocumentSavingAs event handler causes missing elements -- 13917629].

Cheers,

Jeremy

[kirill78zah · Accepted solution]

As a temporary solution

I decided not to allow the user to overwrite existing files in cases where before saving a project it is necessary to add Extensible Storage to ProjectInfo.

To do this, in the DocumentSavingAs event handler, I added a check of the following type:

if (File.Exists(e.PathName) && e.Cancellable)
{
     TaskDialog.Show("THE PROJECT IS NOT SAVED !!!",
     "To write a project with a name " + e.PathName +
     " you must manually delete or rename an existing file with this name.");
     e.Cancel();
     return;
}

This event is cancellable, except when it is raised during close of the application. This situation can only occure in new created document.

Therefore, I decided not to allow the user to perform actions that would lead to the need to save Extensible Storage if the active document has not yet been stored in any location. It can be checked in command code like this:

if (String.IsNullOrEmpty(doc.PathName))
{
      TaskDialog.Show("CANCELED",
      "Before executing this command, you must save the drawing!");
      return Result.Succeeded;
}

[RPTHOMAS108]

Was told to create a DataStorage element rather than attach information to the ProjectInformation singleton.

I think ExtensibleStorage came first and then soon after came the ability to create DataStorage elements so that people no longer had to use ProjectInformation element. If you think about it: every developer is attaching data to ProjectInformation which in a workshared environment (outside of DMU's) have the same issues as all other types of element i.e. element ownership.

If you've created a specific DataStorage element to add data to then this is something only your add-in intends to use. So element ownership issues only arise from two people using your add-in rather than two people using different add-ins or even one person using your add-in and the other changing the shared co-ordinate system or doing some other task that affects the ProjectInformation singleton. To go even further for certain design patterns you can create DataStorage elements specific for each user and therefore no element ownership issues.

Having said the above it still isn't acceptable for elements to go missing after saving, so we need to understand this.

[kirill78zah]

Thank you, Richard.

I did not know about DataStorage class. I read some information about it.

BUT

I tested the same use cases using DataStorage. As I see unfortunately using DataStorage is not a solution.  The same problem occurs (I test only in Revit 2018.2). 

You can take a look on my changed External application class below.

I attached entire testing project to this post again.

#region Namespaces
using System;
using System.Collections.Generic;
using Autodesk.Revit.ApplicationServices;
using Autodesk.Revit.Attributes;
using Autodesk.Revit.DB;
using Autodesk.Revit.DB.Events;
using Autodesk.Revit.DB.ExtensibleStorage;
using Autodesk.Revit.UI;
#endregion

namespace RevitTestAddin
{
    /// <summary>
    /// 2 use cases:
    /// 
    /// 1. Open existing document
    /// -> Create 2 or 3 random objects (I created beams)
    /// -> Save document
    /// -> Close document
    /// -> Open document
    /// -> Everything works fine
    /// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /// 
    /// 2. Create new document or open existing document
    /// -> Save document as to any directory overwriting some existig file
    /// -> Create 2 or 3 random objects
    /// -> Save document
    /// -> Close document
    /// -> Open document
    /// -> The created objects have disappeared !!!
    /// </summary>
    class App : IExternalApplication
    {
        public static Guid Guid { get; private set; }
        public static Schema Schema
        {
            get
            {
                Guid = new Guid("d05893e6-c01e-44e9-8fbb-0d140a26015b");
                Schema schema = Schema.Lookup(Guid);

                if (schema != null) return schema;

                SchemaBuilder schemaBuilder = new SchemaBuilder(Guid);
                schemaBuilder.SetReadAccessLevel(AccessLevel.Public);
                schemaBuilder.SetWriteAccessLevel(AccessLevel.Public);

                schemaBuilder.AddArrayField("ObjectArr", typeof(ElementId));

                schemaBuilder.SetSchemaName("ObjectsSchema");

                return schemaBuilder.Finish();
            }
        }


        public Result OnStartup(UIControlledApplication a)
        {
            a.ControlledApplication.ApplicationInitialized += OnApplicationInitialized;
            return Result.Succeeded;
        }


        void OnApplicationInitialized(object sender, ApplicationInitializedEventArgs e)
        {
            Application app = sender as Application;


            //Add event handlers for DocumentSaving and DocumentSavingAs events
            //Both of event handlers are execute same method which create Entity and save it in DataStorage
            app.DocumentSaving += DocumentSaving_EventHandler;
            app.DocumentSavingAs += DocumentSavingAs_EventHandler;

            //event handler for creating DataStorage
            app.DocumentCreated += DocumentCreated_EventHandler;

            //this event handler shows Message box with information about DataStorage if it exists
            app.DocumentOpened += DocumentOpened_EventHandler;
        }


        public static void DocumentOpened_EventHandler(object sender, DocumentOpenedEventArgs e)
        {
            Document doc = e.Document;
            DataStorage ds = GetMyDataStorage(doc);
            if (ds==null)
            {
                CreateMyDataStorage(doc);
            }
            else
            {
                Entity entity = ds.GetEntity(Schema);
                IList<ElementId> elementIdList = entity.Get<IList<ElementId>>("ObjectArr");

                TaskDialog.Show("DataStorageTest", "DataStorage in this document contains "+ elementIdList.Count + " ids");
            }
        }


        public static void DocumentCreated_EventHandler(object sender, DocumentCreatedEventArgs e)
        {
            Document doc = e.Document;
            CreateMyDataStorage(doc);
        }

        /// <summary>
        /// Create DataStorage corresponding to my Schema
        /// </summary>
        /// <param name="doc"></param>
        private static void CreateMyDataStorage(Document doc)
        {
            Entity entity = new Entity(Schema);
            using (Transaction t = new Transaction(doc, "Create DataStorage"))
            {
                t.Start();
                // Create data storage in new document
                DataStorage createdInfoStorage
                  = DataStorage.Create(doc);
                createdInfoStorage.SetEntity(entity);
                t.Commit();
            }
        }

        /// <summary>
        /// Find DataStorage which contains data corresponding to my Schema
        /// </summary>
        /// <param name="doc"></param>
        /// <returns></returns>
        private static DataStorage GetMyDataStorage(Document doc)
        {
            FilteredElementCollector collector = new FilteredElementCollector(doc);

            collector.OfClass(typeof(DataStorage));

            DataStorage dataStorage = null;
            foreach (DataStorage ds in collector)
            {
                Entity ent = ds.GetEntity(Schema);
                if (ent.IsValid())
                {
                    dataStorage = ds;
                    break;
                }
            }

            return dataStorage;
        }

        public static void DocumentSaving_EventHandler(object sender, DocumentSavingEventArgs e)
        {
            Document doc = e.Document;
            SaveDocumentObjectIds(doc);
        }

        public static void DocumentSavingAs_EventHandler(object sender, DocumentSavingAsEventArgs e)
        {

            Document doc = e.Document;
            SaveDocumentObjectIds(doc);

        }




        /// <summary>
        /// Create Entity
        /// Save ids of all family instances in DataStorage
        /// </summary>
        /// <param name="doc"></param>
        private static void SaveDocumentObjectIds(Document doc)
        {
            if (!doc.IsFamilyDocument)
            {
                Entity entity = new Entity(Schema);
                IList<ElementId> objectArr = new List<ElementId>();
                FilteredElementCollector familyInstanceCollector = new FilteredElementCollector(doc);
                familyInstanceCollector.OfClass(typeof(FamilyInstance));
                foreach (Element elem in familyInstanceCollector)
                {
                    objectArr.Add(elem.Id);
                }
                entity.Set("ObjectArr", objectArr);

                if (entity.IsValid())
                {
                    DataStorage dataStorage = GetMyDataStorage(doc);

                    if (dataStorage != null)
                    {
                        using (Transaction trans = new Transaction(doc))
                        {
                            trans.Start("SaveDataIntoDocument");
                            //doc.ProjectInformation.SetEntity(entity);
                            dataStorage.SetEntity(entity);
                            trans.Commit();
                        }
                    }


                }

            }

        }

        

        public Result OnShutdown(UIControlledApplication a)
        {
            return Result.Succeeded;
        }
    }
}

[jeremytammik]

Dear Kirill,

Thank you for the updates and further research.

I added your new notes and sample code to REVIT-127216 [adding Extensible Storage in DocumentSavingAs event handler causes missing elements -- 13917629] as well.

Cheers,

Jeremy