Extensible Storage "Security" Question

[CoderBoy]

I have a question about Extensible Storage security.

I've been learning about Extensible Storage and working with some test code, and see that when you define a Schema you can declare the security for it.  For example, the most secure settings seem to be:

schemaBuilder.SetVendorId(m_VendorId);

schemaBuilder.SetApplicationGUID(m_ApplicationGUID);

schemaBuilder.SetReadAccessLevel(AccessLevel.Application);
schemaBuilder.SetWriteAccessLevel(AccessLevel.Application);

On the surface, this looks great.  By providing only the Vendor ID and Application ID GUID that I know, I can prevent others from reading or overwriting the data I store into a Revit project.

But then what appears to be a hard truth comes crashing in:  The VendorId and AddInId values that Revit pays attention to at runtime seem to only come from the clear-text XML add-in file, which anyone can read and anyone can spoof in their own add-in file.

So, for example, if my internal GUID m_ApplicationGUID doesn't match what's in my add-in XML file, I get security errors when I try to create an entity for the schema.

What I want to do is to be able to set, through the API only, my own private ApplicationGUID which only comes from my code, perhaps from an encrypted data file I read.

I'm still examining the SDK sample code and doing searches online (including this forum), but I have yet to see anything which indicates there really is a truly secure way to store data in Extensible Storage.

My question:  Am I missing something?  Is there some way (that is NOT trivial to spoof) to achieve real security on my extensible storage data?  As nearly as I can tell, Autodesk went through some good work to set up secure storage, then blew it by making it trivial to bypass.

Thanks very much for any thoughts or pointers on this.

.

[Anonymous]

I think the only way you are going to get any k8nd of real security on extensible data is to store it as an encrypted xml string field. You then have the issue of securing your encryption key. Security is all about balancing the value of the item/information against to effort and risk required to steal it. Data itself is useless unless you know the structure behind it. Even if someone were to go to the effort of spoofing your vendor and appid, they'd still have a hell of a time doing anything meaningfull with it. In my opinion the built in security is good enough unless you are storing national secrets.

[CoderBoy]

Thanks for the idea.  I am aware you can encrypt the data being stored.  Securing the encryption key shouldn't be too difficult if the add-in code is well obfuscated.  I'm not concerned about others reading the information, as at this time I don't believe we plan to store anything particularly secret or proprietary.

I'm partially concerned about others possibly tampering with it (which would make our add-ins reputation for quality be poor, but for which encryption solves that concern), but much more concerned about others deleting it.

We have an add-in whose design essentially requires data be securely stored in the project such that outside code can't delete it.  I simply can't find a good, quality technical solution for that, and I've been looking for quite some time.  Sure, we could store a link to a service which holds the data, but the link itself can be deleted, so there's not much point.

I was initially so pleased to see there's a security engine associated with extensible storage, until I found out how ridiculously easy it was to bypass.

I find that interesting, given Autodesk's big stance on security (cloud storage, strongly encouraging digitally signing add-ins, etc.). But apparently for some reason extensible storage doesn't get the same degree of thoroughness for security.  Go figure.

Unless I'm missing something?

Presuming I'm not, I would like to submit a feature request:  

Add a new AccessLevel.PrivateKey enum value, with a new SchemaBuilder.SetPrivateKey(GUID privateKey) method, whose value can only be set during creation via the API and cannot be read by someone evaluating the schema.

Add an overload to the Entity constructors which take either the Schema, or GUID for the Schema, to have an additional GUID for the private key.

Add overloads to the Entity.Set methods which takes an additional GUID for the private key.  

Add overloads to the Entity.Get methods which take an additional GUID for the private key.

Ensure data stored can't be deleted without providing the private key, for example have an overload on the DataStorage.DeleteEntity method which takes a GUID for the private key.

This includes the (scary) Schema.EraseSchemaAndAllEntities method, which should also require the private key GUID if the schema to be deleted has either access level set to PrivateKey.

There may be other aspects to the Extensible Storage system I haven't learned yet that would be affected, so this may be more of a starter suggestion.

But something like that seems significantly more robust to me.

[jeremytammik · Accepted solution]

Dear CoderBoy,

Thank you for your very nicely worded query.

I love the way you put it, 'and then the truth comes crashing it'.

Well put and well observed.

I agree with Scott's realistic assessment and suggestion for a simple and effective workaround.

Thank you, Scott!

I also agree with your explanation of the need for a truly protected extensible storage data.

Thank you for your query. I am sorry to say that there is currently no API access to this functionality.

I have heard about this concern in the past, but I still did not find a wish list item for it yet, so I submitted the new one for you, CF-4503 [API wish: truly secure extensible storage e.g. AccessLevel.PrivateKey -- 11830802], as this issue requires exploration and possibly a modification to our software. Please make a note of this number for future reference.

You are welcome to request an update on the status of this issue or to provide us with additional information at any time quoting this wish list item number.

This issue is important to me. What can I do to help?

This issue needs to be assessed by our engineering team, and prioritised against all of the other outstanding change requests. Any information that you can provide to influence this assessment will help. Please provide the following where possible:

• Impact on your application and/or your development.
• The number of users affected.
• The potential revenue impact to you.
• The potential revenue impact to Autodesk.
• Realistic timescale over which a fix would help you.
• In the case of a request for a new feature or a feature enhancement, please also provide detailed Use cases for the workflows that this change would address.

This information is extremely important. Our engineering team have limited resources, and so must focus their efforts on the highest impact items. We do understand that this will cause you delays and affect your development planning, and we appreciate your cooperation and patience.

Best regards,

Jeremy

[CoderBoy]

Hi Jeremy.

Thank you so very much for creating the wishlist item for the Autodesk development team!

I am not at liberty to comment on specifically why we need this functionality (financial implications, number of users affected, specific use cases), because my development team sells our Revit add-ins, and we have significant competition.

However, one would hope that the need for having truly secure data storage would be fairly self-explanatory and self-justifying (heh, especially to Autodesk).  I am also hopeful that given that there already exists a security system in place for extensible storage, that extending that system would be a relatively low-risk change.  But obviously only the Autodesk developers, who know the implementation details, could make that call.

Obviously we'd like to see it be available as soon as reaonable, preferably not later than the next significant release (what I'll presumably call "Revit 2018") but we'd be excited to start using it whenever it becomes available, earlier or later than that.

Thank you so much again for your support.

.

[jeremytammik]

Dear CoderBoy,

Thank you for your update and appreciation.

I added your non-business case to the wish list item CF-4503 [API wish: truly secure extensible storage e.g. AccessLevel.PrivateKey -- 11830802] to make the developers aware of its importance.

I received some initial feedback from Miroslav Schonauer that you may or may not agree with:

One thing that I may suggest in the meantime is to use 'serialized string-encoded data' in invisible shared parameters as I elaborated at an AU session a year or two before Extensible Storage was introduced in the Revit API.

However, even that can in theory be deleted by other add-ins if they maliciously loop all bindings and unbind specific parameters.

To be honest, this is a bit like asking for the Revit API to prevent modification of RVTs on a hard disk that someone maliciously and repeatedly smashes with a mallet and then burns in a furnace 😉

Jokes aside, the storage can be made a bit more secure, but never 100%.

Best regards,

Jeremy

[Anonymous]

My concern would be the risk of a malicious or poorly coded add-in injecting copious amounts of un-deletable extensible storage data into the project database.

[jeremytammik]

Dear Scott,

Yes, there really always are two sides to everything, aren't there?

Cheers,

Jeremy

[CoderBoy]

Scott:

While that is a possibility, malicious or poorly written code can do MUCH more damage by simply deleting elements, or (worse) changing parameter values, which wouldn't be nearly as obvious for a human to see/catch.  For example, that 120V circuit gets changed to a 240V circuit.  That air handler's cooling capacity requirement or airflow rate or chilled water flow rate gets cut in half, etc.

Obviously Autodesk would always have the ability, via support, to remove "undeletable" extensible storage data from the database if this wishlist were implemented, though I admit that path would not be pleasant to follow.

But a serious question about that concern - even today where it could be deleted if an add-in developer knew how to spoof it - is:  How would anyone detect that this has happened?  And further, what constitutes "unreasoanbly large?"

A malicious code writer today could create 10,000 extensible storage objects, each of which is, say 3Kb in size.  None of them is excessively large, but there are a lot of them which in total may constitute excessively large.

So while yes, that's absolutely a valid concern, it's kind of always been that way due to its degree of difficulty to detect, and I question whether that concern should be enough to warrant not allowing an add-in developer to securely store the data their add-in needs in a project. 

.

[CoderBoy · Accepted solution]

Hey Jeremy.

Re: the idea of using invisible shared parameters is intriguing, though clearly not ideal and not at all secure solution.  More of a "security by obscurity" approach, which requires extra gyrations and manipluation of the data when being saved and reloaded, though that's not a big deal.  Extensible Storage is so much more powerful and flexible, and we'd like to use that because it was designed to do what we want to use it for.  It's just currently not secure.

While I can't imagine our data would be very large, do you know what a string parameter's maximum value length is?  If it's only, for example, 255 characters (unlikely?), that wouldn't work for us.

Also, if we added an invisible shared parameter, what element could we bind it to that would always be there?  We've had problems binding extensible storage to the Project Information "singleton" when it's in a workshared environment.  It can be checked out by another user, making it unwriteable.  In fact, some customers do that on purpose to keep their regular users from changing anything in Project Information.   Plus we've also seen a few cases where the Project Information object gets corrupt.

Thanks for any additional thoughts.

[jeremytammik]

Dear CoderBoy,

Thank you for your update and new questions.

I can answer both of them:

1. The string size is unlimited as far as Revit is concerned. Ask your computer vendor and performance monitor.   🙂

2. Revit provides dedicated DataStorage elements for holding extensible storage data that can be modified by users without disturbing other user's elements in a workshared environment.

You could try and see whether they have a category, or can be assigned one, so that you could also add shared parameters to them.

If not, you could create your own DirectShape elements. I think you can define what category they have, and use something obscure that is not used elsewhere.

Can shared parameters be bound using sub-categories, or only the main categories that you have no control over and cannot create yourself?

Cheers,

Jeremy

P.S. Sorry, my stupid mouse has a will of its own, and it clicked to accept your questions as a solution independently. I see no way to undo that action. Rather silly interface to enable and allow both mice and men to do things that cannot be undone.

[CoderBoy]

Hi Jeremy.

I ran a little experiment, and on a newly-created DataStorage object the Category returned null.

Further, the Category property appears to be read-only.

So sadly, it does not appear to be an option to bind a project shared parameter to a DataStorage object, which would at least have helped get around the worksharing problems.

Further, if using a DataStorage object outside of the extensible storage world (e.g. by not assigning a schema-specific entity to it and instead binding it's category to a hidden shared parameter), is there anything that prevents code from deleting the DataStorage element itself (e.g. Document.Delete(ElementId)) which would then delete the stored parameter data along with it?

Again, the notion of securely storing information that can't be deleted by other's code (the real goal here) cannot be achieved using invisible shared parameters.  But that's been true all along.

Another thought:  If this wishlist item is to be implemented, and a “private key” is used which prevents deletion, Revit will have to not only let the Entity not be deleted unless you provide the private key, but the object which contains the entity also not be deleted until the contained entity is deleted by the vendor's code.

A less than happy thought:  If the entity is attached to some other, visible entity the user can delete in the GUI (say, a door), they would have to get an error telling them they can’t delete it, which would be very, very bad.

The best workaround I can think of is that if a non DataStorage element has it’s entity set to an entity with a private key in the schema, the API throws an exception telling the developer they can’t do that.  In other words, if your schema has a private key, the entity can *only* be stored into a DataStorage object, which is not accessible via the GUI.

The Document.Delete(ElementId) method would have to not allow a DataStorage element to be deleted unless it either doesn’t have an Entity, or the Entity isn’t of a schema that uses a private key.  Either that, or the Document.Delete(Element.Id) method would need an overload that accepts a private key. 

Yeesh.  I wonder how many other complexities like this may arise.

I can certainly see this as something that would require some serious thought, given that the current security engine only applies to reading and writing values in an entity, and deleting can be done at any time and is not part of the current security engine at all.

. 

[Anonymous]

How about computing and adding a signature hash to your entities so that you can verify the data and inform the user if anything has been changed unexpectedly? If you are worried about a malicious add-in going the extra step of recomputing the hashes, you could salt them with a private key.

[CoderBoy]

Hi Scott.

For this app, our biggest concern is a 3rd party deleting the data.

Thanks for the idea, though.

.

[jeremytammik]

Dear CoderBoy,

Thank you for your update and ideas regarding element deletion.

I added those to the wish list item CF-4503 [API wish: truly secure extensible storage e.g. AccessLevel.PrivateKey -- 11830802] as well to ensure they are not lost or overlooked.

Cheers,

Jeremy

[jeremytammik]

Dear CoderBoy,

Here is the initial feedback on this from the development team:

I appreciate the back-and-forth described here. CoderBoy obviously has a good reason for wanting improved data security on the extensible storage data, but as you found in the conversation, it's definitely not simple. When EStorage was introduced we wanted to ensure that the owner of the model maintained control of their data in the model. So even with a secure solution that could not be read (short of duplicating/spoofing to be a different add-in) deleting the data was still possible, because the owner of the data might decide to stop using an add-in and want to clean the data, including data on any of their design elements, as a result.

At the time, we didn't have the DataStorage element. But there's still a potential problem with locking this element, if the user opts to no longer use the add-in that created the data.

The idea of a private key that is not the vendor id/application id combo might be more approachable, but deletion of the data may need to remain possible, unless there is another option we can come up with. It is complicated and would require a lot of attention. Currently this is not likely to become a priority for the near term as I don't believe we've ever had a developer feedback looking for this higher level of security.

Cheers,

Jeremy

[CoderBoy]

Hi Jeremy.

Thank you for the update.  I do appreciate the concern about project owners wanting to always be able to delete data.

For our add-in, we have a case where the project owners want to make sure no outsiders can ever delete the data.  For example, if they give copies of their Revit documents to outsiders, they don't want the outsiders to ever be able to delete this data.

Unfortunately, I can't say more than that in this public forum, but the idea is the "real" project owners don't want outsiders (contractors, for example) to be able to delete the data.

.

.