on 1/24 one of our files started having problems so that no one could sync to central. Due to a deadline we resolved by creating a new central but the supposed problem was caused by corrupt families. We removed the 4 corrupted families and moved on. They weren't critical to the project but we weren't clear as to why these families were corrupt, we were directed to this article:
Cosmic rays? Phase of the moon? who knows. Random.
Friday (2/3) we started getting errors that users could not edit family parameters. Type or instance, nothing in a family could be adjusted without an error that the family needed to be reloaded. Initially we thought it was isolated to one or two families, further digging indicated it was any family we touched. Reverting to previous versions did not help, even going back a day. After 4 hours of troubleshooting I contact support again. This is the supposed fix:
That article indicates reloading one or two "bad" families. In our case it was 487. From what I can tell there are about 10 that AREN'T corrupt. When I called autodesk, the solution was to look at journal from latest file, collect a list of all corrupt families. Then find the last known good file, export all families as a source for reload. I've done this, it generates about 400 errors of dimensions being removed and constraints being destroyed. Along with a few complex family types (nested shared families) being deleted, despite being reloaded from sources. In finding the latest "good" file, I realized that this corruption, of over 400 families, happened within 5 hours from 2/2 at 8am to 2/2 at 1pm. But Autodesk swears it is random?
The original issue is now being reported by others through our local user group. And I think that burying critical info away in the journal during an audit a pretty poor way to notify users of a potential disaster brewing.
I've updated the ticket with autodesk but I'm adding my response here:
Attached you'll find two error reports from reloading first, just 120 families, then loading the full 480. Regarding file versions. The last known good version is from 2/2/2017 at 8am, the first known BAD version is from 2/2/2017 at 1:30pm. We had to stop working on the file at noon on 2/3/2017. Effectively we would have lost 12 hours in the office multiplied by 6-8 people had we just reverted to the previously known good version. Not acceptable.
What isn't resolved in this process is the fact that all the elevation tags became non-functional in the process. Due to the damage to families, reloading the pointer and the symbol did not work and the nested element in the elevation tag failed. This isn't in any error report. There are views associated with these elevation tag, but because of the damaged family problem they can't be shown. Updating the family with a new version does not resolve the issue. At this point we had to delete all elevations and start over. Luckily we are only in SD phase, but the presentation elevations are now gone.
Here's the problem. While it might seem minor to you this is a disaster to our productivity and we never know when the problem will come back. This is the second instance of this file having "random" corruption problems and that is the best answer we can get. From a duplicate of my post on Autodesk (at Revitforum.org) others have seen this same issue. It could be we did something we shouldn't (nest 124 chairs in a family). It could be because some of our users access the file via Citrix or RDP session. It could be there is a bug in the program. It could be a problem with the database change that happened in 2016 with regards to families. For it to happen often enough to warrant a tech note on both the missing elements and reloading families I would think it would be more of a concern than this.
I also find it surprising that users are now expected to run an audit and troll through their journal file to find such a catastrophic error. Not to mention assuming they know about the random tech note to begin with. And I mean that sincerely.
When we spoke you seemed to think that there would be hundreds more families than the 480 we initially identified as damaged that were fine. After going through the project over the last few days, that's absolutely false. Except for system families I can't find more than a handful of families that WEREN'T damaged. I'm actually concerned that the system families will begin to fail and I have no idea how I'll repair that problem.
That some sort of corruption like that can run through a file with no prompt of trouble by Revit is appalling. I've got hundreds of projects under my belt. I've been working on Revit since 2007. I've had files go bad and for the most part I've figured out why. And I avoid those things like the plague, even if Autodesk says they should work, mostly. But in those cases the file generated errors that let me know something was awry. This did not and the fact that it is considered acceptable for the file to crash like this is disappointing.
Hi TomAtSera,
The family corruption issue you are seeing is not so much “random” as it is “quiet” in your file – that is, it is difficult to detect until certain internal functions take place. You may already know about a family issue that resulted from work implemented in Revit 2015; the issue can sometimes cause small pieces of data in a family to be deleted during Sync with Central. I call the corruption “quiet” because, as long as the family is not forced to regenerate you won’t see any issues. So if the corruption happened to a family you haven’t worked with in a while, it can unfortunately go unnoticed for a long time – hence, seeming random.
The development team has worked very hard to investigate this issue and bring solutions to customers like yourself. As a former architect, I completely sympathize with how debilitating things like this can be to your productivity and workflow. Knowing the significant customer impact, the development teams has created checks in the Audit tool to help you quickly identify corruptions – journaling is one way we deliver information about corrupt families; in later versions of Revit you should see dialogs that point to which families need reloading so that the problem is more visible. If you plan on upgrading to more recent versions of Revit, you will see more direct support in situ.
Steps you can take going forward:
I hope this helps.
Jill
Jill,
I appreciate your thoughtful response but I have some problems with the proposed solutions.
While I will continue to provide updates this is one of those cases where I feel like a guinea pig or a beta tester. I do my best to ensure that new releases and updates are somewhat vetted once they are out, provide quality add-ins that I trust and promote best practices with regards to family content and general practices. But this is something completely out of my control and the solutions provided (i.e. reload every family in your project or just revert a day and a half worth of work) are unacceptable and frankly short sighted. This is the second time in a year that I've found some oddity that has no real solution except "beware". The other is a bug that wasn't fixed until 2017 release.
Last night on twitter, someone else chimed in that they had experienced the same problem with 2-3 jobs recently. Needing to reload 1200 families in one case. How is that sustainable? How is that not a problem big enough to warrant some real investigation instead of an effective ¯\_(ツ)_/¯ and "good luck" from support. While I realize they deal with lots of inane questions there appears to be no avenue for folks without named accounts (i.e. multi-million dollar contracts for licensing from Autodesk) but better than average Revit knowledge to get anything more than canned responses.
Again, I appreciate the time and effort. My frustration is with the process of repeatedly having to prove it isn't a simple, random, inconsequential issue, but something we in the trenches consider catastrophic and disruptive at a large scale.
Hi TomAtSera,
I appreciate your candor and I’m sorry your experience has been so frustrating. I’m working with a team digging into these very issues, so if you are experiencing family corruption trouble in 2016, we want to know about it. I’ll look into the support case you referenced above.
Regarding nested families: I should clarify one point. The list of families reported in the journal after Auditing should include nested families too. As long as you are reloading nested families into the project you should be covered. When asked if you want to replace nested families when reloading hosts, just say ok. Reloading nested families into host families before reloading into the project is a workflow to mediate crashes. If your file isn’t crashing when you are reloading families, you can skip this step. I mentioned it in my original response to be helpful, but I didn’t clarify the bit about crashes (which is obviously a critical point – sorry!).
At the risk of compounding your feeling of being a beta testing guinea pig, can I ask a few questions? Is this project an upgrade from 2015? Did you use any template files from 2015? Does your organization utilizes project files loaded with all your approved families as a library (RVT loaded with RFAs) or do you have a centralized network library (standalone RFA library)? How frequently do you upgrade your family libraries, in whatever capacity you organize them? Have you and/or your team developed specific family workflows? I.e. – are there any special gymnastics you need to employ so that your families work in a certain way?
FWIW, we had a very similar problem with a file back in November of 2014 (Autodesk support case ID: 10122233). All of the component families became corrupt in a way similar to the OP's issue. We also could not place or modify rooms in the model. I do not know if these two issues were related. The factory was able to fix the issue with the rooms, but we had to go back to a functioning copy of the file, export all of the loadable families, then reload all 393 of them into the "fixed" file.
We never found out what caused the problem, and I have been thankful that it has not happened to us again because it was frankly terrifying. I am suspicious that it was somehow caused by inserting a family downloaded from the internet.
We seem to have a similar problem going on in two of our Revit models, first happening on 1/31 and then happening a couple of times after that.
Currently one of our section tags are not working in existing views in one of the affected models. Replacing them with a new version has not resolved the issue in the existing views but they do on the other hand appear in new views (with the same view template applied).
Autodesk Customer Support has described this to us as "random behavior", and "...the corruption happens randomly and it happens to most users. In most cases, we can repair the drawing however it won't mean the file won't become corrupt again in the future."
This is in my opinion a very serious problem that needs immediate attention from Autodesk. Not being able to know if, when or why our models become corrupted again is totally unacceptable.
Jill, I'm going to reply to your questions with as much info as possible:
Is this project an upgrade from 2015? Nope started in 2016 r2 Update 6 maybe. Although it is possible that one or two users have accessed the file using 2016 without R2 being applied.
Did you use any template files from 2015?
Nope, but the template has been completely revamped over the last year. I basically started with a blank file, created a template and started building up with some previous content. Mostly my work has been cleaning up linetypes/patterns, hatch patterns, view types, view templates and filters. With a lot of work on correcting and solidifying naming conventions.
Does your organization utilizes project files loaded with all your approved families as a library (RVT loaded with RFAs) or do you have a centralized network library (standalone RFA library)? As much as I push for using some OOTB families and then UNIFI for our custom content the users will invariably pull from places like RevitCity (shudder). Internally they may be pulling from an older job, our approved libraries, or UNIFI.
How frequently do you upgrade your family libraries, in whatever capacity you organize them?
Unifi has an automated process for updating families to the appropriate version you are working in. Some of the families (before my time) may be as old as 2014 and upgraded through this process.
Have you and/or your team developed specific family workflows? I.e. – are there any special gymnastics you need to employ so that your families work in a certain way?
I don't think so. But perhaps an example would give me an idea of what "special gymnastics" look like? I'm of the school of making the families parametric, using constraints, using voids sparingly, avoiding dwg/skp imports, etc. And ideally avoiding In-Place modeling.
Some things that might be helpful for this discussion. The team is working with the massing environment to produce some really complex geometry. But mostly they are only utilizing the tools within that environment, along with those limitations. I don't think they are using any adaptive components or dynamo for that matter. These objects are definitely being altered on a daily basis, hence my frustration with pulling "originals".
As I mentioned previously, there are some bad practices. I found a family yesterday that had 124 chairs nested in it, all controlled with visibility parameters. While a bad idea, not something that is identified as a file crashing technique as far as I know. One user insists on pulling whatever she needs from anywhere. Be that RevitCity, a manufacturer or a random website.
Hope that helps.
@jsnyder wrote:
FWIW, we had a very similar problem with a file back in November of 2014 (Autodesk support case ID: 10122233). All of the component families became corrupt in a way similar to the OP's issue. We also could not place or modify rooms in the model. I do not know if these two issues were related. The factory was able to fix the issue with the rooms, but we had to go back to a functioning copy of the file, export all of the loadable families, then reload all 393 of them into the "fixed" file.
We never found out what caused the problem, and I have been thankful that it has not happened to us again because it was frankly terrifying. I am suspicious that it was somehow caused by inserting a family downloaded from the internet.
Hi Jsnyder – I’m sorry to hear about all the trouble you had with rooms in 2014, but it isn’t related to the family issues described above. What made you suspect the downloaded family as the culprit?
@elvarjo wrote:
We seem to have a similar problem going on in two of our Revit models, first happening on 1/31 and then happening a couple of times after that.
Currently one of our section tags are not working in existing views in one of the affected models. Replacing them with a new version has not resolved the issue in the existing views but they do on the other hand appear in new views (with the same view template applied).
Autodesk Customer Support has described this to us as "random behavior", and "...the corruption happens randomly and it happens to most users. In most cases, we can repair the drawing however it won't mean the file won't become corrupt again in the future."
This is in my opinion a very serious problem that needs immediate attention from Autodesk. Not being able to know if, when or why our models become corrupted again is totally unacceptable.
Hi Elvarjo – If the section tags are working in some views but not others, that is a different issue than the family issues originating this thread. If you were experiencing the same kind of corruption, you wouldn’t be able to work with the families at all. Hopefully Support can give you some guidance to move forward!
Jill: The section problem may be a different issue but we did also have the same problem as described above. Interestingly the section problem came up at the same time as the model corruption problem so that makes us think there may be a correlation between these two.
@coffeyj wrote:
@jsnyder wrote:FWIW, we had a very similar problem with a file back in November of 2014 (Autodesk support case ID: 10122233). All of the component families became corrupt in a way similar to the OP's issue. We also could not place or modify rooms in the model. I do not know if these two issues were related. The factory was able to fix the issue with the rooms, but we had to go back to a functioning copy of the file, export all of the loadable families, then reload all 393 of them into the "fixed" file.
We never found out what caused the problem, and I have been thankful that it has not happened to us again because it was frankly terrifying. I am suspicious that it was somehow caused by inserting a family downloaded from the internet.
Hi Jsnyder – I’m sorry to hear about all the trouble you had with rooms in 2014, but it isn’t related to the family issues described above. What made you suspect the downloaded family as the culprit?
Jill: Perhaps the problem we had with the rooms is a red herring, but I thought it was worth mentioning because clearly that project database was highly corrupt. I don't really believe in coincidences or random electrons corrupting project files. Something caused this, and based on the responses in this thread, it's not a completely isolated incident. People are having problems with family corruption and other bad things are happening in the files at the same time (rooms/sections/elevations).
There was only one family (#394) in our project that could not be reloaded because it was corrupt. It was clear that someone had downloaded that family from somewhere on the internet. That is the only information I have, so that is the basis of my hypothesis. If you have other ideas, I am anxious to hear them.
I think the real issue here is that we do not get much meaningful feedback when this type of thing happens. We all want it fixed, but we also would like to know what caused it in the first place so that we can make sure it doesn't happen again. When you are staring down a project deadline with hundreds of thousands of dollars on the line, this all ceases to be an intellectual exercise.
Jill,
I don't know if it is relevant but we are also using Revit Server on most of our projects. We have users that access this from northern California, and an external office both with their own accelerators. The configuration of those offices isn't easily explained but I wonder if that has anything to do with it. With that said, we have a project in 2015 (much much larger) that has none of these types of issues utilizing the same accelerators and connections.
As far as user hardware, we're running fairly decent machines. Typically 4.0ghz processors, 32gb ram, Quadro M2000 or graphic cards with 2-4gb of ram.
Jill,
I've pulled journal files from the day that the corruption supposedly occurred or started. I've updated my support ticket 12593720, if you are interested in that. Journals are from 9 users that synced that day.
Tom
Hi Jsnyder – I didn’t mean to imply that there wasn’t a problem with the 2014 family/room project you were telling us about. When I said that it was unrelated to this topic, I meant the type of family corruption discussed in this thread originated in 2015. There may have been a relationship between the bad family and the rooms in your project that somehow caused the corruptions you were seeing, but I couldn’t say what the matter was. To your point, I agree that making corruptions more visible as they are happening is extremely valuable in protecting customers from lost time and data. My team is currently working on ways to realize this in coming releases so that it will become less of an issue in the future.
As a matter of fact, knowing you use Revit Server is very valuable information.
I received the information from your contact in customer support and the team is parsing through the data. I hope to be able to share some insights on this thread soon. Again, I know that you and others are concerned about the corruption issues you have been experiencing, and we are taking this very seriously.
We are experiencing this exact same issue right now in the exact same build of Revit. However, the only difference is we are on C4R. Was this solved? We are working to upgrade to the more recent build, but any advice is appreciated. This is taking our entire team down right before a CD deadline. Let me know, thank you.
-Brendan Mullins
Stantec Architecture
brendan.mullins@stantec.com
Brendan,
The only resolution is to go through the steps mentioned. Here's a breakdown:
It isn't easy and isn't guaranteed. A few possible sources of the problem: Families from older versions added without proper upgrade (i.e. not using Audit on open for them), Leaving file open for extended period of time (days, overnight), not rebuilding central on a regular basis.
Beyond that Autodesk has been unable to provide any source of where the corruption comes from. Just be vigilant about maintenance and clean content.
Good luck,
Tom