Cloud Worksharing Operations are Degraded - 9/10/2018

[Ian.Ceicys]

Revit Cloud Worksharing is currently degraded. Users with BIM 360 Document Management-based projects may experience difficulties with publishing and initiating collaboration. We are aware of this problem and are working to resolve this as quickly as possible. We will provide updates once we know more.

[Ian.Ceicys]

Revit Cloud Worksharing is currently degraded. At this time all publishing operations for BIM360 Team and BIM 360 Document Management are impacted. Users with BIM 360 Document Management-based projects may experience difficulties initiating collaboration. We are aware of this problem and are working to resolve this as quickly as possible. 

[paul.murphy]

Do we have any further information on the status of Revit Cloud Worksharing and time before resolution.  We have project teams currently coming online to use the service and want to ensure that we are able to provide quality advice to the teams on when they can expect performance to be stabilised.

[Ian.Ceicys]

@paul.murphyRevit Cloud Worksharing continues to be operational. We are noticing that the queue for publishing operations is processing very slowly. 

We are working on an updated ETA for service restoration but do not have one to share at this time. 

Multiple engineering teams are engaged and we are working to resolve this issue as quickly as possible. 

[Anonymous]

This is the second time in the last 4 days the service has had issues.  Our production team is working on a deadline and cannot continue to work in this manner.  Please fix this issue ASAP!!  This is really making our company reconsider using the cloud services due to its instability.

[simon.beaton]

Currently wait time of 63 minutes to open a model and its got itself to 75% open and now an unrecoverable error which has crashed Revit.

---

---

[Anonymous]

A further update on this issue would be appreciated. We continue to be delayed in our project deadlines. It is 9:36am in Sydney Australia and it is a critical time for co-ordination meetings.

[mori_hiroshi_ro]

Hi,Ian.Ceicys.

I told you that you have to realize your own C4R is excelent technology  again.

C4R datas are our asset. You must not stop or disgrade C4R even a second.

Do you recognize that data on C4R is very important to us?

Are you investigating why it happens frequently like this? I would like you to publish concretely what measures are taken.That way, the reliability of the C4R will also increase, and we will be able to use it more easily.

Our company places emphasis on C4R and is planning to make better use from now. I hope that Autodesk and us will develop as a good partner.

Thank you.

OBAYASHI Corp.

[Kevin.Short]

Some Revit Cloud Worksharing services are currently operating in a degraded state. All customers are currently experiencing publishing delays, and some users may not be able to initiate collaboration on new models. We understand that this has an impact on your work, and we have categorized this as a top severity issue and are urgently working to restore full operations. We will provide an update on the status when new information is available. If you need immediate assistance, please contact Autodesk for support.

[Anonymous]

I am aware! Since 5:30am this problem come up. @simon.beaton

---

@simon.beaton wrote:

Currently wait time of 63 minutes to open a model and its got itself to 75% open and now an unrecoverable error which has crashed Revit.

---

---

---

[paul.murphy]

@Ian.Ceicys @Kevin.Short, this outage has been running for more than 5 hrs now.   Can you please provide something more concrete to end users to explain what is happening and when the services are expected to return to normal (full) operations.    Retransmission of the original general message does not help us explain to either our staff or our clients why we are unable to provide services, as they expect, or complete design reviews etc, or when our teams will be back at full pace.   

[Anonymous]

We are throwing in the towel.  We need more information on what's going on and what's happening. The service is unusable for most of our users and our help desk is getting dozens of calls. We are now worried about data loss. Was this a hack of Autodesk? 

Fluor Corp. 

[adrian_worboys]

As long as it is ok for when the US get out of bed.......

[Ian.Ceicys]

@paul.murphy adrian.worboys  @Anonymous  Yes, still here. We hear you. Core engineering resources are actively engaged on fixing this issue. More to come soon. 

[Anonymous]

Will you be giving a detailed explanation as to why it took core engineering resources more than 6 hours to fix the issue.

In all seriousness, we need to evaluate whether there is forward risk to our continued use of this platform, and knowing how this occurred and what chances there are of it occurring again will need to factor in.

[Ian.Ceicys]

@Anonymous The Core Engineering team has been working on it non-stop, however the fix was complex. As with all incidents, we will post a summary once we have further information on the cause. 

[Ian.Ceicys]

@Anonymous 

Our engineering teams are working to resolve this issue as soon as possible and provide you with accurate information.

What we currently know: This is not a security issue.

We are experiencing issues with our Access Control Management platform database. This database, which manages access control policies for our data, hit a capacity limit.

We are working around the clock to fix it and will continue to provide updates as we work toward resolution of this issue. 

All requests to this table are queuing up and will begin to run again as soon as we resolve the issue and rebuild the database construct.

[Anonymous]

@Ian.Ceicys I disagree in regard of the problem been solved. We still facing the same problem, were 9hrs later I still don't have the system up running, were We have a major deadline to be derived ( 5 models) and I have around 40 modelers waiting to get the task done, massive impact and as per @paul.murphy request I need to provide an explanation to my management team and client.... problem still up and please change the status as been Solved.

Regards,

[Ian.Ceicys]

@Anonymous Yes, still here. I'm not saying the issue is resolved at this time as core engineering resources are continuing to be engaged on fixing this issue.

When we have resolution of the issue I will update this forum post and health.autodesk.com will be updated to be green. 

[Ian.Ceicys · Accepted solution]

Yes still here. We hear you. We are now seeing recovery of the service.

Customers that attempted to performed publish operations over the last 12 hours should resubmit their models to be published on the service.

Core engineering resources are actively continuing to be engaged on fixing this issue.