Revit Cloud Worksharing Forum
Welcome to Autodesk’s Revit Cloud Worksharing Forums. Share your knowledge, ask questions, and explore popular Revit Cloud Worksharing topics.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Unplanned Downtime - 7.20.2016

18 REPLIES 18
Reply
Message 1 of 19
KyleB_Autodesk
2526 Views, 18 Replies

Unplanned Downtime - 7.20.2016

All,

As many of you have noticed, we have had some degradations of Cloud Worksharing operations today, caused by an unplanned outage of the Autodesk Identity service.  Here's a running status of things:

 

SWC Volume Trend

C4R SWC Trend - 7.20.2016.png

9:18pm EST

After successfully deploying an updated build of our identity service, and a few hours of normal service operation, we are setting the service back to green on the Health Dashboard.  All Cloud Worksharing operations are back to normal, as measured by our service analytics.  We will report back after the retrospective has been carried out.

 

6:10pm EST

We continue to see normal operation of core cloud worksharing operations.  We will set the service health back to green once we have an "all clear" from our Identity Service team.

 

2:42pm EST

The Identity team has made changes that have resulted in normal operation over the past 75 minutes.  We will continue to monitor services, and hopefully be back to green soon.

 

1:22pm EST

The Autodesk Identity service continues to have intermittent degradations of service, causing disruption to cloud worksharing operations in C4R.

 

11:20am EST

Autodesk identity service has been restored, and Cloud Workshaing operations have resumed normal trends.  We will maintain a degraded status while Autodesk Identity service ensures root cause has been resolved.

 

10:28am EST

Autodesk Identity service has experienced another unplanned degradation, causing Cloud Worksharing operations to degrade.  Teams are actively working to restore service.

 

10:06am EST
Autodesk identity service has been restored, and Cloud Workshaing operations have resumed normal trends.  We will maintain a degraded status while Autodesk Identity service ensures root cause has been resolved.

 

9:50am EST
Autodesk Identity service has experienced another unplanned degradation, causing Cloud Worksharing operations to degrade.  Teams are actively working to restore service.

 

9:25am EST

Autodesk Identity service has been restoredand Cloud Workshaing operations have resumed normal trends.

 

8:45am EST

Autodesk Identity service is experiencing an unplanned downtime, causing Cloud Worksharing operations to degrade.  Teams are actively working to restore service.

 

-Kyle



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

18 REPLIES 18
Message 2 of 19
KALH
in reply to: KyleB_Autodesk

10: 50 am EST
after 20 min struggling we were able to access the models or sync.
A360 Services Early Adopter
Message 3 of 19
andre
in reply to: KyleB_Autodesk

the opening & syncing attempt in our office do not warrant the statistics suggested by the graphic. we can't open and can't sync.

Message 4 of 19
KyleB_Autodesk
in reply to: KALH


@KALH wrote:
10: 50 am EST
after 20 min struggling we were able to access the models or sync.

Yes, the Cloud Worksharing service is working normally when we can authenticate team members.  Given the above trend from our analytics system, your experience is what I'd expect.

 

-Kyle



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Message 5 of 19
KyleB_Autodesk
in reply to: andre


@Andre wrote:

the opening & syncing attempt in our office do not warrant the statistics suggested by the graphic. we can't open and can't sync.


Andre,

I would expect that behavior if you attempted to carry out the operation during one of the Identity degradations this morning.  But I'd expect success if you attempted during a period of operation.

 

-Kyle

 



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Message 6 of 19
andre
in reply to: KyleB_Autodesk

great meta-messaging. i love it when tech expert throw lingo at me.

bottom line: are you saying my entire team needs to check the website before they are allowed to sync?

 

Message 7 of 19
KyleB_Autodesk
in reply to: andre


@Andre wrote:

great meta-messaging. i love it when tech expert throw lingo at me.

bottom line: are you saying my entire team needs to check the website before they are allowed to sync?

 


Andre,

I don't believe this website has any impact on the ability of your team to carry out Cloud Worksharing operations.  We're being as transparent at possible on the current status of things.  As of right now, 11:24am EST, things are working normally.  Until we hear a root cause identified by our Identity team, and and a subsequent "all clear" from that team, we will continue to provide updates here.

 

-Kyle



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Message 8 of 19
andre
in reply to: KyleB_Autodesk

so, the currently expected user experience is "intermittent success"

Message 9 of 19
pastarm
in reply to: KyleB_Autodesk

Ours have been working OK since right after the 10:28 EST update. Been slower than normal, but working.
Message 10 of 19
KyleB_Autodesk
in reply to: andre


@Andre wrote:

so, the currently expected user experience is "intermittent success"


Correct.  Looking at the trend lately. it's more intermittent failure.



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Message 11 of 19
KALH
in reply to: KyleB_Autodesk

1:00 pm EST -

Last status: we are unable to reload latest / SWC / Relinquish on numerous of our projects on C4R. The User Profile sign-in is still an issue since we cannot save back any preferences or any information from that profile. We are crossing toes that everything will be back in order in few hours. Thanks for the updates Kyle.

A360 Services Early Adopter
Message 12 of 19
KyleB_Autodesk
in reply to: KALH


@KALH wrote:

1:00 pm EST -

Last status: we are unable to reload latest / SWC / Relinquish on numerous of our projects on C4R. The User Profile sign-in is still an issue since we cannot save back any preferences or any information from that profile. We are crossing toes that everything will be back in order in few hours. Thanks for the updates Kyle.


Kal,

Yes, we had a sustained degradation of Identity from 1-1:10pm EST.  The the updated chart in my original post reflects that.

 

-Kyle



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Message 13 of 19
hjinirio
in reply to: KALH

We are continuing to experience issues where the "Operation cannot be completed" prompt is being shown to our users. In addition to this we are seeing "An error occurred while contacting the central model which prevents the local model from being opened." and "Requested service not available". We have had some success syncing from time to time but are losing productivity.

Message 14 of 19
hhbim
in reply to: hjinirio

we've been getting the 'Loading...' message for about an hour now...

 

c4r-loading.JPG

 

 

Message 15 of 19
andre
in reply to: hhbim

It seems we're having higher success rates when first reloading latest and then syncing.

This might be more of a "felt" increase that a measurable increase, but we're desperate and rolling with it.

Message 16 of 19
KyleB_Autodesk
in reply to: hhbim


@mduran wrote:

we've been getting the 'Loading...' message for about an hour now...

 

c4r-loading.JPG

 

 


Is it doing this if you cancel out of the File Browser, and then try again?  Judging by recent metrics things are back online.

 

-Kyle



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Message 17 of 19
hhbim
in reply to: KyleB_Autodesk

Thanks Kyle, after rebooting my machine etc...i'm able to log back-on successfully.

also, i'm hearing most of our team members are able to get back-on now as well.

 

thanks,

-md

 

 

 

Message 18 of 19
KALH
in reply to: KyleB_Autodesk

4:40 pm EST

Everything looks back to normal. 

Status from our side is back to green. We are able to make changes to our Autodesk ID's - A360 Hub is back to normal too.

We are able to SWC, open new session, reload latest, editing requests and finally "relinquish mine". All the above were not possible 4 hours ago. By exactly 3:04 pm EST all our operations are back to normal. Thank you.

A360 Services Early Adopter
Message 19 of 19
KyleB_Autodesk
in reply to: KALH


@KALH wrote:

4:40 pm EST

Everything looks back to normal. 

Status from our side is back to green. We are able to make changes to our Autodesk ID's - A360 Hub is back to normal too.

We are able to SWC, open new session, reload latest, editing requests and finally "relinquish mine". All the above were not possible 4 hours ago. By exactly 3:04 pm EST all our operations are back to normal. Thank you.


Kal,

That's what we're seeing on our end.  We are still waiting on the "all clear" from our Identity Service team before we set things back to green.

 

-Kyle



Kyle Bernhardt
Director
Building Design Strategy
Autodesk, Inc.

Can't find what you're looking for? Ask the community or share your knowledge.

Post to forums  

Autodesk DevCon in Munich May 28-29th


Autodesk Design & Make Report