RESOLVED - Fusion 360 Service Outage - 9/10/18

[brianrepp]

All - as most of you have hopefully already seen either through the in-product notification, or our Health Dashboard, we had to take Fusion 360 offline due to a service issue that started around 11:30am PT.

Teams have been actively working to resolve the issue, but I do not have an ETA at this time.  As soon as I have more information, I will update this forum thread, along with our Health Dashboard.

I understand this is disruptive, and appreciate your continued patience as we work to get you back up and running.

UPDATE as of 10:00pm PT - Thank you for your continued patience!  We have resolved the issue and are taking Fusion 360 back online now.  Please note that it may take up to 20 minutes for your session to reconnect.  If you run into any delays or would like to speed up the process, you can also restart Fusion 360.  Please note, as we go back online, there may be some delays in processing due to the backlog.

What happened & how will we prevent it in the future?  

The recent issue with our Access Control Management service impacted customers’ access to certain workflows within several Autodesk applications. We understand these issues are disruptive to your business and apologize for any inconvenience.  All services were fully operational by 11:30pm PT, September 10 and we do not expect any further disruptions related to this issue. 

What happened: We experienced an issue with the capacity scaling limit configuration for our Access Control Management database. This database manages access control policies for data stored by Autodesk products.  This was not a security issue. 

  

Mitigation: The issue was fully resolved with a code fix and we are reviewing all our production databases for any similar issues as a preemptive measure to avoid any further customer disruptions. 

[brianrepp]

Update as of 2:45pm PT, we have found the root cause of the issue and are working to restore the impacted services.  I will report back by 4pm PT with an update on the status.

[brianrepp]

Update 4:08pm PT - the teams are getting close to finalizing the fix and will then be working through testing before releasing.  I still do not have an ETA, but I would expect this could still take up to a few hours to make sure everything works as expected before releasing to the wild.

[Aly-Tek]

Thanks for the Updates Brian. FYI, I also tried to log into the Health page to get notifications but it too is giving me an error not sure if this is purely because my account is for Fusion 360 or not. Lets hope it's sorted soon 🙂 Cheers again 😉

[brianrepp]

---

@Aly-Tek wrote:

Thanks for the Updates Brian. FYI, I also tried to log into the Health page to get notifications but it too is giving me an error not sure if this is purely because my account is for Fusion 360 or not. Lets hope it's sorted soon 🙂 Cheers again 😉

---

You bet.  I believe the Health Dashboard uses single sign-on, so you should be able to login with the same user id/password as what you use for F360 (and these forums).  Do you mind trying again and sharing the error message you're seeing?  Feel free to send to me via private message.

[Aly-Tek]

This is what comes up

[brianrepp]

Thanks for sharing, very odd.  I've passed along to the devs that run the Health Dashboard site, and will circle back via private message or email with an update.

[Rigid_Concepts]

Thanks for posting updates. 

Do you have any other updates/information.  I'm anxiously waiting to have access to all of my files again.

[brianrepp]

Update 7:30pm PT - it's looking promising, but work still being done along with significant testing before deploying the fix.  While I still dont have a firm ETA, it could still be a few more hours.

[brianrepp]

@Aly-Tek - I believe we have fixed the issue keeping you from logging in.  Do you mind trying again?

Also curious, did you by chance change your Autodesk username recently?

[Aly-Tek]

That fixed the login for the Health Dashboard, Cheers

As for the Username, My son has an educational account and I have a subscription purchased over a year ago.

---

@brianrepp wrote:

@Aly-Tek - I believe we have fixed the issue keeping you from logging in.  Do you mind trying again?

Also curious, did you by chance change your Autodesk username recently?

---

[Rigid_Concepts]

I've been thinking about switching all of our engineering over to fusion from Creo.  I've been hesitant to do so because the cloud isn't always there.  We do use fusion for all of our CNC machining toolpaths and it is absolutely amazing for that.

The problem with today is that we can't access files that are over two weeks old (new designs completed in Fusion).  Can you make the local storage keep the last 30 days of work?  Even if you have a down time like today, if we had access to 30 days of data we wouldn't complain too much.  

The problem at this moment is that we have to get some new motorcycle parts machined and to powdercoat on Wednesday.  The design is brand new and we need to make a couple changes and also need to access our parts to revise our CNC programs.  We can't do either today because the data isn't there, so I have machines just sitting waiting on the cloud.

[brianrepp]

---

@Rigid_Concepts wrote:

I've been thinking about switching all of our engineering over to fusion from Creo.  I've been hesitant to do so because the cloud isn't always there.  We do use fusion for all of our CNC machining toolpaths and it is absolutely amazing for that.

The problem with today is that we can't access files that are over two weeks old (new designs completed in Fusion).  Can you make the local storage keep the last 30 days of work?  Even if you have a down time like today, if we had access to 30 days of data we wouldn't complain too much.  

The problem at this moment is that we have to get some new motorcycle parts machined and to powdercoat on Wednesday.  The design is brand new and we need to make a couple changes and also need to access our parts to revise our CNC programs.  We can't do either today because the data isn't there, so I have machines just sitting waiting on the cloud.

---

@Rigid_Concepts - for starters, I would bump up your cache setting in Preferences ("Offline cache time period").  Also, if you have known projects you're working on, our selective cache feature should be of help - more info HERE.

As an aside, coming from a fellow gear head, I'd love to know what kind of moto parts you do?  Street?  Dirt?

[Rigid_Concepts]

Thanks for the tip on the offline cache.  I didn't realize we had that option, I've now bumped it to 45 days.

We are currently building some new parts for Harleys.  We design products (consumer, industrial, etc.) and then machine them for various companies.  

[Aly-Tek]

The only problem you will have now is if you have more than one computer as I do. I did a load of work at home last night, came to work this morning and can't access those files as they are not in this cache on this computer 😕 

---

@Rigid_Concepts wrote:

Thanks for the tip on the offline cache.  I didn't realize we had that option, I've now bumped it to 45 days.

We are currently building some new parts for Harleys.  We design products (consumer, industrial, etc.) and then machine them for various companies.  

---

[brianrepp]

Update 9:15pm PT - thank you for hanging in there!  We're getting closer and will update this thread as soon as we begin deployment.  As you can imagine, once we go back online there will be a bit of a backlog for the systems to process, so there will be a ramp-up period.

[brianrepp · Accepted solution]

Update as of 10:00pm PT - Thank you for your continued patience!  We have resolved the issue and are taking Fusion 360 back online now.  Please note that it may take up to 20 minutes for your session to reconnect.  If you run into any delays or would like to speed up the process, you can also restart Fusion 360.

Please note, as we go back online, there may be some delays in processing due to the backlog.

[mavigogun]

---

@Rigid_Concepts wrote:

I've been thinking about switching all of our engineering over to fusion from Creo.  I've been hesitant to do so because the cloud isn't always there.   *snip*

The problem with today is that we can't access files that are over two weeks old (new designs completed in Fusion).  Can you make the local storage keep the last 30 days of work?  Even if you have a down time like today, if we had access to 30 days of data we wouldn't complain too much.  

The problem at this moment is that we have to get some new motorcycle parts machined and to powdercoat on Wednesday.  The design is brand new and we need to make a couple changes and also need to access our parts to revise our CNC programs.  We can't do either today because the data isn't there, so I have machines just sitting waiting on the cloud.

 For this very reason, I use the Data Panel and all related features only as a back up to local storage.   "Export" is code for "don't send the file anywhere, save it to the machine you're working at".    We used to just call that "save".   Ya, it makes sharing more arduous- BUT IT'S THERE, where you can get at it.

[brianrepp]

Now that we're back to normal, wanted to provide an update on exactly what happened leading to the outage earlier in the week:

The recent issue with our Access Control Management service impacted customers’ access to certain workflows within several Autodesk applications. We understand these issues are disruptive to your business and apologize for any inconvenience.  All services were fully operational by 11:30pm PT, September 10 and we do not expect any further disruptions related to this issue. 

What happened: We experienced an issue with the capacity scaling limit configuration for our Access Control Management database. This database manages access control policies for data stored by Autodesk products.  This was not a security issue. 

  

Mitigation: The issue was fully resolved with a code fix and we are reviewing all our production databases for any similar issues as a preemptive measure to avoid any further customer disruptions.