RESOLVED: Fusion 360 Service Outage

[keqingsong]

UPDATE DEC 8, 2;22 A.M. PST -

All, tests on our end have shown that we are back online. You should now be able to log in and access your data and use Fusion 360 as you normally would. Things are still ramping up, but if you continue notice any sluggishness or issues, please let us know. Thank you for your patience, sympathy, words of encouragement, and continued support. It meant a lot to all of us!  

----------------------------------------------------------------------------------------------------------------------------------------------------

UPDATE DEC 7, 9:17 P.M. PST -

All – A quick update on the outage.  We are close to a resolution.  We have started testing Fusion in our internal staging environment.  Once we are sure everything works, we will go live and bring Fusion online. I know this has been very painful for all of you and all of us in the fusion team feel your pain.  I really appreciate your patience as we work the problem.  Once we get thru this and you are all up and running, I will update you on the work we are doing and are doubling down to make Fusion resilient and more importantly make sure that you have access to your data all the time.

In case you are wondering what the current issue is why it is taking so long to address it, here are the details.  Fusion 360 is a network of services and one such service is what we call ACM (Access Control Manager) Service.  The purpose of ACM is to govern project level access to all data resources (items, projects, folders…) and its derivatives (views, comments, …).  We periodically optimize our Fusion Item Access Control Database, based on changing application needs and usage of ACM.  An optimization done on Dec 6 made calls to ACM getting locked up in the database and never return. Ultimately, we had to bring our ACM service down to roll back the changes and make sure all the services operate normally.  It is an involved effort and hence the delay.

Prabakar.

----------------------------------------------------------------------------------------------------------------------------------------------------

UPDATE DEC 7, 11:05 A.M. PST - We are now estimating to have the outage resolved by 1pm PST.  I will keep this thread updated if anything changes.  Thanks for your continued patience.

UPDATE DEC 7, 7:19 A.M. PST - We are making progress on battling the outage. We have been having issues with our Access Control System that governs project level access to all data resources (items, projects, folders...) and its derivatives (views, comments, ...) in Fusion for creators and collaborators.

Our Operations and Engineering Team is engaged ferociously since midnight Dec 6 to address this issue. We expect a speedy resolution of this issue soon.

Right now we just deployed a banner in-product that will take you into offline mode if you haven't already done so. More details to come, and again apologies for the downtime.

Hey all, 

We are experiencing an outage that is impacting all data access interactions.  These include but not limited to Open, Save, Upload, Export and Viewing. We recommend that you work in offline mode for the time being. We are working actively to correct the problem and will update this thread once it is resolved.  We apologize for the inconvenience and appreciate your patience.

You can also keep track of the status via the health dashboard. 

Here's how to switch to offline mode to continue working while we dig into the issue. 

https://knowledge.autodesk.com/support/fusion-360/troubleshooting/caas/sfdcarticles/sfdcarticles/How...

If you restarted Fusion without going into offline mode before, it may seem like fusion is hanging at the launch screen. Wait a few minutes and you should be able to get in. As soon as you get in, switch to offline mode. Then you should be good to go while we fix the downtime.

[Beyondforce]

Fusion is working for me in Denmark 🙂

Ben.

[krunoslav_knezic]

Thanks Autodesk team for all the sweat and tears making it alive again.

Fusion is live and kicking from Croatia

(now I only need large quantities of coffee)

[mslatem]

Working for me again in Australia - thank you Fusion Team 🙂

I hope you all get some well earned rest, you've probably pulled an all nighter on this one!

[arnou-verfaillie]

Belgium working again!

Big up for the autodesk team! Thanks for getting things back up!
Hopefully you will find a better solution to avoid these type of problems!

I will wait to do my cloud simulation later this day 😉

[Anonymous]

All working in the UK - many thanks

[LMD001]

Hello Brian,

Seemed a tough nut to crack this time!

Hope that measures will be taken to avoid these kind of outages, I know stuff happens, more reason to take some kind of local back up in consideration.

For now, up and running again.

Nice work!

Best regards,

Ludo

[burnandreturn]

Congratulations on the fix!

Two things I see need to happen as soon as possible.

!.   Local storage of all files with backup to the cloud being an option not mandatory.  That would bring in a lot of new users.

2.  The option to not have an automatic update to the software for a select period of time to enable users to finish a project in a working version before updating the software to the newest version that might be broken as in what happened in the not too distant past.  An option to hold off for a few days to facilitate helping a user up against a dead line to finish a project or at least restructure one before an update.  Some of the burden should fall on the user so a mandatory update say after 5 days.  They would get a notice/warning stating that an update is happening and they could elect to have it installed immediately or hold off for 5 days.  5 days might not be enough but you get the idea.

[Remy18]

No issues going back online here in the Netherlands ! Primary using Fusion on 2 computers, we had a good experience with the offline files in cache to switch between them and keeps the machines running for the offline period.

[cekuhnen]

Michigan checking in - all system operational - back to work - yeah !

[Anonymous]

What's all this fuss and buzz about some epic outage ?  My Fusion is working great.

Mind you, I just fired it up for the first time in a week - LOL

[neljoshua]

Back online in Colorado.  It is great to see that all my work from yesterday has been successfully uploaded.

I think that @susibiker is exactly right in saying that there is a potential here for some good things to come out of this.

@keqingsong, @brianrepp, @prabakarm, & the rest of the Fusion team (who likely were not posting much due to working on getting things back up): thanks for your work and communication.  I know of no other software (cloud-based or otherwise) that gives this level of feedback and transparency.

[Anonymous]

Now that the outage is resolved, how do I get Fusion 360 back to 'online mode'?  I can't find any navigation to help me, and when I try to 'Save' or 'Export' my design, I get the following message:  "In order to work offline we need an up to date list of your projects. Please return online to automatically refresh the list."  

I only started using Fusion last night, at an Autodesk workshop, so I may be missing something obvious.  I have my first design we spent 3 hours on, but no way to save it... hoping you can help me so I don't lose all that work!

Thanks so much!  😄

[Anonymous]

Following up with a screen shot of my Fusion window, so you can see what's happening/not happening.  I don't seem to be in 'Offline Mode' either... there is no check next to it.  But the cloud has a line through it, and I'm clearly not in Online Mode either.  I'm somewhere in between.  Or nowhere at all.

This must be where bots live there private lives..

[Anonymous]

I can't get in to Fusion as a new user.  I tried creating a new account yesterday before finding this post about the outage.

I get an error popup with the title: "Sign in Failure Due to Service outage."  The message is: "There was a unexpected authentication error while logging in to Fusion 360. Please try again later."

I've tried reinstalling and disabling my virus software and firewall.

I realize your priorities are understandably elsewhere, but offer this as a heads up.  I'm not sure this error is related to this thread, but thought it might be based on the title of the error message mentioning a service outage.

Before this error message it Fusion starts with the page: "Hold tight... Just another minute as we finish setting up Fusion 360."

Then, goes to "Unable to set up Fusion 360 for your Autodesk account." page with the message "Please contact us through the Fusion 360 Help & Support forum to resolve this issue. We apologize for the inconvenience."

Thanks!

 - Chris

[kshea9RNL8]

Normally, the return to OnLine is automatic when it becomes available.

Seems a re-boot of Fusion 360m may be needed, I don't have a good feeling about your work surviving

Check to be sure auto save is turned on, I believe those files are saved locally.

If it is it should prompt you an opportunity to recover those files during the boot up process.

My habit to save files locally numerous times during work sessions if the system allows this.

[James.Youmatz]

Hi @Anonymous,

Welcome to the Fusion 360 Community!

I believe this is related to this post here. Right now there is a backlog for new hub creations which we are rapidly working on fixing and should have an update as soon as the new hubs are ready to go! 

Thanks,

[James.Youmatz]

Hi @Anonymous,

Hopefully I can help you out here! Just to check, can you force Fusion into offline mode by selecting "Work Offline". Then, are you able to save the file to your offline cache or does it throw up another error message. My number 1 priority is to get your file backed up or saved and then we can address the offline issue and get Fusion back online. If that doesn't work, please send me a private message (or email, james.youmatz@autodesk.com - new forum accounts have limited messaging privileges and we can work on getting this resolved!)

Thanks,

[Anonymous]

Thanks a lot , we really appreciate it. 🙂

[Anonymous]

I can no longer generate turning tool paths when generating new setups in CAM!  I can generate tool paths in setups that have already been created but not new setups.  Help!

[andersEQHR7]

@James.Youmatz

Hi, found a strange behavior from this outage. Did some work yesterday evening at my Home computer and saved it to the cloud, all synced at it should 🙂 . When I started my work computer this morning Fusion360 was in Offline mode, changed to Online mode before I opened any project. Then I opened the project that I was working on last night on my Home computer...... Now the strange part are coming. The last version are not the version (V5) that I edited back home yesterday instead I got a newer version (V6) that are not correct, since (V6) are the old version from work computer Offline yesterday during the day......

For me it looks like my Work computer sent up an older version to the cloud and the cloud system named it as the newest version.............

I hope you follow the above, not so easy to explain in English for me 😉

I solved the problem using Version 5 and as it is a hobby project it was not any big issue 🙂 🙂