Did AutoDesk miss a key portion of Project Sync, or did I?

[mmaes]

Does anyone know if there is a way to only sync the files that have been changed in the past 24 hours to the cloud?  Or is there a work around that would let us sync on lifecycle transition?  I have not found that functionality and it seems to me like this should be a MUST have for collaboration.  We have several engineers working on projects all day every day and I do not see a clean way to only upload the data that has changed each day.  All I have found is either we have to depend on each engineer to manually push each and every file they change to the cloud or push entire project folders which can contain over 100,000 parts.  Surely there must be a simpler way to only push the files that are changed???

Any input is greatly appreciated?

[ihayesjr]

It sounds like you have not configured the schedule for syncing the folder. After you set the schedule for syncing, it will automatically upload only changed or new files.

See the following Help topic: Manage Cloud Drive Folder Mappings for Project Sync

[mmaes]

Thanks for the reply.  I do have a sync schedule setup but it seems to take a very long time and all the files are in the sync queue as though it is uploading everything.  From what I found on the Autodesk website it will resync all files on upload.  Please see the attached screen shot and link. https://knowledge.autodesk.com/support/vault-products/learn-explore/caas/CloudHelp/cloudhelp/2019/EN...

The folder I am syncing contains 110,000 files (about 50gb) and the information is shared with our suppliers and needs to be up to date at all times.  Ideally ONLY the changed files would sync and even better than that would be if we could get those individual files to sync on state change.  We are new to Project sync but in the past we used Buzzsaw which allowed the sync on state change of individual files.   It seems very odd that this feature is not available with Project Sync in my opinion.  

[ihayesjr]

The help topic you quoted is about manual uploads of files, not the automatic synchronization with Project Sync.

Upload to Cloud Drive

Selected files are added from the Vault server to the cloud drive based on the folder mapping configured on the Project Sync Settings dialog. If the file exists on the cloud drive already, a new version is added.

If a user clicks the button "Upload to Cloud Drive", the selected files will be uploaded as a new version is the files already exist or a new file if they don't exist in the cloud.

If you let the synchronization happen automatically, only new and changed files are synchronized.  Currently, you have a large number of files to upload initially. Once that is complete, only changes or new files are synchronized in that configuration.

I believe there is a custom method to synchronize on lifecycle change however, this still uses the Job Processor queue. I will look into this.

[mmaes]

Edit:  I will test again to be sure I am doing exactly how you mentioned (automatic sync as opposed to manual).

Thank you for your help.  If you can find information related to the lifecycle change I would greatly appreciate it!!

[ihayesjr]

When you say you are doing another sync, are you clicking the "Upload to Cloud Drive" button?

[mmaes]

I went into the collaborate folder and forced a new sync by changing the schedule and hitting apply.  This is basically a manual sync so I will test further letting it run automatically.  The problem I have been having is it errors out before a complete sync ever happens so my only option thus far has been a manual sync.

Thank you again for looking into my issues, I appreciate the help.

[ihayesjr]

So, I don't consider that a manual sync. It is a scheduled automatic sync however what you may be seeing is that if the previous sync never completed and some files didn't get synchronized, those files will get re-added to the job queue. This would make it seem like it is synchronizing all files versus the files that just changed.

[mmaes]

That makes sense to me.  I will monitor and continue to test.  

If you can look into a sync on state change that would be great.  Alternatively we will hire a third party like Imaginit to write code to make this happen but obviously if there is a known fix from AutoDesk that would be preferred.

Thank you again!

[mmaes]

Following up with what I found with the sync process.  I did a new sync right after my last post here and after it ran for 2 days it found there were 97000 of 110000 files that still needed to be sync'd.  After allowing those files to attempt to sync for 7 days only 17000 files actual moved to the cloud (80,000 remain not sync'd).  Is this normal?  It seems painfully slow if so.  

Buzzsaw is coming to its end of life January 31 and it seems there is no good solution to replace it that I have found and it seems difficult to get answers from anyone at AutoDesk.  

[ihayesjr]

Are you sending the files to Fusion Team?

[YounesYammouri]

Hi,

regarding your question for  is there a work around that would let us sync on lifecycle transition? 

to achieve this you can use within the project sync setting the filter tab where you can setup the state of the files as a Property and the value "Released". With this you can achieve the same results as syncing on lifecycle transition.

See details described within the following blog:

http://blogs.autodesk.com/vault/2018/04/external-collaboration-project-sync/

With that you ensure existing released files as well as every time a new file is released  within a specific folder get uploaded to Fusion Team automatically. 

Let me know whether this help with your request.

Best regards

Younes

[mmaes]

Hi, thanks for the link, however, I may not have been clear.  What I am looking to do is avoid the minimum required 8 hour delay between synchronizations.  Buzzsaw would allow a sync between Vault and Buzzsaw every time a file changed to the released state instantly.  I would prefer if this functionality existed within Project Sync so I could be 100% positive the latest version was always in the cloud.

Note "Value should be between 8 and 999":  For us this 8 hour window is too big.

[mmaes]

We are sending files to BIM 360.  We initially wanted Fusion but the functionality we required was not available with Fusion but was available with BIM.  

Currently we are Buzzsaw users and we send our part and drawing files from Vault to Buzzsaw for the sole purpose of giving our vendor access to our files and manufacturing individual parts.  Basically we send our vendors a purchase order, they go to buzzsaw, grab the drawings they need, and then manufacture the parts we need to build machines.

The problem we found with Fusion is there was no way for us to give strictly read only rights to our vendors.  Since multiple vendors will be sharing these files we do not want the vendor to be able to upload anything to cloud, only download.  BIM 360 allows this functionality so, maybe not our first choice but it does seem to have the features we need.

[YounesYammouri]

Hi,

what will be an acceptable interval window for your use case?

is for example every 2 h is acceptable?

Thanks

Younes

[mmaes]

Hi Guys, 

I want to update the situation we are having.  Since my last post I got some help from a gentleman at name Matthew Brown at Autodesk and he got all of our Buzzsaw files migrated over to BIM 360.  At this point I am doing more testing before we go live with our vendors and BIM 360.  

Log from migration

During testing I tried to re-sync the folder containing all of our files and it errors out in the job processor before it completes.  I really had high hopes that once the initial migration was complete that the sync would go fast because of Irvin's comment stating only new and changed files would being sync'd.  In my mind if the folder contains 110,000 files but only 50 of them changed then the sync would go very fast because all it would have to do is upload those 50 files.  That doesn't seem to be the case though (at least from a time standpoint).

Younes, 2 hours would be a great solution, however, I believe that would still require entire folders to be sync'd and not individual files.  Our file folder that needs to be synchronized contains about 110,000 files and the job in the job processor tends to error out after attempting to process the job so changing the interval at this time does not seem to be the solution.  

I do understand that what I am hoping for/requesting is not a current feature of Project Sync, my point is simply that with Buzzsaw it was possible to sync individual files on state change and this worked very well for us for years. Autodesk approached us with BIM 360 as a solution for the end of life of Buzzsaw and so far it doesn't seem to be working at all for us and we have been struggling to get support.

Thanks again for taking the time to look into our issue.

[YounesYammouri]

Hi,

can you please send me an email: [email protected]. I would like to schedule a call with you next week to go through the workflow you are covering today with Vault and Buzzsaw and I will make some advice how we might achieve the same results but with different way to get there.

Thanks 

Younes

[mmaes]

Hi Younes,  I sent you a reply this morning. 

As a note, I attempted another sync over the weekend and resulted in another error.

[JamesMcMullen]

When you logged into the Job Processor, did you log into it with both your vault credentials and cloud credentials.  In particular, your cloud credentials should have write control over the location you are attempting to write too.

Thanks,
Jim

[mmaes]

Hi James, 

Yes, I am logged in both in the job processor with my Vault ID and into the cloud with my Autodesk ID.  I believe that is what you are asking.  Also, I have admin rights for both.