2015Vault backup fails

[karthur1]

Running Valt Pro full backup. The backup fails so I look at the log to try to figure out why.  I see an entry there....

"Warning: Deleting backup folders content failed

Exception: Access to the path '004c9fcb-2a09-4d8d-b1a2-79faf31d0443.ipt' is denied.

10/24/2014 2:12:46 PM Connectivity.Application.VaultManager.ServiceException: BACKUP DATABASE permission denied in database 'AI2014_CustomContent'.
BACKUP DATABASE is terminating abnormally. ---> System.Data.SqlClient.SqlException: BACKUP DATABASE permission denied in database 'AI2014_CustomContent'.
BACKUP DATABASE is terminating abnormally.

."

(entire log file attached)

I am sure this is a permissions issue, but how do I correct it?  I searched the folders for the filestore, but did not find this folder.  I am the administrator on this server, so not exactly sure why the acces would be denied.

Thanks,

Kirk

[Neil_Cross]

The Vault backup routine doesn't really fully utilise the credentials that you're logged into the server with.  It's more to do with the credentials assigned to the SQL services by the looks of it.

Verify which user account is assigned to the service "SQL Server (AUTODESKVAULT)" and check that it still has full access to the source and destination areas.

I think this may not have an impact on the backup issue you're seeing, but what user account is set as the impersonation account? And do these users have write access to the backup destination and current filestore location?

Second and most obvious question would be, it looks like it worked perfectly fine on the 15th, then failed on the 16th.  Have you or your IT made any security changes relating to accounts and folders etc during that time?

On a side note, I'd also be getting quite concerned that it's taking 17 hours to backup your filestore! 

10/14/2014 5:52:34 PM Backing Up Databases... 2/17
10/15/2014 10:44:32 AM Backing Up Databases... 3/17

Starts at 5:52pm and finishes that stage of the backup the next day at 10:44am! You either have a monstrous Vault sized in several terrabytes, or dreadful server disk write speeds, or you're backing up to a network location/external drive.  If it's the latter, then you could be looking at the reason why you're now having rights issues during the backup if someone has changed permissions somewhere.

[ihayesjr · Accepted solution]

Kurt,

Try detaching and re-attaching this library through the ADMS console.  This should reset the permissions for the database inside of SQL.

[karthur1]

---

@Neil_Cross wrote:

The Vault backup routine doesn't really fully utilise the credentials that you're logged into the server with.  It's more to do with the credentials assigned to the SQL services by the looks of it.

Verify which user account is assigned to the service "SQL Server (AUTODESKVAULT)" and check that it still has full access to the source and destination areas.

I think this may not have an impact on the backup issue you're seeing, but what user account is set as the impersonation account? And do these users have write access to the backup destination and current filestore location?

Second and most obvious question would be, it looks like it worked perfectly fine on the 15th, then failed on the 16th.  Have you or your IT made any security changes relating to accounts and folders etc during that time?

On a side note, I'd also be getting quite concerned that it's taking 17 hours to backup your filestore! 

10/14/2014 5:52:34 PM Backing Up Databases... 2/17
10/15/2014 10:44:32 AM Backing Up Databases... 3/17

Starts at 5:52pm and finishes that stage of the backup the next day at 10:44am! You either have a monstrous Vault sized in several terrabytes, or dreadful server disk write speeds, or you're backing up to a network location/external drive.  If it's the latter, then you could be looking at the reason why you're now having rights issues during the backup if someone has changed permissions somewhere.

---

As far as I know, nothing has changed since it worked on the 15th.  I haven't changed anything and I seriously doubt the IT guy has..... thus the question.

The vault file store is about 268GB.  I am backing up to a external SSD drive thats connected directly to the server via USB.

@ihayesjr 

I have detached/reattached the library and running the backup now.  Will let you know how it goes.

Thanks,

Kirk

[Neil_Cross]

Ok, see how that goes with the detach & attach suggestion.

With regards to the 268GB filestore, that shouldn't take anywhere near 14 hours to copy to an external drive.  That's an average transfer speed of 5 MB per second, too fast for USB1.0 but way too bad to be USB2.0, how is the external drive hooked into the Vault Server?    

I appreciate this wasn't the primary reason for you posting on the forum so I apologise for being nosey and instrusive! But the 15th & 16th October is midweek, so your backups are running well into the next working day, as you know Vault Pro has live backup capability, but still, it's far from ideal?

Just putting it into perspective, my Vault is 500GB in total and backs up fully in around 6 hours which is still far too long for my liking.  

Just a thought, I'll hush if you're not fussed about it 

[ihayesjr]

There are multiple reasons for a backup to take a long period of time.  That will require troubleshooting in different areas.  The first area would be to see if the scheduled task has a Normal priority level to run.  By default, it has a Low Priority unless changed manually.  With a low priority other things on the system will have first priority to the CPU.

[karthur1]

---

@Neil_Cross wrote:

Ok, see how that goes with the detach & attach suggestion.

With regards to the 268GB filestore, that shouldn't take anywhere near 14 hours to copy to an external drive.  That's an average transfer speed of 5 MB per second, too fast for USB1.0 but way too bad to be USB2.0, how is the external drive hooked into the Vault Server?    

I appreciate this wasn't the primary reason for you posting on the forum so I apologise for being nosey and instrusive! But the 15th & 16th October is midweek, so your backups are running well into the next working day, as you know Vault Pro has live backup capability, but still, it's far from ideal?

Just putting it into perspective, my Vault is 500GB in total and backs up fully in around 6 hours which is still far too long for my liking.  

Just a thought, I'll hush if you're not fussed about it 

---

No problem.... appreciate the insight.  I have been trying to work out some issues with the backup routine, that is why it was done mid-week.  Normally, my backup starts on Saturday afternoon.

The external drive is a Toshiba DWC120 (2TB).  Its connected via the rear USB port. Pretty sure the server supports USB 2.0, but will check on that.

Thanks,

Kirk

[Neil_Cross]

Didn't you say it was a SSD? That link references a standard external 5700rpm drive.  No matter, that still wouldn't explain the lengthy backup time.  Try the above suggestion regarding task CPU priority.

Another conclusive test would be to simply copy a large file using standard Windows Explorer over to the external drive and take note of the dynamic reading Windows gives you for file transfer rate.

When I was out and about doing Vault installs in bulk, I would always avoid scheduling backups direct to external drives.  I would personally (if there's enough disk space) backup to the same drive & partition as the Vault filestore, then schedule a separate task to Xcopy the Vault Backup over to the external drive after it's finished.

This was usually much faster as the backup routine isn't coping over partitions & drives, which can slow down the data transfer rate.  Also USB connections although not regularly, can disconnect from the OS.  The drivers can bug out and disconnect the drive, or under heavy load.  A load of reasons can cause a USB drive to vanish when you're not present, that's your backup failing unless you're eagle eyed and spot it in good time.  I found that backing up locally then scripting a copy to the external HDD was a much safer solution, if that workflow suits your circumstances.

But yea check the USB port version, if it's USB1.0 then you've maybe got a very old server! If it's USB2.0 then something else is causing a write bottleneck as USB2.0 is pretty fast.

[karthur1]

Neil,

Will try the suggestions you made about writing to the server first, then copying to the external.  Never thought of that.

Yes, you are correct, the external is not a SSD, but a regular harddrive. I had it confused with a different one.

Thanks,

Kirk

[karthur1]

Neil,

I changed the backup routine to backup to the c:\ then copy the files over to the external drive.  Doing it this way brought the time down to 10 hrs.

That is much better than backing up directly to the external drive.  I have made another change (running at highest priority) and will run it again tonight when everyone is off the server and see how that goes.

Thanks,

Kirk