Vault General Discussion

Reply
Active Member
DBD-AU
Posts: 10
Registered: ‎06-22-2006
Message 1 of 9 (126 Views)

Vault Backup Question

126 Views, 8 Replies
06-22-2006 06:56 PM
I'm currently preparing to setup my first major Vault with full SQL installation and have been reading these forums and as many other Vault documents I can to prepare.

This morning I read the the 'Best Practices for Implementing Autodesk Vault' white paper and it stated you should not use any backup utility which backs up SQL databases directly and should only use the backup feature available in Vault itself to effectively create an exported copy of your vault store, and then back that up. The document also said this could all be automated via a simple batch script.

The biggest downside I see here is that the client would effectively require enough storage to hold 2 copies of their entire vault store (the original and the exported 'backup' copy). Is this correct?

I'm curious if this is still the case or if the current version of Vault's SQL store is capable of being backed up directly by a tape backup solution (like Bakbone) with an SQL live access plugin license?

Any help with this would be greatly appreciated.
Syama
*Ross Tanner \(Autodesk\)
Message 2 of 9 (126 Views)

Re: Vault Backup Question

06-23-2006 05:05 AM in reply to: DBD-AU
You are correct, the best practice temporaraly requires double the disk
space. Use the included utility to create a full backup then archive it off
to the media of your choice. This process continues to be the recommended
best practice.

If you were to use any of the 'live' SQL plugins to backup the db you run
the risk of getting out of sync with the filestore. This will occur if a
user makes a change that is not captured in the backup of both the file
store and the db. This is one of the reasons why we recommend using the
included backup utility. If the filestore and db get out of sync there is
potential for much badness. The included backup utility also makes a single
backup package that contains all that is needed to restore, if necessary.
Contrast this against two seperate backups of the file store and database
which now both have to be managed and if a restore is requried you MUST have
a matching set that were created from the exact same state. This manual
process is certainly possible but we do not recommend it due to the
potential for the loss of data.

ross


wrote in message news:5216380@discussion.autodesk.com...
I'm currently preparing to setup my first major Vault with full SQL
installation and have been reading these forums and as many other Vault
documents I can to prepare.

This morning I read the the 'Best Practices for Implementing Autodesk Vault'
white paper and it stated you should not use any backup utility which backs
up SQL databases directly and should only use the backup feature available
in Vault itself to effectively create an exported copy of your vault store,
and then back that up. The document also said this could all be automated
via a simple batch script.

The biggest downside I see here is that the client would effectively require
enough storage to hold 2 copies of their entire vault store (the original
and the exported 'backup' copy). Is this correct?

I'm curious if this is still the case or if the current version of Vault's
SQL store is capable of being backed up directly by a tape backup solution
(like Bakbone) with an SQL live access plugin license?

Any help with this would be greatly appreciated.
Syama
*James Wedding
Message 3 of 9 (126 Views)

Re: Vault Backup Question

06-23-2006 07:14 AM in reply to: DBD-AU
OK, so can we agree that this is an asinine backup plan? No other major
service requires shutting down to back it up. Haven't you guys ever heard of
all nighters? Then to require the doubling of storage just to make backup a
possibility? Since C3D is going to vault as solution, and eventually all
products will, this plan of attack just seems more and more inane.

I know you didn't design it Ross, so please don't take it personally, but I
do have to wonder who did design it, and what the hell they were thinking.

--
James Wedding, P.E.
Engineered Efficiency, Inc.
Civil 3D 2007
XP Tablet, SP2, 2GHz, 2G
www.eng-eff.com
www.civil3d.com
*Dave Hoder
Message 4 of 9 (126 Views)

Re: Vault Backup Question

06-23-2006 09:40 AM in reply to: DBD-AU
Couldn't you use a standard backup if you scripted it to shut down the
services? You couldn't keep working through a backup but it wouldn't require
the extra space. (by the way, it's triple not double with the supplied
script)
*James Wedding
Message 5 of 9 (126 Views)

Re: Vault Backup Question

06-23-2006 11:26 AM in reply to: DBD-AU
The magic question is:

Why are we shutting down services to do backups???

And I've asked everyone on the ISD side, they point to the Vault team, so
I'm still waiting for a decent answer.

--
James Wedding, P.E.
Engineered Efficiency, Inc.
Civil 3D 2007
XP Tablet, SP2, 2GHz, 2G
www.eng-eff.com
www.civil3d.com
*Irvin Hayes \(Autodesk\)
Message 6 of 9 (126 Views)

Re: Vault Backup Question

06-23-2006 12:56 PM in reply to: DBD-AU
James,

The key here is to keep the information in the database in sync with the
files that is in the filestore.
So during the backup we shut down the web service so no one can add or edit
files that is in the vault.
Here is an example of what could happen if we did allow a live backup.

Let say during the backup we perform a backup of the database first then
once the database is backed up
A user logs in and checks in some new files.
The files make it to a filestore directory before the backup reaches that
directory.
So now the new files are backed up in the filestore but the database knows
nothing about these files.
If you go to restore this backup, the Vault Manager will indicate that there
are extra files in the filestore but not in the database.
Now your database and filestore are not in sync with each other.

The same can happen in reverse. If we backup the filestore prior to backing
up the database, the user checks in a file.
The information is in the database but since the filestore has already been
backed up the file will be missing from the backup.
When your restore this backup, the Vault Manager will indicate that the
files are in the database but not in the filestore.

I hope this helps.

Irvin
"James Wedding" wrote in message
news:5217301@discussion.autodesk.com...
The magic question is:

Why are we shutting down services to do backups???

And I've asked everyone on the ISD side, they point to the Vault team, so
I'm still waiting for a decent answer.

--
James Wedding, P.E.
Engineered Efficiency, Inc.
Civil 3D 2007
XP Tablet, SP2, 2GHz, 2G
www.eng-eff.com
www.civil3d.com
*Jon Rizzo
Message 7 of 9 (126 Views)

Re: Vault Backup Question

06-23-2006 02:20 PM in reply to: DBD-AU
Couldn't the Vault API be modified so that it detects whether or not a
backup is currently being run, and if so, it "does something else" with any
changes until the backup is complete (such as storing the changes in a
temporary database and file store). Then, when the backup is complete, any
changes are merged with the database. Obviously the changes would not be
part of the backup, but I would not expect them to be.

I know it is much, much more complicated than that, but it seems like this
should be possible...

Jon


"Irvin Hayes (Autodesk)" wrote in message
news:5217423@discussion.autodesk.com...
James,

The key here is to keep the information in the database in sync with the
files that is in the filestore.
So during the backup we shut down the web service so no one can add or edit
files that is in the vault.
Here is an example of what could happen if we did allow a live backup.

Let say during the backup we perform a backup of the database first then
once the database is backed up
A user logs in and checks in some new files.
The files make it to a filestore directory before the backup reaches that
directory.
So now the new files are backed up in the filestore but the database knows
nothing about these files.
If you go to restore this backup, the Vault Manager will indicate that there
are extra files in the filestore but not in the database.
Now your database and filestore are not in sync with each other.

The same can happen in reverse. If we backup the filestore prior to backing
up the database, the user checks in a file.
The information is in the database but since the filestore has already been
backed up the file will be missing from the backup.
When your restore this backup, the Vault Manager will indicate that the
files are in the database but not in the filestore.

I hope this helps.

Irvin
"James Wedding" wrote in message
news:5217301@discussion.autodesk.com...
The magic question is:

Why are we shutting down services to do backups???

And I've asked everyone on the ISD side, they point to the Vault team, so
I'm still waiting for a decent answer.

--
James Wedding, P.E.
Engineered Efficiency, Inc.
Civil 3D 2007
XP Tablet, SP2, 2GHz, 2G
www.eng-eff.com
www.civil3d.com
*dMiller
Message 8 of 9 (126 Views)

Re: Vault Backup Question

06-27-2006 05:16 AM in reply to: DBD-AU
I buy this given the architecture... but why exactly is there a file store
when SQL server is quite capable of storing the files internally? I realize
this would take a total rebuild (and a total teardown of the .dwg structure
to make it efficient) but a .dwg is just a database that could be rebuilt
into SQL tables, or you could just store the .dwg intact in it's original
form for now... until you figured out how to restructure. ESRI already does
this with their ArcSDE product.

Using a powerful database like SQL server as a file pointer is kind of a
waste of resources IMHO. I guess it keeps Autodesk from forcing everyone to
purchase a full version of SQL server because this option would probably
overtax MSDE or SQL express.



"Irvin Hayes (Autodesk)" wrote in message
news:5217423@discussion.autodesk.com...
James,

The key here is to keep the information in the database in sync with the
files that is in the filestore.
So during the backup we shut down the web service so no one can add or edit
files that is in the vault.
Here is an example of what could happen if we did allow a live backup.

Let say during the backup we perform a backup of the database first then
once the database is backed up
A user logs in and checks in some new files.
The files make it to a filestore directory before the backup reaches that
directory.
So now the new files are backed up in the filestore but the database knows
nothing about these files.
If you go to restore this backup, the Vault Manager will indicate that there
are extra files in the filestore but not in the database.
Now your database and filestore are not in sync with each other.

The same can happen in reverse. If we backup the filestore prior to backing
up the database, the user checks in a file.
The information is in the database but since the filestore has already been
backed up the file will be missing from the backup.
When your restore this backup, the Vault Manager will indicate that the
files are in the database but not in the filestore.

I hope this helps.

Irvin
"James Wedding" wrote in message
news:5217301@discussion.autodesk.com...
The magic question is:

Why are we shutting down services to do backups???

And I've asked everyone on the ISD side, they point to the Vault team, so
I'm still waiting for a decent answer.

--
James Wedding, P.E.
Engineered Efficiency, Inc.
Civil 3D 2007
XP Tablet, SP2, 2GHz, 2G
www.eng-eff.com
www.civil3d.com
*Jeff Pek \(Autodesk\)
Message 9 of 9 (126 Views)

Re: Vault Backup Question

06-27-2006 05:42 AM in reply to: DBD-AU
You're right -- I think the main reason was space conservation. The
information in the filestore is typically quite a bit (5x or more) larger
than the metadata, and given MSDE's 2GB database limit, using a separate
file store allows a "free" implementation to hold many more files.

Rest assured, there are still plenty of things for SQL Server to do.

Jeff

"dMiller" wrote in message
news:5219746@discussion.autodesk.com...
I buy this given the architecture... but why exactly is there a file store
when SQL server is quite capable of storing the files internally? I realize
this would take a total rebuild (and a total teardown of the .dwg structure
to make it efficient) but a .dwg is just a database that could be rebuilt
into SQL tables, or you could just store the .dwg intact in it's original
form for now... until you figured out how to restructure. ESRI already does
this with their ArcSDE product.

Using a powerful database like SQL server as a file pointer is kind of a
waste of resources IMHO. I guess it keeps Autodesk from forcing everyone to
purchase a full version of SQL server because this option would probably
overtax MSDE or SQL express.



"Irvin Hayes (Autodesk)" wrote in message
news:5217423@discussion.autodesk.com...
James,

The key here is to keep the information in the database in sync with the
files that is in the filestore.
So during the backup we shut down the web service so no one can add or edit
files that is in the vault.
Here is an example of what could happen if we did allow a live backup.

Let say during the backup we perform a backup of the database first then
once the database is backed up
A user logs in and checks in some new files.
The files make it to a filestore directory before the backup reaches that
directory.
So now the new files are backed up in the filestore but the database knows
nothing about these files.
If you go to restore this backup, the Vault Manager will indicate that there
are extra files in the filestore but not in the database.
Now your database and filestore are not in sync with each other.

The same can happen in reverse. If we backup the filestore prior to backing
up the database, the user checks in a file.
The information is in the database but since the filestore has already been
backed up the file will be missing from the backup.
When your restore this backup, the Vault Manager will indicate that the
files are in the database but not in the filestore.

I hope this helps.

Irvin
"James Wedding" wrote in message
news:5217301@discussion.autodesk.com...
The magic question is:

Why are we shutting down services to do backups???

And I've asked everyone on the ISD side, they point to the Vault team, so
I'm still waiting for a decent answer.

--
James Wedding, P.E.
Engineered Efficiency, Inc.
Civil 3D 2007
XP Tablet, SP2, 2GHz, 2G
www.eng-eff.com
www.civil3d.com
Announcements
New button for Platinum & Gold Subscription customers
Gold & Platinum Subscription customers can alert Autodesk if they require assistance with their forum post

Manufacturing Community
The Manufacturing Community provides additional access to tips, tutorials, blogs and networking with peers.
Need installation help?

Start with some of our most frequented solutions or visit the Installation and Licensing Forum to get help installing your software.