Mimic Task Scheduler Migrate with a Custom Task

[petestrycharske]

All,

I am working on a large file migration project with lots of folders that contain 1 or 2 files in each, so it is impractical to select each folder and the migration task keeps failing when I try to migrate the entire overarching folder.  Therefore, I decided to go down the route of a custom task based on a blog post that I read (here is the post) and can get most of the way there with a ton of assistance from one of my programming colleagues.  However, I am a little stuck because I get different results if I migrate with this custom task versus using the task scheduler.

In this case, I migrated a fileset of 678 IPT files with the custom task and all the files passed successfully.  However, when I migrated the same set of files as a test, I get three failures via the Task Scheduler migration task.  These failures are based on a mass update failure.  This got me thinking that I am probably not checking the same criteria that the task migration task is checking.  Does anyone know the  criteria that one should be checking to closely mimic task scheduler when doing custom file migration?

Currently I am opening, rebuilding, saving and closing each file in a CSV file list.  I will look to add criteria to update the mass, but want to make sure that there isn't something else that I am missing.  If you have any questions, please do not hesitate to contact me.  Hope all is well and have a most blessed night!

Peace,

Pete

[sth]

Hi Frederic,

I can confirm that I'm at least the 2nd one in this world having the same trouble. To migrate our ~ 170 000 Inventor documents we used to buy a new "power machine"
Intel(R) Xeon(R) CPU E5-1630 v4 @ 3.70GHz 

NVIDIA Quadro P4000 21.21.13.7735 8192MB 21.21.13.7735

128 GB RAM (8 modules Samsung 2400 MHz  16GB M393A2G40DB1-CRC)

2 PCIe SSD disks

NVMe SAMSUNG MZVLW512 SCSI Disk Device 512,11GB

INTEL SSDPEDMW400G4 400,09GB

OS is Win 10

And, what to say - it's less fast and stable than a more than 5 years old Win 7 workstation  I requested this at Autodesk, after waiting a week I got a first answer, but yet no solution.

What I saw in this unlucky migration:

Task scheduler didn't terminate the Inventor tasks proper (once more than 100 started with all the acwebbrowser instances)

Sometimes access to this mdb failed

Sometimes Inventor launch fails.

Due to this facts I agree with you that there might be changes or heavy Win10 compatibility issues. I've the feeling that since generations the Task Scheduler wasn't updated or any error fixed. I used it first migrating from R4 to R5 years ago - today I see all the same issues still persist (empty tasklist, ServiceModule.exe crash,etc, etc......)

@ChrisMitchell01: Try out any "large assembly" having at least one unmigrated component - see the awesome performance decrease caused by this old part and enjoy the performance when you migrated your files to the current release. "Zero Migration Impact" is only a solution to avoid the suicide of all Vault users .  

So I can confirm that this migration to 2018 hurts more than the awful projects in the past .

And this after the subscription fee increased dramatically because we won't be forced to use only the rent model.

More money for less service

I believe that we're too old fashioned here in Europe. Only boring drawings and old fashioned desktop software - we don't understand that the "future of making things" is cloud based and mobile. (Never seen any A0 drawing on my smartphone, don't believe that it's a charme )
May someone at Autodesk wakes up from cloud dreams and considers that it's our fees securing that he's paid (well I hope )

Sorry for annoying anyone with my thoughts about the current situation, but I've the feeling that most employees of Autodesk don't know about the things we customers are struggling with.

Cheers,

Thomas

[frederic_joubert]

Hi Thomas,

I very glad to see that I'm not the only one awake in a sky without cloud!

And yes, as you said, it's really not the same at Autodesk sitting on their pedestal.

I got, from good source, that everything at Autodesk is a marketing thing. They bought all their competitors. They don't give a ... about users problems. Except for paying more money there is no advantage, for user, to go cloud. He also strongly suggest me that if you are on a subscription STAY THERE, KEEP IT as long that Autodesk let us this privilege...

All workstations here are on Windows 7 pro and me too I had better migrating performance with older and slower workstation then the speedy recently one. We are changing all workstations next mount and I'm eager to see if that will help. I don't know why but I'm a bit sceptic.

This morning, I stop a 50 folders coma migrating task that the preprocessing was still stuck at the same file then when I left yesterday. I start Inventor and open the "sticking" files. No message problem at the opening, no unresolved references, no sketch doctor and save it just like that. This, again, confirmed me that there is a problem with the 2018 Task Scheduler.

But I notice something this morning, I make a migrating task of one special product folder containing 842 files and it take less then an hour. Yesterday it took 6 hours to migrate 516 files from 50 products folders. Does a multiple folders migrating tasks are now a problem?

I completely understand Pete to program is own migrating tool. Don't stop Pete you could make money of that I think.

Well enough, I have to migrate things now.

Good luck and peace to all Autodesk migrators

Fred

[sth]

Hi Frederic,
wow – we’re 2 now fighting for old school CAD and we’re not able to use all the awesome and amazing cloud and mobile toys.
Yesterday I was able to finish a benchmark for migration of ~4200 Content Center models (simple screws and nuts)
Machine

Processing time

HP Z 600, Intel XeaonX5675@3,07GHz, 12GB RAM, NVIDIA Quadro 4000, age elder than 5 years
Win7 64 Enterprise, Performance Index 5,9

10:52 – 16:17 (5 hours, 25 mins)

HP Z420, Intel Xeon E51620@3,6 GHz, 32GB RAM, NVIDIA Quadro K4000, Dual SSD RAID HD with 256GB, age ~3 years
Win7 64 Enterprise, Performance Index 7,6

16:44 - 18:07 (1 hour, 23 mins)

Tarox Custom machine
Intel Xeon E5-1630 v4 @ 3,7Ghz
128GB RAM, NVIDIA Quadro P4000, 2 PCIe connected SSD’s
Win10 Pro

14:00 – 21:37 (7 hours, 37 mins)


For me this shows the potential issue the Task Scheduler (and Inventor?) have with Win 10 and current hardware.
Also the user experience on the power machine is poor – not faster than the Z420 and opening old format
larger designs takes a long time (after migration better, zero migration is the ultimate performance killer)
By the way: The empty scheduler window (no tasks visible) seem to be caused by a “hanging” TaskDBE.exe
instance. When I kill this instance and start the scheduler again the jobs are visible (and clear, the started one
died over night, (Servicemodule.exe passed away for some reason, the status of the job is “Unknown Exception,
and that’s true – I’ve no idea what this exception is).
Here’s the “Good Morning” from the “Tarox High End machine”
[cid:image001.png@01D3230C.0634A710]
72 Inventor CER windows were open, 27 Inventor processes still were started (I made a small tool to count all these instances)



I become more and more afraid what will happen when they try to force us migrate to Fusion 360 and Fusion lifecycle.
But the current issues show that all development power went away from Inventor and Vault to support the Cloud platform.
Dev Blogs passed away (Mod the machine, It’s only ones and zeros) because the gurus shifted to the cloud team.
Bad future for old school business when staying at Autodesk if they don’t change their policy I fear.
So thank god it’s Friday ☺ - the terrible show will be interrupted for 2 days
Cheers,
Thomas

[sth]

Hi All,

unfortunately the picture disappeared - hope now this screenshot is visible.

Today I received a new message from the Autodesk Support: "No known issues for the R2018 migration". May I tool the wrong pills today morning 

Cheers,

Thomas

[ChrisMitchell01]

Hi Thomas,

Are you migrating directly from vault ? If so then that may explain some of the delays since the files need to be checked-out/retrieved locally before the migrations occur. Have you checked for activity on the vault server during these migrations ?

We regularly migrate 000's of real customer files, both locally & from vault, using both win7 & win10, as part of our normal QA activities. This issue has not yet been reproduced.

When migrating I typically do things separately & in this order:

Library parts,

Normal parts

Assemblies,

Presentations,

Drawings

Often it makes sense to further break down the migration jobs based on separate folders, etc so that it's more manageable. For 170K files I could easily imagine creating 20 or 30 separate migration jobs, (not sequential) & would likely use multiple different physical workstations. After a job completes I usually keep the log file, delete the job from task Scheduler, compress the database & then start the next one. Yes, I know not exactly batch processing in its' true sense....

For large assemblies it's always best to migrate the data, (although this is a project we are actively working to improve right now). For smaller assemblies the impact of not migrating if often negligible.

What settings do you use for multi-processing ? I typically use something like this:

Note that the number of processes & RAM are set to be less than that actually available. I never enable hyper-threading at the BIOS level. I always disable MyHome & any other unrelated add-ins prior to starting the migrations. If possible I disable virus checking & any real-time execution monitoring software.

The people who read & act on these ideas (& very much understand what our customers go through) have no input on anything related to subscription or move to cloud. If you have comments about those then they are better suited in this forum. To learn from the experiences of others similar to yourselves you may get more response by posting in the normal Inventor forum as opposed to under ideas.

Hope this helps,
Chris

[sth]

Hello Chris,
thanks for this reply. Unfortunately this doesn’t help really because

- I’ve already done a bunch of migrations, I started with Inventor with R1 18 years ago

- Yes, I know the order of migrating files

- I tried to break down the job in small parts and created a Sequential task to combine all snippets

- All not needed add ins are not loaded

- I try to process only ipt and set the number of processes to “8” – no documentation for this available but should work hence the 128 GB are not used more than 10% yet.
The thing is that the process isn’t working acceptable on our most powerful workstation with Windows 10. On an older PC with Windows 7 it runs better (with the same data
and similar tasks).
I’m in contact with the support team and still I hope they have any idea what goes wrong.

Cheers,
Thomas

[sth]

Dear Chris,

we do not use Vault yet. This migration is a preparation task within or Compass ("Productstream Professional") > Vault project.
We won't to load only R2018 format data to Vault due to the fact we know that "zero migration" impacts the performance
for us in an inacceptable way.

This is what I found after the weekend on the Windows 10 machine.

It seems that the close of the Inventor instances does not work stable (like in ancient times where this was an issue for a long time).

For me this is the probably the reason for all the failures.

However, we'll see.

Sorry to use this blog, but it was in response to Frederic to confirm that there's a second one in the great wide world having

trouble with the migration.

Regarding your amazing and awesome cloud products I’ve the feeling that Desktop customers become forgotten. It’s “old fashioned”

Europe and we behave strange from American’s point of view – but we need to earn money with your products.

Wish you all a wonderful day, I still believe that we’ll have an happy end

Cheers,

Thomas