assert when loading obj file containing utf-8 string

[heelys]

when I load a obj file containing utf-8 character I get the following assert for each utf-8 character:

Debug Assertion Failed!

Program: C:\Perforce\Epic\cpu6\Engine\Binaries\Win64\libfbxsdk.dll
File: minkernel\crts\ucrt\src\appcrt\convert\isctype.cpp
Line: 36

Expression: c >= -1 && c <= 255

with the following callstack:

libfbxsdk.dll!_chvalidator() Unknown
libfbxsdk.dll!fast_check() Unknown
> libfbxsdk.dll!isspace() Unknown
libfbxsdk.dll!fbxsdk::FbxString::UnPad(enum fbxsdk::FbxString::EPaddingType,char) Unknown
libfbxsdk.dll!fbxsdk::FbxReaderObj::ReadLine(char *,char *) Unknown
libfbxsdk.dll!fbxsdk::FbxReaderObj::Read(class fbxsdk::FbxDocument *) Unknown
libfbxsdk.dll!fbxsdk::FbxImporter::ImportProcess(class fbxsdk::FbxDocument *) Unknown
libfbxsdk.dll!fbxsdk::FbxImporter::Import(class fbxsdk::FbxDocument *,bool) Unknown

Is there a way to configure the fbx sdk objreader to make sure it can read utf-8/unicode character? I try with fbx preview and have no issue with the file. With maya it feel like a rename strategy go throught the file and the invalid character get change to something else like: M0034_Стул -> M0034_Ð_Ñ_уÐ_.

Thanks

Alexis

[Numerator]

Can you post chartest.mtl?

[heelys]

Hi Numerator,

sorry I forget the mtl, here is the file.

Alexis

[regalir]

The assertion you received is from Windows 'isspace()' function and I am afrait it cannot be disabled from the FBX SDK.

The FBX SDK OBJ Reader does correctly read the file (and the UTF-8 name) and stores the group name as the ANSI representation of the UTF-8 string, therefore, the string "M0034_Стул" becomes "M0034_Стул"

The assertion is raised because we pass this ANSI string to the FbxString::UnPad() function to remove possible leading and trailing spaces but the character '»" resolves to the integer -69 and isspace is not happy! 🙂

In the FbxReview application you don't see any "problem" because there is no way to see the object name. Maya and 3dsMax have their own OBJ readers and I don't know how they decided to handle UTF-8 encoding. For sure, Maya has more stricter rules about the supported characters in the object names and the '¡', '‚' and '»' are replaced with an underscore.

[piddy]

I'm guessing the bug I found is related.

When I try to load an FBX file from a folder that contains "non-ascii" characters, then it fails.

eg. "C:\temp\Tomaš\Cone.fbx" will fail, however the same object just from "C:\temp\Cone.fbx" will work.

Any idea if this can be fixed? Many Thanks!

[regalir]

Not quite! 😉

The previous post was related to Unicode/WideChar names inside the OBJ file. In your case, it is the file path/name that contains Unicode/WidChar characters. The FBX SDK assumes that all the strings it manipulates are UTF-8 therefore, if you make sure to convert your file path/name to UTF-8 before passing it to the FbxImporter::Initialize(), you should be able to read your file.

for example:

int main(int argc, char** argv)
{    
    char* utf8 = NULL;
                                  // argv[1] is a file path/name (ex: C:\Temp\Tomaš\cube.obj) 
    FbxAnsiToUTF8(argv[1], utf8); // utf8 = C:\Temp\Tomaš\cube.obj
    FbxString fileName(utf8);
    FbxFree(utf8);
    const bool lImportStatus = lImporter->Initialize(fileName.Buffer(), -1, pManager->GetIOSettings()); 
    ...

The FBX SDK provides the following conversion functions:

• FbxUTF8ToWC
• FbxWCToUTF8

and, on Windows platform only:

• FbxWCToAnsi
• FbxAnsiToWC
• FbxAnsiToUTF8
• FbxUTF8ToAnsi

[piddy]

Thanks a million for the example and explanation! Works wonderfully!

I had taken my code from another example so it never crossed my mind it would need converting. Lesson learned!