We need the app.config loaded. When we have Conosle Client App the config is loaded, but when we make a library client ( .dll that we have to NETLOAD ) it is not loaded anywhere and therefore the config is unavailible.
I have solved my problem by pasting the configuration in acad.exe.config. That way my WCF client config is availible the to .dll which is hosted inside AutoCAD.
Here I have found a pretty good intro video for WCF hosted inside AutoCAD. ( Mine was outside of AutoCAD ) :