There is no difference between C# and VB.NET aside from the syntax. In fact, you can run VB code through the Microsoft.VisualBasic namespace in C#.
Really it only depends on your preffered language, since both of them use the same managed libraries, and get compiled into the same IL (intermediate language) for .NET.
VB has some easy-to-use features (like the My.Computer or My.Application namespaces) that are handy, but C# has access to all of the same things, if you know where to look. For example, most properties/methods in the My.Application namespace can be found under the System.Reflection namespace under the Assembly class.
Anyway, if you are developing in .NET, do not be afraid of choosing wrong language for the job.