The most common problem when you're trying to build your first data provider is simply getting all the interfaces implemented! Because of the strong interdependency among the interfaces, you won't have the luxury of starting with a small program that compiles, which you can then incrementally "grow."
One way to solve this problem is to make a note of what name you intend on giving each of your interfaces. Another helpful solution is to take a data provider implementation, such as the one in this chapter, and change each object to then work with your custom data source.
Additionally, you can also build in sections and wrap each object in a test case. For example, implement the Connection object first and write a test case that exercises each of the APIs to make sure you're getting predictable results. Then implement the IDbCommand object with a related test case to make it's working as well. Doing this will require a little more coding, but it will likely save you more time in debugging than trying to trace the needle in the haystack.
Finally, if you're serious about debugging your data provider, add trace logging at the entry and exit of each method. This can be as simple as writing to the debug stream or as sophisticated as writing to the Windows event log. This data can be valuable in performing a postmortem on your application when it crashes.