Well, didn't Ben say that they were all connected with Widmore.
I mean let us face it the more you think about this and watch the shows you find out there are common denominators and if in fact that Widmore came and left the island as he please he and Eloise woudl have had to gain permission for Dharma to enter the island. I have no doubt that Widmore parlayed his knowledge of the island natural and "super"natural resources to get these folks to come out and do experiments while investing in the areas that the islands resources would bring financial benefit.
Widmore is no dummy and he is in the background of many things there is no doubt. Did anyone ever read Stephen Kings the Talisman? The main character, Jack's father is able to flip between these two worlds. Well, the point is that that they gain great power and knowledge from the other world and use to their benefit in the world in which they live.
So for all th time, which again is another undefined variable, Widmore was going back and forth. Building himself an off island family and becoming a very wealthy and powerful man. Of course he networked well and has many powerful and influential friends, ergo the Sundra Trench incident.
Well, sorry but not surprising and Oldham may have figured out what was going on and got off of there.
That whole thing was an odd episode, almost wasted.