It'd be interesting if it was a larger view and the parts that the wearer isn't currently looking at is merely blurred out. That way you'd just have the in-focus viewport changing and updating while still having a larger sense of what was previously looked at.

Not sure how that'd work for the camera-wearer actually moving around... but yeah.