grammar - Use of otherwise to indicate contrast

You're confusing two usages of 'otherwise'.

The first is the pragmatic (contrastive sentence connector) marker usage (once, and still by some, called a sentence adverb):

Taking part in the many games and pastimes available nowadays has helped people to engage in a wealth of physical activities. (Otherwise they would not have done so.)

The other is the non-connector contrastive pragmatic marker / *sentence adverb:

Many games and pastimes have helped people to engage in physical
activities they otherwise would not have engaged in.

*I'm at a loss with this one. I like to separate everything that doesn't obviously modify a verb from the 'adverb class', usually into the 'pragmatic marker' mix, but 'otherwise' really connects with the matrix sentence more cohesively than true pragmatic markers do. On the other hand, it sets the frame for engaging / non-engaging rather than adding information about the manner etc of engaging.