Ctrl-b then [ then you can use your normal navigation keys to scroll around (eg. Up Arrow or PgDn). Press q to quit scroll mode.
Alternatively you can press Ctrl-b PgUp to go directly into copy mode and scroll one page up (which is what it sounds like you will want most of the time)
You can also scroll up/down line by line using Shift-k and Shift-j (if you're already in scroll mode).
Excerpts from the man page:
tmux may be controlled from an attached client by using a key combination of a prefix key, ‘C-b’ (Ctrl-b) by default, followed by a command key.
The default command key bindings are:
[ Enter copy mode to copy text or view the history.
Function vi emacs
-------- -- -----
Half page down C-d M-Down
Half page up C-u M-Up
Next page C-f Page down
Previous page C-b Page up
Scroll down C-Down or C-e C-Down
Scroll up C-Up or C-y C-Up
Search again n n
Search again in reverse N N
Search backward ? C-r
Search forward / C-s
Plus a bunch more.
You can set the key binding mode using Ctrl-b, then
:set-window-option mode-keys emacs
or vi
.