No need to remove the cam. In fact, you're supposed to install the cam seal with the cam installed, and the cam caps in place! You shouldn't even need to remove the cam cap to take the old ones out.
Get a hook set, something along these lines:
Use those to get the cam seal out. Just make sure you don't gouge the cam seal bore.
Taken straight from the FSM:
NOTE: Bearing end caps must be installed before seals can be installed.
Install camshaft seals into cylinder head until flush with head.
Don't push them in further than flush!