According to this page:
http://people.zoy.org/~walken/wgt634u/HOWTO.html
The voltage regulators of your WGT634u are reputed to accept anything in the 12V to 6V range. I'm sure if you search the web you can find more detailed information.
This will assume of course you have the amperage to back that up. The "A" figure is important. If your device uses a 12V 1A wart for example, and you substitute that with a tiny adapter that can supply 12V but only at 200 mA this will not work. A larger battery would have more depth to it as far as supplying current, so would run longer.
I do not know about your device, but the WRT54GL units will run from deep-cycle marine batteries for a week or more. These batteries are quite large though. In general for battery applications, I would look for a router that will accept a wide voltage range. The voltage regulator parts will be labelled and you can look up the numbers. Some of the VRM's in common usage will accept anything from 7V to 40V which gives you a bit more room to play with than the 6V-12V type.