I got it for under 200 Euros from http://www.gp2x.co.uk although you could probably get a better deal from another distributer ( List here: http://wiki.gp2x.org/wiki/Where_to_Buy%3F ).
It uses SD card for memory, things are downloaded either directly to the SD card, or through a usb cable and the GP2x when the card is in the GP2x.
No wireless though.
A full list of specs can be found here: http://wiki.gp2x.org/wiki/GP2X
So far numerous things have been developed for it, emulators, homebrew games, interpreters for other games, useful utilites and so on. An archive for all this stuff can be found here: http://archive.gp2x.de/cgi-bin/cfiles.cgi?0,0,0,0,1