There is no setup mode button on a PSP. The setup button is found on the LocationFree Base Station. The button is used to establish a connection between the Base Station and the PSP.
The LocationFree Base Station is a device manufactured by Sony that allows users to view TV programs or recorded video from various rooms in a home. The Base Station is connected to a set-top video device and transmits to a client device such as a PSP or personal computer via wireless Internet. Video devices connected to the Base Station can be controlled remotely using the PSP's on-screen remote control function.