The most important consideration for buying a graphics card is the model number, which is derived based on the graphics processor, clock rates and memory bandwidth. Higher numbered/newer models are almost always superior to older models, regardless of other specs.
Memory size is largely irrelevant when choosing a graphics card. Bandwidth is a much more significant factor in the quality of a graphics card. GDDR5 memory offers twice the bandwidth of DDR3 memory. An example given by PC Gamer states that a graphics card with 1 GB of GDDR5 RAM is superior to one with 4 GB of DDR3.
The rest of the equipment on the computer can also create bottlenecks and limit the effectiveness of a higher-end graphics card. Older dual-core central processing units cannot make full use of the most recently released graphics cards, so it is a waste of money to buy anything better than a mid-range card. Lower resolution displays also cannot reach the full potential of the newest graphics cards.
It is also important to ensure that the graphics card is compatible with your system. This includes ensuring that it fits in the case and checking the power supply to ensure that it can connect.