If you want or need to stay in Athens, for a night or more, you have 4 choices: Piraeus, the airport, the coastal suburbs or the city.

A good reason to stay at the port of Piraeus is to be conveniently located to the ferries for an early departure to Milos the next day (there is an 8.00am ferry on most days).  There is a direct bus service from the airport.  There are many hotels within a 5 minute walk of the port.  Try the Acropole or Hotel Delfini

The only accommodation at the airport is the 5 star Sofitel with rooms starting at around 180 euro per night, although they sometimes offer good web deals.

The coastal suburbs of Glyfada and Voula have many hotels and therefore a cheaper alternative to staying at the sofitel and easier to get to than going into the city.  Take the X96 bus, takes half an hour.

If you want to stay in the city of Athens, to see the sights or just to experience the Greek metropolis, then stay in Plaka, the old city at the foot of the Acropolis.

Avoid staying near Omonia or the "train station" areas.  While they are not as seedy as they used to be, they are noisy, far from the sights and lack the charm of Plaka.

For a more information see the Athens Survival Guide or Rick Steves Europe: Athens