First of all, if you haven't checked out the student life handout from the visit day student panel, you can find it here: http://www.cs.cornell.edu/projects/phdvisit/gradstudentlife.pdf. Among other interesting things, it has a "heat map" showing what neighborhoods there are and how many students in our department (live there && responded to the survey). The discussion below will refer to areas shown on the map.
The first big question is "on-campus" versus "off-campus". The two Cornell-owned grad housing options are Maplewood and Hasbrouck, and they're not exactly on-campus, but they're within walking distance of Gates and owned by Cornell. See the neighborhood map for their exact locations. The general consensus on these is that they're a safe but relatively expensive choice; they're still affordable given our stipend but you can do much better living off-campus. Cornell's website has more info on these, but I think Maplewood has some one-bedroom apartments that tend to go quickly and lots of 4-bedrooms; I think Hasbrouck only has multi-bedroom apartments, but I don't know for sure. Hasbrouck is generally marketed more towards grad students with families, and has some outdoor play areas for children.
Note that Maplewood will be permanently closed as of Fall 2016. They are building a new grad housing complex on the same site that will open sometime in the future See https://living.sas.cornell.edu/live/wheretolive/gradhousing/ for up-to-date info on grad housing options.
For off-campus options, there are several popular areas. I'll start with places that are within easy walking distance (<20 minutes or so). Pretty much everybody agrees that living in downtown Collegetown (see the map) is a Bad Idea(TM) because it's populated almost exclusively by loud obnoxious undergrads. Basically, the further you are from College Ave the better, up to 4 or 5 blocks. The noise falloff is steeper to the east than to the west, so the area marked "Collegetown" on the map is quieter than Eddy St. Lower Collegetown is quiet and you get a sort of compromise as far as walking up the hill goes - you'll live about halfway (elevation-wise) between Downtown and Gates. East Hill is another popular area - it's a very quiet, nice neighborhood (I think it's a mix of grad students and families). The walk time is about the same (15-20 minutes) and there's not much uphill at all. The north end of East Hill has a shorter walk time than some areas of Collegetown because of its proximity to the Hoy Rd entrance to campus.
These neighborhoods have reasonably convenient access to various bars, restaurants and shops in Collegetown. As a rule these establishments cater to undergrads - you'll find better quality and value downtown, but there are some good places, and many passable places to eat in Collegetown. One notable exception here is the Chapter House (at the intersection of Stewart Ave and Williams St), which is a bar frequented mostly by grad students and locals - people from the department go here many Fridays.
Going a little further away, we get to downtown and Fall Creek. Downtown has very convenient access to the Commons, which contains many of the best restaurants, bars, etc. in town, but will be a bit noisy (as you might expect from living downtown). Fall Creek will be less expensive, still pretty close to downtown, and very quiet. Walk times to Gates range from 20 to 30 minutes, and involve walking up a pretty big hill; however, most people either don't mind (or like!) walking up the hill, or they just use Ithaca's very convenient bus service. The buses running from downtown leave very frequently (every 5 or 10 minutes), and I think you'll never be more than a 10 minute walk from a bus stop (most places will be much less). Bus routes and schedules can be found at www.tcatbus.com. Some people also ride bikes from downtown (and further out), but unless you're insane (and we do have some of these) you probably won't want to commute by bike all the way through the winter.
Many grad students (including some in our department) live a ways North of campus in apartment complexes in the vicinity of Uptown Road (off the map in the handout). This is definitely a bus or bike commute, but there's a direct bus route that runs from there through campus at least every hour.
Most people use Ithaca Craigslist to find houses and apartments for rent. The best time to look for housing is one year in advance, during the fall semester - most landlords rent during the fall for a lease that starts next academic year. The housing market can be very competitive, especially if you want to live near campus, so some places will go on and off the market in a matter of days.
Besides Craigslist, the Cornell Off-Campus Housing Database is a good place to find apartments for rent: http://melian3.campuslife.cornell.edu:7790/och/faces/adf.task-flow?adf.tfId=task-flow-definition6&adf.tfDoc=/WEB-INF/task-flow-definition6.xml
If you want to get here early and need a sublet for the summer (many leases in Ithaca start in August), several current PhD students are subleasing their rooms. Check them out at Summer 2014 Sublets.
Before committing to live in an apartment, it's a good idea to do a background check on the landlord/management company to make sure they aren't exploitative, incompetent, or both. Most current grad students are happy to share their experiences with their landlords if you ask them. There are also several websites that offer reviews of landlords and apartment complexes, and the one with the most information is apartmentratings.com. If you want to write a review but don't see your landlord/property address listed there, you can add it by filling out this form.