Holiday is the first one I think of, too. The legendary bartender doesn't work very late anymore, though. A much better place for drinking beer than liquor. Plus, it closes at midnight, so you feel like you've done something special by closing one bar down and being able to find another one open just down the road.
Other good dives: The Subway Inn at 60th Street and Lex. Not to be confused with the late lamented Siberia, which was actually underground. This one is just next to the 59th and Lex Subway station.
Mars Bar. I could look it up, but it's either 2nd Ave or 2nd Street or at the intersection. I usually just stumble in there, against my better judgment. Best in breezy weather because the stagnating air in there is sometimes too much. Good juke.
Marie's Crisis. Gay piano dive. It's possible. Christopher and 7th, downstairs. Great for people who like to drink cheap and sing showtunes.