I've been playing with Google SketchUp (http://sketchup.google.com/) and can confirm it's an excellent, easy-to-use program. Its toolset and workflow were designed for architectural design, which is highly flexible; but certain tasks are unusually difficult -- lathing or making a sphere, for example.
It works mostly by extrusion. Draw a rectangle on the baseplane, draw some smaller rectangles, extrude them upward, and you've got walls. Draw rectangles and arcs on the walls, and extrude those sideways. Draw a closed set of line segments around the four walls and a new face springs into existence. Draw a new segment across that face, and you can drag it upward into a peaked roof.
It uses an "inference engine" to guess how you want to model, and it usually guesses well -- you snap to midpoints, extrude faces to the height of adjacent ones, and so forth. Draw a new segment across existing ones, and they're fragmented for independent editing. You can input exact dimensions at any time, but it's not fully vector-editable -- certain shapes can't be modified after they're created, e.g. the number of sides on a circle.
The texturing interface is the best I've ever used, but the fact that most everything is planar makes that easy. :)