If you decide not to use the user interface at all, you should be able to do away with the templates, pages, styles, and js directories, along with a handful of others scattered around (i.e. *Form.inc.php). Off the top of my head, you'll need a few things that aren't currently included in order to administer an installation without the web front-end, e.g. a tool to create and delete archives from the command line. If you delete the web-based administration interface you'll also lose the ability to change your site settings. Since disk space is cheap these days, and since the application isn't enormous, I'd suggest leaving the unused portions on place just in case you need to go back and change your site settings or something similar at a later date.
The Harvester is designed to be a turn-key web-based application, so if you find that it's lacking a couple of details for the kind of deployment that you've got in mind, please feel free to let us know. I suspect that the pieces that you're missing won't be enormous and we'd be happy to expand our support.
I'd also encourage you to look at the current development version, available in CVS (see docs/README-CVS in any current release of the Harvester). It's undergoing a lot of development at the moment and will be shaping up with a lot of exciting new features in the coming months.
Public Knowledge Project Team