Since my one day map hack a few months ago I've been doing more work with open source geographic software. There's a lot to learn and I've yet to find a good simple introduction, so here's some notes on resources and how they fit together.
Libraries. There's a lot of core math and data structures for GIS. GEOS implements the basics of OpenGIS geometry: data types, simple calculations, etc. GDAL is the library + utilities for working with raster georeferenced data (maps). GDAL also includes OGR for working with vector data (shapefiles). Finally Proj is the library for translating data from one spatial reference system (SRID) to another. GEOS + GDAL + Proj give you most of the core library functions you need for working with geographic data. They're C libraries and fast but have bindings for more humane languages as well.
Data formats. GeoTIFF is a common format for raster data. There's a zillion others GDAL supports, but GeoTIFF is the native format. Esri Shapefile is a common format for vector data, but you also see a lot of GPX, KML, GeoJSON, etc.
Database. PostGIS is the most common OSgeo database. It adds a bunch of datatypes and functions to Postgres. There are also geo extensions for other databases like MySQL and Oracle. Often a relational database is unnecessary: files + GDAL go a long way.
Desktop applications. Not everyone wants to write code. QGis is a good desktop application for manipulating geographic data. There's also GRASS, OpenEV, and uDig. These programs are focused on displaying and editing geographic datasets, for making and updating custom maps by hand.
Datasets. OpenStreetMap is a huge resource for free data for road maps. geodata.gov is a portal for US government datasets where with enough digging you can find interesting data. There's a lot of other data scattered about, I haven't found a good catalog.
Iím still new to all this so if I misstated or overlooked something please email me.