There are two significant changes from city improvements in Civ3. The first difference is that city improvements no longer have upkeep costs in Civ4 -- the city maintenance cost now depends on the number of cities your have, distance from your capital, and your civic choices. The second difference is the new health system. Now, some buildings, such as aqueducts and supermarkets, can improve the health of your city, while some other improvements, such as forges and factories, can reduce your city's health.
The table below lists all city improvements, some of which are related to each of the seven religions in the game, with their own temple, monastery, and cathedral level building.
NOTE: Information in this section is correct as of Beyond the Sword v3.19. For a listing of what the icons mean, please see this page.