I'm not really clear on why bosses are considered an almost mandatory component of videogames. At the end of a level, or a set of levels, we almost always see a boss, particularly with console games. It does have a couple of consequences:
Game tension changes. Beating up minions and exploring a level tends to be something of a rote, routine activity. Bosses ramp up the difficulty and provide an area that requires more concentration to beat. From a game difficulty perspective, bosses are almost always intended to be harder than the gameplay sequences before and after it. This rougher difficulty gradient appears to be more satisfying than a constant, gradual increase in difficulty, perhaps because the boss fight provides both an increased challenge to conquer and a reward in the form of a 'breather' in difficulty.
Control game balance. By making a boss sufficiently hard, a game can guarantee that the player has acquired sufficient power-ups. This requires the player to complete a certain number of sub-quests. Sometimes it will just be the final boss that is this difficult. (Chrono Trigger) [Side note: I frequently don't bother to beat the final boss, if it is too hard. I'm too lazy to go back and complete side quests, and I have no interest in whatever final cinematic the designers have pre-rendered for me--if I wanted to see a movie, I'd just go see a movie--I'm just happy to have unlocked the final challenge. But I'm weird that way.]
Allows insertion of a mini-game for variety.
The boss of one level can become the minion of a later level, once the player has powered-up sufficiently. In *Game Design Perspectives* there's an article on introducing new encounters.
Examples: Zelda, Metroid, Eternal Darkness, Final Fight, Spider-Man, Devil May Cry, Jak & Daxter, Half-Life, Doom, Metal Gear Solid, Chrono Trigger, Mario, Shinobi, Ico.
Side Note about Bosses from another Game Theorist:
Bosses typically block the way of a reward of some sort, whether it be a new power, or a new level to play on. They also put themselves into the position of being a new goal for the player to defeat, and they fit into the psychological model of positive reinforcement as far as consistent positive rewards go. (I know that when I reach the end of this level, I'll fight the boss. I know if I defeat this boss, I'll get a reward, whether it be level 2, or the bow with some arrows.)
CategoryGoodIdea CategoryLevelDesignPatterns?