Nerf is a term that means to be rendered ineffective or less effective by a change in the rules or game system. The term nerf is based on Non-Expanding Recreational Foam, a substance used to make toy weapons. To nerf could be translated as "turn a real weapon into a toy weapon". See official Nerf toys website.
Abilities, strategies and encounters, or even entire classes can be nerfed. Despite the frustrated cries of players personally affected by a nerf, such things are usually necessary to protect the integrity of the game balance as a whole.
A "stealth nerf" refers to a change that was not announced by the developers, and was discovered (much to the dismay of the hapless victims) by the players themselves without any advance warning. Players tend to be especially frustrated with stealth nerfs, for obvious reasons.
The official WoW Class Forums are often filled with so-called "nerf threads"; topics in which the poster states that they consider a certain class to be imbalanced. Perceived imbalances can lead to serious or jocular outcries calling for a nerf, i.e. "NERF RET PALLYS!!!" in-game or in said forum posts. This is at times frowned upon by the rest of the community and can result in responses like Learn2Play.
The opposite of a nerf is a buff. However, "buff" can also refer to a temporary boost in power.