Guild fame works similarly to player fame - when you are a member of a guild and you earn some fame, a portion of the game is added to your guild's fame.
However, the fame you had before you joined a guild is not transferred to the guild and when you leave the guild, the fame is not removed.
This means that only fame that you earn while being a member counts toward the guild fame.