It deals 30 damage every 3 seconds for 15 seconds(10 DPS), and can stack up to 5 times for 150 damage every 3 seconds totalling 750 damage for the full period. Melee strikes with the Seal active have a chance to refresh the 30 second timer, as well as add another stack. This is unlike some other seals, such as Seal of the Crusader, which have a chance to be refreshed by any melee strike made by the paladin regardless of seal (if any) active. This makes it possible to Judge Crusader, apply Vengeance, and keep both debuffs active at the same time.
Holy Vengeance is modified by 17% of +spell damage, and is holy damage so the effects Judgement of the Crusader apply.
Without citation of Blizzard for the damage co-efficient by the initial poster, the basic formula for this spell is:
Total Damage per tick adjusted for Spell Damage = [P*(D*S+C*K*R^-1)]*fB
Total Damage per Full Period adjusted for Spell Damage = [T*P*(D*S+C*K*R^-1)]*fB
T = Number of tick intervals in total period = R*P^-1 = 15*3^-1 = 5
P = Tick period = 3 seconds
D = Damage per second before bonus damage = 30*3^-1=10
S = Stack total = Variable
C = Bonus coefficient(drawn from WoWWiki) = 17% = 0.17
K = Total spell damage = Variable
R = Total period upon which total spell damage is applied = 15 secs
fB = Function of buffs/debuffs that will increase damage but require individual formula based upon the rules for certain buffs being additive or multiplicative in value.
Total damage equates to (RDS+CK)*fB, but this formula is irrelevant other than as a maximum damage proof, as for concision the per tick equation will suffice.