I can't say there ISN'T another factor, but every time I have had 2+ tomes (I have tested up to 5 now) the common speed boost has done nothing, and every time I tested 0-1 tome it increased the speed.
Ok, I just tested time for attacks for 0 and 1 tome, 0-3 upgrades, time for 10 shots, upgrades are 15% attack speed:
Time % of Base
No upgrades: 14.18s 100
0 Tomes:
1 Upgrade: 12.11s 85.4
2 Upgrades: 10.22s 72.1
3 Upgrades: 8.72s 61.5
1 Tome:
1 Upgrade: 13.03s 91.9
2 Upgrades: 12.30s 86.7
3 Upgrades: 11.14s 78.6
So it looks like instead of increasing the effect by 50%, the tome is DECREASING the attack speed bonus by 50%. So once I get 2 tomes, it is decreasing the effect by 100%, or no effect at all.
Just for further clarification, the 0 tome 1 upgrade had an additional card as my rare, and 1 common damage upgrade then 1 attack speed upgrade. The 1 tome 1 upgrade value was a tome as my rare, and the common attack speed boost as my only other upgrade.