That's correct, it is a downside. However, you gain more stats with books than cards, so if you can afford less dps, books are better