So I actually have a question about that... isn't it actually an upside? If you put the book on a damage dealer, you take longer to kill the enemies => you have more time to gain stats => you become way stronger
And in a boss fight you just remove the book and enjoy the carnage.