Stone pyramid likes stone,
person statue likes fabric,
moon statue likes food.
There's also some shenanigans with offering a slave and getting them buffed depending on some conditions. Mostly though, it's one of those mechanics where, if you know how it works, you get bonuses, but if you don't, you lose stuff, which is essentially a penalty for being new. Some new mechanic might be a better solution to the shrines.