It's a very peaceful game and I love the colorful environments. The stuttering is a problem, but I'm not even sure why it's stuttering - a first guess would be that it's because there are so many objects in the scene. But I've seen Unity scenes run smoothly with many more objects than this. You might want to check your Update() loops to see if you can optimize them.
If I had to spitball a guess, I'd say you're probably calling an expensive operation like GetComponent on many many objects at once in an update loop. If you're using GetComponent somewhere, put it in Start() or only use it when collisions happen - that will cut down on a lot of processing!
Calls to Camera.main have also historically been extremely expensive. That changed in a recent Unity update, but it might be worth investigating just in case.