Yes, capsule art is basicly thumbnail/initial image. It’s all the same just called different things by different platforms. (Honestly no clue why)
First impressions are very important, having good art on the first impression tells people you put effort to make the game look good. After that, it’s all about showing off your gameplay.