You have to press in the direction you want to dig AND press Space Bar to dig. You can't dig over stone (The light colored parts of the level) See screen shot for a hint.
Sorry if it wasn't clear: But you use the arrow keys and spacebar to dig in whatever direction you are pressing. You can't dig through the stone, so look for dirt that is not white-ish on top.