I said an escape room game would count (although it is on the line for sure) because escape room games usually do have some sort of plot and the puzzles relate directly to that plot and end up telling some sort of story through the puzzles. Unlike a game like Love Balls which is just a bunch of disconnected puzzles, and has no plot at all, and is at least somewhat twitch based, and plays much like a puzzle platformer, all the things that adventure games are not, thus I said it is the exact opposite of an adventure game.
I will go on to say that I don't know if you could shape your game idea into something that resembles an adventure game. However, as a basic rule of thumb, if you don't know whether or not the game you are planning is an adventure game, it probably isn't.
For people who have played hundreds of adventure games and are extremely familiar with the genre, it would be possible to stretch the definition a bit for their games because they are able to instinctively tell where the line is. However, if you're not as familiar with the genre to the point where you are wondering whether your idea is an adventure game, it's probably a good idea to just stick to the classic adventure game formula: point and click, inventory puzzles, dialog puzzles, and so forth.