I actually think I already have the answer. Whenever you open up the restart panel you can call "EventSystem.current.SetSelectedGameObject(restartButtonGameObjectName);". This should in theory make the restart button automatically selected.
Source: https://stackoverflow.com/questions/73923902/how-to-select-an-object-by-script-u...
https://docs.unity3d.com/Packages/com.unity.ugui@2.0/api/UnityEngine.EventSystem...