With the Information you have given it is hard to give any advice/help.
- What is your objective?
- Is this purely self-educational?
- Are you making a Game?
- Do you require network training in the shipped game or just execution?
One thing I learned though:
Use an existing Framework (e.g. TensorFlow, Torch)
- They are well tested and stable
- They come with debugging, visualization.
- Save/load features of your trained model.
- People can help if you are stuck.
I hope it helps you to refine your question.