You can get a horse to ride by talking with the caravan at Riverbank. After that he will give you one horse (you can should your own horse), and after you build a stable. A horse you chosen will be right there in you stable.
After you have a stable, you can catch some rideable monster like woubulk, rapter to ride.