Hello!
I was tempted to answer you before because I'm working at the moment in a simple RTS game with ships and planes, (but I'm still in the protoype phase and actually, reestructuring a lot of code because I decided to jump into Bolt for AI design), but I didn't because IMHO you were not asking for something concrete and asking even for the research part... tsk tsk tsk.
How deep you want to simulate? Is it top view or side-view? Is it actually more arcade oriented as 'Wings of Fury'?
Or maybe something completely different, like 'Achtung Spitfire!', a top view turn-based game where you command an air group in WW2 with a lot of emphasis in the simulation through abstraction:
"What about the turning and avoid being tailed? I wanna do Ai vs Ai if possible with my player object so there will be big battles?
Like how a object would react if it was tailed or how sharp the turns will be? "
Now that you put some specific questions on the table is way easier to help you. But it could be easier if you start defining your idea and what tools will you be using.
This is just some basics, but for turning, I'd recommend you set your planes with variables to set turning speed and maximum turning, so you can calculate in game how agile your planes are. And of course, things like engine power, mass/weight and ascent rate if you want to get deep in the simulation part.
(I'll add at the end of the post a GIF of what I had a month ago related to dogfighting so you can see what can be achieved).
About how they react, I implemented it this way: every plane has a visual range (I'm using Unity, so for this I use a collider and its events), as a circle around the plane, if another plane gets in visual contact (enter/exit collider) and its faction ID is hostile, the planes will turn each other to try to make a kill. And added to that, a list of contacts in range (yes, somehow you may need to code it so every plane will know what's in range and what not).
Then you asked something very interesting: how to react if the plane's under attack. For that, I'm using messages between objects. It's like they are telling each other if they are in their sights or not or firing or not: "hey man! I got ya in me sights! I'm gonna hurt ya!' "No way! I'll turn the other way around". I'm using messages even for projectiles, so they know whom they should hit and who fired them.
As I commented before, I'm adding a very badly done GIF (scaled down and faster than it should be) with an earlier version of the prototype I'm working on (or fighting against). At the beginning of the video, I spawn two fighters from a carrier that will start a defensive combat patrol, a few moments later I'm spawning three enemy fighters mid-air that will engage the carrier's fighters. You'll see that the behaviour is very primitive, they just turn to the closest one trying to find the shortest angle and they do not try nothing fancy.
I hope you can find some inspiration on some of the old games that tried so many strange ideas like 'The Ancient Art of War in the Skies', mixing strategy and arcade.
Or maybe not so old like this one:
SteamBirds
Cheers!!