A latecomer has to follow the Imam in all the actions of the Salah. Thus, when the Imam sits at the end of his second Rak`ah to say Tashahhud, you must do the same even if it is your first Rak`ah. The Prophet (peace be upon him) said,
The Imam is appointed so that he should be followed, so do not be at variance with him.
(Recorded by both Al-Bukhari and Muslim.)
May Allah grant us success. May peace and blessings be upon our Prophet Muhammad, his family, and Companions.