A: If the case is as you mentioned, you are excused from
attending Congregational Prayers in the Masjid. A proof for this is Allah's statement:
...and He has not laid upon you in religion any hardship
So keep your duty to Allâh and fear Him as much as you can
Moreover, it is authentically reported that the Messenger of Allah (peace be upon him) stated:
When I command you to do anything, do of it as much as you possibly can.
May Allah grant us success. May peace and blessings be upon our Prophet Muhammad, his family, and Companions.