i think my body shouldn't work out..

normally I'm way healthier than most people about two years ago I thought I'd start working out and the next day I got a cold straight away last december I pushed hard again and got a cold too in Jan‑Feb I worked out a lot, took about a week off, then started again and got either a cold or the flu and have been suffering ever since.. not sure if it's really because of the workouts, but my normally healthy body always gets sick the day after I do a hard or rare workout..

it sounds crazy but I keep doubting it