Anyone who gets upset because you are exercising very necessary self care is making your circumstances all about them. And it's probably not the first time they will have done it or the last if you don't put up some boundaries.
Your guilt about letting people down is something else. BC turns out world's upside down and sometimes we need some help adjusting to our new capacity to care for other people. You can access some counselling which can be very useful--if only because it can illustrate just how 'normal' it is to feel this way.
It's fine to say no. If you do need help--and most of us don't need much, really-- make a list of tasks and give it to anyone who is nagging you. If you don't want to do something, say no, thanks, I'm not up to it.