Letrozole - aches and pains continue

I started Letrozole 5 wks ago.   Rheumatism is much worse but tolerable with a lot of walking & major increase in anti-inflammatories.   I've had anxiety/depression for years but Letrazole seems to be making it much worse.    I'm often very anxious in the mornings and this can sometimes go away with exercise or later in the day.   However it seems this anxiety now continues into the evenings  when previously I was able to switch off at night.   I'm talking to my oncologist  but I know his agenda is to keep me on hormone therapy.  Is anyone else feeling this increase in anxiety?   I've got other side effects (thrush in mouth, dry vagina) but it's the anxiety that might be the deal breaker.   Living like this for 5-10 years is a horrible prospect and my GP said  Letrozole has the least side effects so I'm reluctant to try others.  

I did surgery & radiotherapy but refused chemo and I'm struggling with this decision.  I was quite sure when I said no to chemo but this decision is harder to make.   I'm sure my constant anxiety isn't helping me make this decision!  (Grade 2 tumour that had spread to lymph nodes and Endopredict results said 5-10% chance it will return without chemo and endocrine therapy).

I'm new to forums so not sure if I've just made a new post or reply here.