My mum didnt take the pill, had 4 kids and breast fed them all, both of which are meant to be protective. I don't have kids but had periods start at 14 (ie not too young), and went through premature menopause (pre cancer), both of which should have been protective even though I took the pill. Both of us were diagnosed with breast cancer before age 45. Probably genes involved (not the known ones, as I have been tested).
I stopped taking the pill as soon as I knew my cancer was hormone receptive. My surgeon concurred with that decision although she said the low dose of hormones from the pill would not have made much difference.