Many people also found it hard to believe that 'we' were responsible for the ozone hole!
My answer to this general point is always to consider some of the other impacts we've had on the planet - replacing forests by monocultures of plants that suit us, damming rivers to create new lakes, (probably) causing the extinction of many species of megafauna - and the fact the we now live in what are mostly artificially engineered surroundings. When you think about some of these impacts and then the fact that we are measurably altering the composition of the atmosphere, it's not so difficult to imagine that we might also, ultimately, be responsible for the decline in the sea ice.
Also, because something has happened before, it doesn't necessarily mean that when it happens again the cause is the same!