Uniquely among the genre fictions, SF is always judged in the popular mind by its very worst examples. This can lead to some less than desirable outcomes...
Popular culture is epitomised by movies and television programmes, and the makers of these things are very familiar indeed with most of the generic categories. They have no difficulty understanding the tropes of detective, crime and mystery fiction; they are very happy with the western; they are fully aware of how to handle a romance story. And so there are many first class examples of all of these things available for your viewing pleasure. Therefore, because everyone is very well aware that excellent works of art can be found in these genres, there is no sense of shame attached to working in them. Consequently, when a writer is contemplating a new novel, they are usually quite happy if their plot and theme requires them to write (say) a detective novel. Well respected mainstream writers such as Kate Atkinson seem perfectly happy to have their books shelved with the other crime novels. And even writers who intentionally restrict themselves solely to genre fiction (Ian Rankin, Peter Robinson, John Harvey and Reginald Hill spring to mind) quietly produce books of genuine literary merit which are often reviewed purely as novels rather than as examples of the genre within which the publisher has categorised them.
Furthermore, sometimes a moribund genre can be completely re-vitalised when a mainstream novelist chooses to go slumming in it. The western/cowboy genre was generally agreed to be a literary dead end with so few authors writing in the field that it was, for all practical purposes, non-existent when Larry McMurtry wrote Lonesome Dove and won a Pulitzer prize for it.
The same cannot be said of science fiction. The popular media don't understand it at all and seem to be completely unaware of exactly what the genre is capable of. They mainly use it as an excuse to produce lots of sound and fury which generally signify nothing very much. And so we have utterly gorgeous spectacles such as the movie Avatar which is at one and the same time both visually stunning and yet so na´ve and simple-minded in the story it tells that it is positively embarrassing to sit through. This happens so frequently that many people simply assume that such shallow triviality is all that the SF genre is capable of. It is well known that the only way to enjoy the vast majority of film and television SF is to turn off your brain and just drool and dribble mindlessly at all the pretty colours.
Curiously, this attitude is now so firmly established in the popular mindset that the SF label seems to be reserved purely for such lowest common denominator, utterly thoughtless, simple-minded banality. When a science fiction film with depth, and subtlety and artistic merit is produced, its purveyors never attach the science fiction label to it. It would probably never even occur to them to do so, trapped as they are in their narrow-minded world view of SF-as-shallow-spectacle. A perfect example of this attitude would be the reaction to Quentin Tarantino's movie Inglourious Basterds. The film is an alternate history story, a legitimate branch of science fiction that has been played with by many people over the years both within and without the genre Mark Twain and Winston Churchill both dabbled in it, as did Philip Roth (not to mention L. Sprague de Camp and Harry Turtledove). But you'll find no mention of that SF aspect in the publicity material and the many critical discussions of the film.
So when a mainstream novelist such as Margaret Atwood chooses to write a story that is thematically SF it is quite understandable that she feels a certain amount of shame at being tarred with that very dirty brush, and she squirms and wriggles and insists that she isn't writing science fiction at all; no, no she's writing something much more serious than that. She's writing speculative fiction; quite a different thing. It's got fully rounded characters, it deals with important and interesting themes, it has things to say about the human condition. It doesn't have any talking squids in space; it can't possibly be science fiction. (She ignores the fact that Robert Heinlein seems to have coined the term 'speculative fiction' to describe the kinds of things he was writing in the 1940s and 1950s, and you don't get any more prototypically SF than Robert Heinlein. But I digress...)
The only reason she does it, of course, is because she has accepted the popular definition of science fiction. Consequently she assumes that this view of SF as an essentially shallow and completely trivial (not to say childish) genre means that, if she allows her own writing to be categorised as science fiction, it will automatically be trivialised and dismissed by her peers. And who can blame her for this attitude when the evidence is so overwhelmingly in her favour?
Also, it doesn't help at all that many SF commentators, hugely vocal, anally-retentive, obsessive-compulsives with an urgent need to categorise and pigeon-hole everything that crosses their tiny attention spans, fall upon her books with glad cries of glee. Seeking legitimacy for themselves and for her, they try immediately to claim her as an SF author and they write pompously nit-picking essays that discuss the derivation and applicability of her themes, comparing her work to SF classics that she has almost certainly never heard of and which she would probably find impossibly boring to read. Perhaps they are unconsciously hoping to redeem themselves by proving to the world that not all the works that inhabit their beloved genre are banal trash.
Wouldn't you run away screaming if such a thing happened to you?
It doesn't have to be that way, of course. Doris Lessing, winner of the Nobel prize for literature, has no problems at all with having some of her novels categorised as science fiction. She is fully aware of the mature and literarily significant work that has been produced by other writers in the genre and she is happy to have made a significant contribution to the field. She's a science fiction fan from way back who thoroughly enjoys the company of other fans and who feels no sense of shame in considering herself a genre writer. Indeed, she has stated quite publicly that she is very proud of her science fiction novels and considers them to be among her best works.
In many ways it's a silly storm in a very tiny teacup. Does it really make any difference what label we use to define a specific work? Surely it is the work itself that matters rather than the category we choose to place it in?
In his guest of honour speech at a science fiction convention many years ago, the writer Gene Wolfe said something along these lines. This isn't a verbatim quote, but it certainly reflects the spirit of what he said:
I just write stuff. I write the story that needs to be written in the way that the story requires itself to be told. It's just stuff. When it's finished, I send it off to my agent and my publisher and they tell me what I've written. Science fiction, fantasy, whatever. They know best how to categorise it; that's their job. My job is different. I just write, and what I write is just stuff.
Given that these two extremes exist in the world, perhaps we can do something constructive with them. Since the science fiction label has apparently been adopted by the popular media and since the perception of it in the public mind is probably now unalterable, let's just give it to them, accept their interpretation of it, and move on. Meanwhile, those people in the SF ghetto whose principle passion is categorisation and pigeon-holing should now add the term 'speculative fiction' to their vocabulary. Wow! An extra category for the filing cabinets of the collective mind! I can practically hear the orgasmic screams already.
And the rest of us, who simply don't care, can just carry on reading stuff but only the stuff we enjoy, of course.