A Hugo cautionary tale
Apr. 2nd, 2013 05:30 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
calimacYesterday, Cheryl Morgan posted "A Hugo Cautionary Tale", a story concerning Hugo administration that I was involved with. For various reasons I do not wish to attempt to comment there directly, so I'll put my observations over here.
First off, once Cheryl corrected the date (an error that was not originally her fault), the facts are essentially correct. What I should say is first, about the 1994 Hugo Administrators. Though the Hugo Subcommittee does bear legal collective responsibility for its official actions, readers should be informed that Peter and Athena Jarvis did not participate in the decision to relocate nominees between categories. They were essentially the local arrangements subcommittee - in charge of the plaques and the administration end of the ceremony - and did not participate in counting or verifying the nominees. Neither, really, did Kevin Standlee, the source of Cheryl's information. He was our supervisor and liaison with the larger concom. All the actual counting, verifying, and determining of the identities of the nominees and winners was done by myself and Seth Goldberg, and that includes the relocation decision. This got attributed just to me in some of the popular writings about it, because I was the one who spoke of it publicly (Seth preferred to be purely a backroom boy), but it was a joint decision by the two of us. But Seth is no longer with us, so if anyone is going to speak today with authority of the intent behind the decision, it will have to be me.
What we were trying to do was to let the ballot better express the will of the voters by aiming for equity among the nomination thresholds of the short fiction categories, that is, the minimum number of nominations that a story needed in order to make the final ballot. The threshold for Novelette was distinctly higher than that for Short Story, which was so low that only three stories cleared the 5% threshold, and it seemed unfair that two stories in the novelette category with so many nominations should be excluded from the ballot. Fortunately, the WSFS Constitution allowed for a grey zone between each set of fiction categories adjacent in word length, and this permitted us to move enough stories around to achieve a rough equity in the three short fiction category nomination thresholds.
The argument that Mike Resnick made at the time was that this set an absurd precedent, because various Hugo categories receive varying amounts of nominator attention - people pay more attention to novels and movies than they do to fan artists, for instance; sad but true - and thresholds therefore differ. To attempt to reach uniformity across all categories would result in complete arbitrariness of placement between categories. Mike even wrote a fan-fiction story in which the precedent was used to declare that a short story belonged in the Novel category (between which and Short Story there is, in fact, no overlap) and finally got its award from the convention masquerade.
The comic absurdity of this shows the invalidity of Mike's objection. According to the WSFS Constitution, movement among categories is only possible between two whose wordage or running time directly abut one another, and this is necessary because the categories are, in fact, fuzzy sets, and determining exact wordage is in fact a Heisenbergian process. Without a computer file it's impractical to make an exact count of a story of any length, and even with a file, there's the definition of a word to consider. (Favorite example: how many words is "Los Angeles-San Francisco flight"?) The traditional SF print magazines customarily identify stories by categories in the ToC, but many other publications do not, and nominators cannot be expected to make accurate determinations. My experience as Hugo administrator was that nominators constantly put unlabeled stories in the wrong categories. Unless there was an obvious attempt at stuffing the categories (e.g. putting obvious short stories in the Novelette blanks because you'd run out of room in the Short Story blanks), Seth's and my practice was to give the nominator the benefit of the doubt and allow these in.
And story lengths may be genuinely doubtful. In 1996, for instance, when Seth and I administered again, Charles Brown of Locus insisted to me that Connie Willis's Remake was less than 40,000 words and therefore belonged in Novella, but every time I made a word count of it by counting sample pages and extrapolating, it came out well over 40,000, so into Novel it went.
Mike Resnick defended his position that the categories were non-commutative by claiming that a short story and a novelette are fundamentally different things, not just longer or shorter stories. That is not the impression I've gotten from what other masters of the genre have said on the subject of story length, but I'm not here to adjudicate and there is a sense in which Mike may be right. But his position against relocation amounted to a claim that, if the cutoff between Short Story and Novelette is 7,500 words, then a story of 7,499 words is an entirely different kind of beast than a story of 7,501 words, and I don't think that's a tenable argument. If it isn't, then there must be a grey or fuzzy zone of some size, and we're back to the possibility of relocation. And, again, my experience seeing, for instance, novelettes nominated in everything from Novella to Short Story has convinced me that the voters just don't see the categories as different in kind.
What Mike says today is that it's unfair to make a short story "compete against a novelette that is naturally going to be more complex since it contains close to twice the wordage." Again, there may be truth in this (someone else will have to determine if the longest stories in a category usually win), but any injustice in this is built into the categories already. Novella runs from 17,500 to 40,000 words; the longest novella is over twice as long as the shortest. Novelette has a similar gap. As for Short Story itself, some authors, like Fredric Brown, have written classic SF short-shorts of less than 150 words. The top of Short Story is 7,500 words. That's fifty times as long as a 150-word story.
Yes, Mike might say, some yawning gaps are inevitable. But only so far: some gaps are just too much. Very well, maybe they are. But who is to determine how far is allowable, and how far is too much? For the Hugos, that decision must be made by the WSFS Business Meeting and be reflected in the Constitution. And the Constitution of that time provided for a fuzzy zone of 5,000 words on either side of the category limits, and by that rule Seth and I abided.
Subsequently, the BM narrowed the fuzzy zone. That is their right, and if they were unhappy with the results of the process, I encourage them to make whatever changes they see as fit.
Do I acknowledge that the decision to move the stories between categories was a mistake? Well, yes and no. Relocation for this reason was unprecedented, and hard to understand. I first realized that we were in for trouble when I informed the relevant nominated writers of the impending move. One of them asked me if that meant the story had to be cut, and I had to correct this misunderstanding of the process. But then, there's a lot about the Hugo process that confuses a lot of people. SF fans are supposed to be smart people, but many are absolutely baffled by the Instant Runoff final-ballot voting system. Many people have asked me to explain it to them. People with Ph.D.s have asked me this.
We did get a lot of flack for the decision, but then, Hugo administration naturally attracts flack. One furious fellow demanded my resignation for obvious bias in 1996 when I announced during the nomination period that, if the DP nominators chose to pick the movie Apollo 13, we would count it, and not disallow it on the grounds that it wasn't SF. His idea was that we were somehow giving our blessing to the movie. But that ignores that the nominators still had to nominate it. (In the end, it did get a nomination, but lost the Hugo to a B5 episode.) We couldn't control that. All I could do was assure anxious voters who did wish to nominate it that they wouldn't be wasting one of their five nomination blanks. The only bias was towards assuring that it didn't miss nominations from voters who wanted it, but who feared we'd arbitrarily reject their choice.
But what would have happened if we hadn't made the relocation in 1994? The first thing that surely would have happened is that we'd have gotten a lot of flack for a Short Story category with only three nominees in it. And then, when the nomination figures were released after the Worldcon, we would have gotten flack from some of the few people who read such things for such a high cut-off point in Novelette, refusing worthy stories a place on the ballot.
In short, you'll get flack no matter what you do, so you might as well do what you consider, according to your best judgment at the time, to be the right thing in the first place, and that is what Seth and I did.
(Note: I wrote this without the actual nomination statistics of 1994 to hand, thus a necessary vagueness on some points, and the risk of the vagaries of memory.)
First off, once Cheryl corrected the date (an error that was not originally her fault), the facts are essentially correct. What I should say is first, about the 1994 Hugo Administrators. Though the Hugo Subcommittee does bear legal collective responsibility for its official actions, readers should be informed that Peter and Athena Jarvis did not participate in the decision to relocate nominees between categories. They were essentially the local arrangements subcommittee - in charge of the plaques and the administration end of the ceremony - and did not participate in counting or verifying the nominees. Neither, really, did Kevin Standlee, the source of Cheryl's information. He was our supervisor and liaison with the larger concom. All the actual counting, verifying, and determining of the identities of the nominees and winners was done by myself and Seth Goldberg, and that includes the relocation decision. This got attributed just to me in some of the popular writings about it, because I was the one who spoke of it publicly (Seth preferred to be purely a backroom boy), but it was a joint decision by the two of us. But Seth is no longer with us, so if anyone is going to speak today with authority of the intent behind the decision, it will have to be me.
What we were trying to do was to let the ballot better express the will of the voters by aiming for equity among the nomination thresholds of the short fiction categories, that is, the minimum number of nominations that a story needed in order to make the final ballot. The threshold for Novelette was distinctly higher than that for Short Story, which was so low that only three stories cleared the 5% threshold, and it seemed unfair that two stories in the novelette category with so many nominations should be excluded from the ballot. Fortunately, the WSFS Constitution allowed for a grey zone between each set of fiction categories adjacent in word length, and this permitted us to move enough stories around to achieve a rough equity in the three short fiction category nomination thresholds.
The argument that Mike Resnick made at the time was that this set an absurd precedent, because various Hugo categories receive varying amounts of nominator attention - people pay more attention to novels and movies than they do to fan artists, for instance; sad but true - and thresholds therefore differ. To attempt to reach uniformity across all categories would result in complete arbitrariness of placement between categories. Mike even wrote a fan-fiction story in which the precedent was used to declare that a short story belonged in the Novel category (between which and Short Story there is, in fact, no overlap) and finally got its award from the convention masquerade.
The comic absurdity of this shows the invalidity of Mike's objection. According to the WSFS Constitution, movement among categories is only possible between two whose wordage or running time directly abut one another, and this is necessary because the categories are, in fact, fuzzy sets, and determining exact wordage is in fact a Heisenbergian process. Without a computer file it's impractical to make an exact count of a story of any length, and even with a file, there's the definition of a word to consider. (Favorite example: how many words is "Los Angeles-San Francisco flight"?) The traditional SF print magazines customarily identify stories by categories in the ToC, but many other publications do not, and nominators cannot be expected to make accurate determinations. My experience as Hugo administrator was that nominators constantly put unlabeled stories in the wrong categories. Unless there was an obvious attempt at stuffing the categories (e.g. putting obvious short stories in the Novelette blanks because you'd run out of room in the Short Story blanks), Seth's and my practice was to give the nominator the benefit of the doubt and allow these in.
And story lengths may be genuinely doubtful. In 1996, for instance, when Seth and I administered again, Charles Brown of Locus insisted to me that Connie Willis's Remake was less than 40,000 words and therefore belonged in Novella, but every time I made a word count of it by counting sample pages and extrapolating, it came out well over 40,000, so into Novel it went.
Mike Resnick defended his position that the categories were non-commutative by claiming that a short story and a novelette are fundamentally different things, not just longer or shorter stories. That is not the impression I've gotten from what other masters of the genre have said on the subject of story length, but I'm not here to adjudicate and there is a sense in which Mike may be right. But his position against relocation amounted to a claim that, if the cutoff between Short Story and Novelette is 7,500 words, then a story of 7,499 words is an entirely different kind of beast than a story of 7,501 words, and I don't think that's a tenable argument. If it isn't, then there must be a grey or fuzzy zone of some size, and we're back to the possibility of relocation. And, again, my experience seeing, for instance, novelettes nominated in everything from Novella to Short Story has convinced me that the voters just don't see the categories as different in kind.
What Mike says today is that it's unfair to make a short story "compete against a novelette that is naturally going to be more complex since it contains close to twice the wordage." Again, there may be truth in this (someone else will have to determine if the longest stories in a category usually win), but any injustice in this is built into the categories already. Novella runs from 17,500 to 40,000 words; the longest novella is over twice as long as the shortest. Novelette has a similar gap. As for Short Story itself, some authors, like Fredric Brown, have written classic SF short-shorts of less than 150 words. The top of Short Story is 7,500 words. That's fifty times as long as a 150-word story.
Yes, Mike might say, some yawning gaps are inevitable. But only so far: some gaps are just too much. Very well, maybe they are. But who is to determine how far is allowable, and how far is too much? For the Hugos, that decision must be made by the WSFS Business Meeting and be reflected in the Constitution. And the Constitution of that time provided for a fuzzy zone of 5,000 words on either side of the category limits, and by that rule Seth and I abided.
Subsequently, the BM narrowed the fuzzy zone. That is their right, and if they were unhappy with the results of the process, I encourage them to make whatever changes they see as fit.
Do I acknowledge that the decision to move the stories between categories was a mistake? Well, yes and no. Relocation for this reason was unprecedented, and hard to understand. I first realized that we were in for trouble when I informed the relevant nominated writers of the impending move. One of them asked me if that meant the story had to be cut, and I had to correct this misunderstanding of the process. But then, there's a lot about the Hugo process that confuses a lot of people. SF fans are supposed to be smart people, but many are absolutely baffled by the Instant Runoff final-ballot voting system. Many people have asked me to explain it to them. People with Ph.D.s have asked me this.
We did get a lot of flack for the decision, but then, Hugo administration naturally attracts flack. One furious fellow demanded my resignation for obvious bias in 1996 when I announced during the nomination period that, if the DP nominators chose to pick the movie Apollo 13, we would count it, and not disallow it on the grounds that it wasn't SF. His idea was that we were somehow giving our blessing to the movie. But that ignores that the nominators still had to nominate it. (In the end, it did get a nomination, but lost the Hugo to a B5 episode.) We couldn't control that. All I could do was assure anxious voters who did wish to nominate it that they wouldn't be wasting one of their five nomination blanks. The only bias was towards assuring that it didn't miss nominations from voters who wanted it, but who feared we'd arbitrarily reject their choice.
But what would have happened if we hadn't made the relocation in 1994? The first thing that surely would have happened is that we'd have gotten a lot of flack for a Short Story category with only three nominees in it. And then, when the nomination figures were released after the Worldcon, we would have gotten flack from some of the few people who read such things for such a high cut-off point in Novelette, refusing worthy stories a place on the ballot.
In short, you'll get flack no matter what you do, so you might as well do what you consider, according to your best judgment at the time, to be the right thing in the first place, and that is what Seth and I did.
(Note: I wrote this without the actual nomination statistics of 1994 to hand, thus a necessary vagueness on some points, and the risk of the vagaries of memory.)
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2013-04-03 04:08 am (UTC)At least most people don't claim that last. Except that it's always apparently more likely that there's a conspiracy than maybe the voters don't agree with the complainers' personal opinions. Sigh.
no subject
Date: 2013-04-03 10:29 am (UTC)no subject
Date: 2013-04-03 10:41 am (UTC)no subject
Date: 2013-04-03 05:02 am (UTC)1. Icon is not a reaction to content.
2. As will be obvious, I'm not familiar with the mechanics of the Hugo nomination process.)
The threshold for Novelette was distinctly higher than that for Short Story, which was so low that only three stories cleared the 5% threshold
DOES NOT COMPUTE. (A.k.a. "Wha'?") The Short Story threshold was so low that only three stories cleared it? Not to clear a threshold means it's high, at least relative to your ability, or something. This sounds like a limbo bar. Something needs explanation here.
("Was it not obvious?" From C.J.Cherryh's Chanur series, I think Pride of Chanur. The implied unspoken answer is "IN A MAHEN HELL, NO!!")
no subject
Date: 2013-04-03 10:19 am (UTC)no subject
Date: 2013-04-03 05:31 pm (UTC)a level, point, or value above which something is true or will take place and below which it is not or will not (Merriam-Webster, 3b)
I still don't understand what criterion is meant in "The threshold for Novelette was distinctly higher than that for Short Story".
no subject
Date: 2013-04-03 06:00 pm (UTC)In this case, N=94. "The threshold for Novelette was distinctly higher than that for Short Story" means simply that N was a higher number for Novelette than it was for Short Story.
no subject
Date: 2013-04-03 06:23 pm (UTC)So, let me see. Is this correct?: There were so many nominees for Best Short Story, and so few nominations per nominee, that only three nominees cleared the 5% barrier (were named on at least 5% of the ballots) to reach the final ballot.
no subject
Date: 2013-04-03 06:31 pm (UTC)In the Hugos, the threshold is determined by criteria that are set in advance, and the specific numbers are generated by the data. "The fifth-highest nominee" as a concept is not a threshold, but the number of nominations that a specific fifth-highest nominee gets is a threshold.
If you don't like the word "threshold" in that context, then download all my comments into a Word document and perform a global search-and-replace with a word of your choice that means the same thing, and then reply to that. I'm not going to play word games with you.
no subject
Date: 2013-04-03 09:03 pm (UTC)Once you've said "fifth-highest nominee", the number of nominations doesn't matter.. The criterion "top five nominees" is the same and unchanging no matter what category, no matter how many nominees and nominations. Talking about the number itself is an unnecessary extra step.
no subject
Date: 2013-04-03 09:13 pm (UTC)The number of nominations required to make the ballot is significant when the topic is the difference between that number in one category and the equivalent number in another category, and that was the relevant topic in this post.
no subject
Date: 2013-04-03 09:20 pm (UTC)