?

Log in

No account? Create an account
Brasklapp [entries|archive|friends|userinfo]
Karl-Johan

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Grokking E Pluribus Hugo [Aug. 28th, 2016|04:29 pm]
Karl-Johan
Right now I see a bit of pushback against the newly ratified E Pluribus Hugo rules (see eg Jed Hartman and Rachael Acks). In part this is because the test runs on prior Hugo nominations didn't yield as good results as some may have hoped for, another might be that many fans do not feel they can exactly understand how EPH works. FPTP may be unfair, but it's simple to understand.

At its core, E Pluribus Hugo isn't about selecting the works with the most "support". It's more about selecting the set of works that generates the most voter happiness, where happiness is defined as "getting a work onto the final ballot". I think this framing has gone missing from the discussion.

But in order to help with understanding, no, grokking how EPH works, here is my manually run example. To show how this is done, here is my (hardly random) example set of ballots, with 15 ballots, each with four nominations, and the goal is to reduce the 26 nominees to four finalists. Four of the ballots are also showing very similar taste (due to being on a slate or for some other reason).

I recommend keeping a set of the WSFS agenda from Midamericon 2 at hand, since I'm going to refer to the rules there.


Here are the ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C D 0,25
2 A B C D 0,25
3 A B C D 0,25
4 A B C E 0,25
5 A F G H 0,25
6 F G I J 0,25
7 F I J K 0,25
8 G I J L 0,25
9 J K L M 0,25
10 M N O P 0,25
11 Q R S T 0,25
12 U V X Y 0,25
13 K O T Z 0,25
14 L P R V 0,25
15 N M Q U 0,25


Ballot is the ballot ID; Works is the set of works that the ballot nominates (FIELD3-5 can be ignored); and Points is the number of points that the ballot generates per work, per the calculation phase 3.A.1(1).

This yields the following table, sorted by work.

Work Nominations Points Ballots
A 5 1,25 1, 2, 3, 4, 5
B 4 1 1, 2, 3, 4
C 4 1 1, 2, 3, 4
D 3 0,75 1, 2, 3
E 1 0,25 4
F 3 0,75 5, 6, 7
G 3 0,75 5, 6, 8
H 1 0,25 5
I 3 0,75 6, 7, 8
J 4 1 6, 7, 8, 9
K 3 0,75 7, 9, 13
L 3 0,75 8, 9, 14
M 3 0,75 9, 10, 15
N 2 0,5 10, 15
O 2 0,5 10, 13
P 2 0,5 10, 14
Q 2 0,5 11, 15
R 2 0,5 11, 14
S 1 0,25 11
T 2 0,5 11, 13
U 2 0,5 12, 15
V 2 0,5 12, 14
X 1 0,25 12
Y 1 0,25 12
Z 1 0,25 13


Work is the ID of the work; Nominations is the raw number of ballots with the work on it; Points is the number of points from ballots as used in the selection phase; and Ballots is simply a list with the ballots which list the work, I found it useful for bookkeeping purposes.


Per the old FPTP rules for Hugo nomination, A, B, C, and J would be the nominees. Lets see what happens when we start using EPH.


First comes the selection phase. All of the works with the least points total are selected as possible targets for elimination. E, H, S, X, Y, and Z all have 0.25 points, and are selected. Since they all have the same number of nominations (ie 1), they are tied. Per 3.A.3(4) of the WSFS rules, that means all of them are eliminated.

Round 2 has the following set of adjusted ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C D 0,25
2 A B C D 0,25
3 A B C D 0,25
4 A B C 0,333333333333333
5 A F G 0,333333333333333
6 F G I J 0,25
7 F I J K 0,25
8 G I J L 0,25
9 J K L M 0,25
10 M N O P 0,25
11 Q R T 0,333333333333333
12 U V 0,5
13 K O T 0,333333333333333
14 L P R V 0,25
15 N M Q U 0,25


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 1,41666666666667 1, 2, 3, 4, 5
B 4 1,08333333333333 1, 2, 3, 4
C 4 1,08333333333333 1, 2, 3, 4
D 3 0,75 1, 2, 3
F 3 0,833333333333333 5, 6, 7
G 3 0,833333333333333 5, 6, 8
I 3 0,75 6, 7, 8
J 4 1 6, 7, 8, 9
K 3 0,833333333333333 7, 9, 13
L 3 0,75 8, 9, 14
M 3 0,75 9, 10, 15
N 2 0,5 10, 15
O 2 0,583333333333333 10, 13
P 2 0,5 10, 14
Q 2 0,583333333333333 11, 15
R 2 0,583333333333333 11, 14
T 2 0,666666666666667 11, 13
U 2 0,75 12, 15
V 2 0,75 12, 14


Note that the points from most of the "slate" ballots (ballots 1-3) are unchanged, while the points to each work from other ballots have increased noticably.

N and P are selected in round 2. Since they each have the same number of nominations and points, both are eliminated.

Round 3 has the following adjusted set of ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C D 0,25
2 A B C D 0,25
3 A B C D 0,25
4 A B C 0,333333333333333
5 A F G 0,333333333333333
6 F G I J 0,25
7 F I J K 0,25
8 G I J L 0,25
9 J K L M 0,25
10 M O 0,5
11 Q R T 0,333333333333333
12 U V 0,5
13 K O T 0,333333333333333
14 L R V 0,333333333333333
15 M Q U 0,333333333333333


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 1,41666666666667 1, 2, 3, 4, 5
B 4 1,08333333333333 1, 2, 3, 4
C 4 1,08333333333333 1, 2, 3, 4
D 3 0,75 1, 2, 3
F 3 0,833333333333333 5, 6, 7
G 3 0,833333333333333 5, 6, 8
I 3 0,75 6, 7, 8
J 4 1 6, 7, 8, 9
K 3 0,833333333333333 7, 9, 13
L 3 0,833333333333333 8, 9, 14
M 3 1,08333333333333 9, 10, 15
O 2 0,833333333333333 10, 13
Q 2 0,666666666666667 11, 15
R 2 0,666666666666667 11, 14
T 2 0,666666666666667 11, 13
U 2 0,833333333333333 12, 15
V 2 0,833333333333333 12, 14


Q, R, and T are selected in round 3. Since they each have the same number of nominations and points, all are eliminated. (This also makes ballot 11 a sad puppy, since none of the works from that ballot will make it onto the final ballot.)

Round 4 has the following adjusted set of ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C D 0,25
2 A B C D 0,25
3 A B C D 0,25
4 A B C 0,333333333333333
5 A F G 0,333333333333333
6 F G I J 0,25
7 F I J K 0,25
8 G I J L 0,25
9 J K L M 0,25
10 M O 0,5
12 U V 0,5
13 K O 0,5
14 L V 0,5
15 M U 0,5


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 1,41666666666667 1, 2, 3, 4, 5
B 4 1,08333333333333 1, 2, 3, 4
C 4 1,08333333333333 1, 2, 3, 4
D 3 0,75 1, 2, 3
F 3 0,833333333333333 5, 6, 7
G 3 0,833333333333333 5, 6, 8
I 3 0,75 6, 7, 8
J 4 1 6, 7, 8, 9
K 3 1 7, 9, 13
L 3 1 8, 9, 14
M 3 1,25 9, 10, 15
O 2 1 10, 13
U 2 1 12, 15
V 2 1 12, 14


So far, EPH has only eliminated works with few nominees. However, this changes now. Since ballots 10, 12, 13, 14, and 15 only have two works left on them, they give more points to them. Instead, the two works with the least amount of points in round 4 are D and I, each with three nominations. Since they each have the same number of nominations and points, both are eliminated.

This is the key feature of EPH. Ballots with few works remaining on them will gain in strength and make it less likely that the remaining works will have to face elimination.

Round 5 has the following ajusted set of ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C 0,333333333333333
2 A B C 0,333333333333333
3 A B C 0,333333333333333
4 A B C 0,333333333333333
5 A F G 0,333333333333333
6 F G J 0,333333333333333
7 F J K 0,333333333333333
8 G J L 0,333333333333333
9 J K L M 0,25
10 M O 0,5
12 U V 0,5
13 K O 0,5
14 L V 0,5
15 M U 0,5


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 1,66666666666667 1, 2, 3, 4, 5
B 4 1,33333333333333 1, 2, 3, 4
C 4 1,33333333333333 1, 2, 3, 4
F 3 1 5, 6, 7
G 3 1 5, 6, 8
J 4 1,25 6, 7, 8, 9
K 3 1,08333333333333 7, 9, 13
L 3 1,08333333333333 8, 9, 14
M 3 1,25 9, 10, 15
O 2 1 10, 13
U 2 1 12, 15
V 2 1 12, 14


F, G, O, U, and V are all selected in round 5. F and G have three nominations each, while O, U, and V have two each. Per 3.A.1(3), works with the least number of nominations are eliminated, so O, U, and V are eliminated (making ballot 12 a sad puppy).

Round 6 has the following adjusted set of ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C 0,333333333333333
2 A B C 0,333333333333333
3 A B C 0,333333333333333
4 A B C 0,333333333333333
5 A F G 0,333333333333333
6 F G J 0,333333333333333
7 F J K 0,333333333333333
8 G J L 0,333333333333333
9 J K L M 0,25
10 M 1
13 K 1
14 L 1
15 M 1


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 1,66666666666667 1, 2, 3, 4, 5
B 4 1,33333333333333 1, 2, 3, 4
C 4 1,33333333333333 1, 2, 3, 4
F 3 1 5, 6, 7
G 3 1 5, 6, 8
J 4 1,25 6, 7, 8, 9
K 3 1,58333333333333 7, 9, 13
L 3 1,58333333333333 8, 9, 14
M 3 2,25 9, 10, 15


F and G are selected in round 6. Since they both have the same number of points and nominations, both are eliminated.

Round 7 has the following adjusted set of ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A B C 0,333333333333333
2 A B C 0,333333333333333
3 A B C 0,333333333333333
4 A B C 0,333333333333333
5 A 1
6 J 1
7 J K 0,5
8 J L 0,5
9 J K L M 0,25
10 M 1
13 K 1
14 L 1
15 M 1


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 2,33333333333333 1, 2, 3, 4, 5
B 4 1,33333333333333 1, 2, 3, 4
C 4 1,33333333333333 1, 2, 3, 4
J 4 2,25 6, 7, 8, 9
K 3 1,75 7, 9, 13
L 3 1,75 8, 9, 14
M 3 2,25 9, 10, 15


B and C are selected in round 7. Since they both have the same number of points and nominations, both are eliminated. Note that the entire slate but work A has been eliminated now.

Round 8 has the following adjusted set of ballots:

Ballot Works FIELD3 FIELD4 FIELD5 Points
1 A 1
2 A 1
3 A 1
4 A 1
5 A 1
6 J 1
7 J K 0,5
8 J L 0,5
9 J K L M 0,25
10 M 1
13 K 1
14 L 1
15 M 1


Also an adjusted table sorted per work.

Work Nominations Points Ballots
A 5 5 1, 2, 3, 4, 5
J 4 2,25 6, 7, 8, 9
K 3 1,75 7, 9, 13
L 3 1,75 8, 9, 14
M 3 2,25 9, 10, 15


K and L are selected in round 8. Since they have the same number of points and nominations, both should be eliminated, but since that would leave the final ballot at three nominees, 3.A.2 comes into effect.


The final ballot from running EPH would thus be:

A, J, K, L, and M

All the nominators but poor 11 and 12 (or 13%) will have at least one of their nominated works on the ballot, and the slate (numbering 4 out of 15, or about 27%) have one work on the final ballot.

We can compare this with the FPTP system, which would have yielded the following ballot:

A, B, C, J

Here ballots 10, 11, 12, 13, 14, and 15 (40%!) would have had no impact at all on the final ballot.


The calculations of points were done with the help of Numbers for Mac OS, but all selections and adjustments of the eliminated works were made by hand.

My set of CSV files (note: uses ; as separator and , as decimal marker) for import into spreadsheets.
LinkReply

Comments:
[User Picture]From: madfilkentist
2016-08-28 05:46 pm (UTC)
It took me a bit to understand the column headers, but after that it made sense. Thanks!
(Reply) (Thread)
[User Picture]From: kjn
2016-08-28 05:54 pm (UTC)
Thanks for pointing that out, I've added a short explanation about the headers now.
(Reply) (Parent) (Thread)
From: Lise A
2016-09-11 08:45 pm (UTC)

Table headers

It's okay to ignore the table headers field3-5. Don't ignore the actual columns.
(Reply) (Parent) (Thread)
[User Picture]From: nwhyte
2016-08-28 08:52 pm (UTC)
Thanks very much for doing this. I will have to do something similar for the Worldcon website to inform next year's voters.

Two thoughts on your piece. First, it's a bit conveniently neat that the trailing two slate entries cancel each other out of contention on the penultimate round. If the tie between them was broken to eliminate one rather than both, would the slate then get two finalists rather than one? (To be honest, I don't think it's unreasonable that an organised bloc of 27% should get two places out of five - would not be particularly remarkable in an Irish election where 33% would guarantee you a second place in a five-seater.)

Second, it would be helpful to see the candidates ranked by number of *points* in the tables at every stage. It's not all that easy to see what is going on otherwise.

But thanks again for kicking off the explanatory process.
(Reply) (Thread)
[User Picture]From: kjn
2016-08-28 09:09 pm (UTC)
Oh, yes, you're involved in Worldcon 75 too - me and my family are organising the children's programming there.

Yes, I was a bit surprised that works B and C managed to be eliminated in the same round. It wasn't planned - honest! I toyed with the idea of adding another slate ballot after publishing this, but you will always run into edge cases in any proportional voting system. And yes, from what I can tell, if only B survived round 7, then both K and L would have been eliminated in round 8, giving a final ballot of A, B, J, M.

(My favourite edge case might have been in the municipal election in Växjö, Sweden back in 1984 or so. If the Left Party had taken 10 votes from the Social Democrats they'd have taken the last mandate instead of the Centre party (moderately conservative) and with it a majority for their bloc. Likewise, if the Social Democrats had taken 10 votes from the Left Party, they'd also have taken the last mandate instead of the Centre party.)

Feel free to get in touch with me (or liberally steal) when you set out to explain EPH for next year.

As for your second point, yes, it'd help, but I didn't find the way to sort per that column based on the way I had things up in Numbers. At times, you just have to bite the bullet and publish.
(Reply) (Parent) (Thread)
[User Picture]From: theweaselking
2016-08-29 03:55 pm (UTC)
Good explanation. One major problem with it, though:

(This also makes ballot 11 a sad puppy, since none of the works from that ballot will make it onto the final ballot.)

....you probably want to find a different term for "ballot that gets nothing onto the final" than "whining rules-abusive right-extremist poor-taste imbecile who doesn't claim to be a nazi but who sides with nazis."

The Sadly Rabids have deeply tainted the term "sad puppy", especially in the context of the Hugo awards, and calling someone who gets nothing on the ballot a "sad puppy" is adding a deeply personal insult to injury.
(Reply) (Thread)
[User Picture]From: kjn
2016-08-29 04:39 pm (UTC)
*shrug* I try to reclaim the term and toss it back. It's the fhannish thing to do.
(Reply) (Parent) (Thread)
[User Picture]From: theweaselking
2016-08-29 07:39 pm (UTC)
"Reclaim" would probably work better outside of a context specifically about getting works onto the Hugo ballot, IMHO.

(Reply) (Parent) (Thread)
[User Picture]From: madfilkentist
2016-08-29 07:31 pm (UTC)
I don't see anyone here making fabricated accusations of siding with Nazis except you.
(Reply) (Parent) (Thread)
[User Picture]From: kjn
2016-08-29 07:36 pm (UTC)
Please take that discussion somewhere else, if you want to continue.
(Reply) (Parent) (Thread)
[User Picture]From: madfilkentist
2016-08-30 09:56 am (UTC)
Of course. It's your home ground. I just didn't want to let that sort of spitting go unchallenged.
(Reply) (Parent) (Thread)
[User Picture]From: patoadam
2016-09-05 02:40 am (UTC)
Thank you for your informative post. I can see that you put a lot of effort into it. I find EPH quite ingenious, but that doesn't mean that it is the best solution.
(Reply) (Thread)
[User Picture]From: kjn
2016-09-05 09:01 am (UTC)
I'm not so much interested in the best solution as in a good solution that can be implemented within a reasonable timeframe. All possible solutions will have drawbacks, too, including doing nothing.
(Reply) (Parent) (Thread)
From: Lise A
2016-09-11 08:49 pm (UTC)

Plausible ballots

Apparently it's realistic that the slate ballots are filled out and the regular ballots aren't. It would be nice to see that example too.
(Reply) (Thread)
[User Picture]From: kjn
2016-09-11 09:02 pm (UTC)

Re: Plausible ballots

While true, it sort of falls outside what I tried to do here, which was to show how EPH worked behind all the detailed written rules and the big tables and charts that the software generates, that usually has been used to demonstrate the system.

From another perspective, it doesn't really matter anyway, since a ballot with say three popular works on it and two works with hardly any other nominators will have the two latter works eliminated quickly, and then it's just like a ballot with just three nominees on it after the first two-three rounds.

The goal of nominating shouldn't be to give max points to your chosen nominees anyway. It is to maximise the number of works that you and others agree to be worthy finalists. The more works you nominate, the greater chance you have to find those shared nominees.

Edited at 2016-09-11 09:03 pm (UTC)
(Reply) (Parent) (Thread)