torspec/proposals/147-prevoting-opinions.txt

Filename: 147-prevoting-opinions.txt
Title: Eliminate the need for v2 directories in generating v3 directories
Author: Nick Mathewson
Created: 2-Jul-2008
Status: Rejected
Target: 0.2.4.x

Overview

  We propose a new v3 vote document type to replace the role of v2
  networkstatus information in generating v3 consensuses.

Motivation

  When authorities vote on which descriptors are to be listed in the
  next consensus, it helps if they all know about the same descriptors
  as one another.  But a hostile, confused, or out-of-date server may
  upload a descriptor to only some authorities.  In the current v3
  directory design, the authorities don't have a good way to tell one
  another about the new descriptor until they exchange votes... but by
  the time this happens, they are already committed to their votes,
  and they can't add anybody they learn about from other authorities
  until the next voting cycle.  That's no good!

  The current Tor implementation avoids this problem by having
  authorities also look at v2 networkstatus documents, but we'd like
  in the long term to eliminate these, once 0.1.2.x is obsolete.

Design:

  We add a new value for vote-status in v3 consensus documents in
  addition to "consensus" and "vote": "opinion".  Authorities generate
  and sign an opinion document as if they were generating a vote,
  except that they generate opinions earlier than they generate votes.

  [This proposal doesn't say what lines must be contained in opinion
   documents.  It seems that an authority that parses an opinion
   document is only interested in a) relay fingerprint, b) descriptor
   publication time, and c) descriptor digest; unless there's more
   information that helps authorities decide whether "they might
   accept" a descriptor.  If not, opinion documents only need to
   contain a small subset of headers and all the "r" lines that would
   be contained in a later vote. -KL]
  [This seems okay.  It would however mean that we can't use the same
   parsing logic as we use for regular votes. -NM]

  [Authorities should use the same "valid-after", "fresh-until",
   and "valid-until" lines in opinion documents as they are going to
   use in their next vote. -KL]
  [Maybe these lines should just get ignored on opinions.  Or
   omitted. -NM]

  Authorities don't need to generate more than one opinion document
  per voting interval, but may.  They should send it to the other
  authorities they know about, at
     http://<hostname>/tor/post/opinion ,
  before the authorities begin voting, so that enough time remains for
  the authorities to fetch new descriptors.

  Additionally, authories make their opinions available at
     http://<hostname>/tor/status-vote/next/opinion.z
  and download opinions from authorities they haven't heard from in a
  while.

  Authorities SHOULD send their opinion document to all other
  authorities OpinionSeconds seconds before voting and request
  missing opinion documents OpinionSeconds/2 seconds before voting.
  OpinionSeconds SHOULD be defined as part of "voting-delay" lines
  and otherwise default to the same number of seconds as VoteSeconds.

  Authorities MAY generate opinions on demand.

  Upon receiving an opinion document, authorities scan it for any
  descriptors that:
     - They might accept.
     - Are for routers they don't know about, or are published more
       recently than any descriptor they have for that router.
  Authorities then begin downloading such descriptors from authorities
  that claim to have them.

  Authorities also download corresponding extra-info descriptors for
  any router descriptor they learned from parsing an opinion document.

  Authorities MAY cache opinion documents, but don't need to.

Reasons for rejection:

  1. Authorities learn about new relays from each others' vote documents.

  See git commits 2e692bd8 and eaf5487d, which went into 0.2.2.12-alpha:
  o Major bugfixes:
    - Many relays have been falling out of the consensus lately because
      not enough authorities know about their descriptor for them to get
      a majority of votes. When we deprecated the v2 directory protocol,
      we got rid of the only way that v3 authorities can hear from each
      other about other descriptors. Now authorities examine every v3
      vote for new descriptors, and fetch them from that authority. Bugfix
      on 0.2.1.23.

  2. Authorities don't serve version 2 statuses anymore.

  Since January 2013, there was only a single version 3 directory
  authority left that served version 2 statuses: dizum.  moria1 and tor26
  have been rejecting version 2 requests for a long time, and it was
  mostly an oversight that dizum still served them.  As of January 2014,
  dizum does not serve version 2 statuses anymore.  The other six
  authorities have never generated version 2 statuses for others to be
  used as pre-voting opinions.

  3. Vote documents indicate that pre-voting opinions wouldn't help much.

  From January 1 to 7, 2014, only 0.4 relays on average were not included
  in a consensus because they were listed in less than 5 votes.  These 0.4
  relays could probably have been included with pre-voting opinions.

  (Here's how to find out: extract the votes-2014-01.tar.bz2 tarball, run
  `grep -R "^r " 0[1-7] | cut -c 4-22,112- | cut -d" " -f1,3 | sort | uniq
  -c | sort | grep " [1-4] " | wc -l`, result is 63, divide by 7*24
  published consensuses, obtain 0.375 as end result.)