<?xml version="1.0" encoding="US-ASCII"?>
<!-- This template is for creating an Internet Draft using xml2rfc,
     which is available here: http://xml.resource.org. -->
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!-- One method to get references from the online citation libraries.
     There has to be one entity for each item to be referenced. 
     An alternate method (rfc include) is described in the references. -->

<!ENTITY RFC2014 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2014.xml">
<!ENTITY RFC2119 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml">
<!ENTITY RFC3833 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3833.xml">
<!ENTITY RFC1918 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.1918.xml">
<!ENTITY RFC3330 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3330.xml">
<!ENTITY RFC3927 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3927.xml">
<!ENTITY RFC5156 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5156.xml">
]>
<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>
<!-- used by XSLT processors -->
<!-- For a complete list and description of processing instructions (PIs), 
     please see http://xml.resource.org/authoring/README.html. -->
<!-- Below are generally applicable Processing Instructions (PIs) that most I-Ds might want to use.
     (Here they are set differently than their defaults in xml2rfc v1.32) -->
<?rfc strict="yes" ?>
<!-- give errors regarding ID-nits and DTD validation -->
<!-- control the table of contents (ToC) -->
<?rfc toc="yes"?>
<!-- generate a ToC -->
<?rfc tocdepth="3"?>
<!-- the number of levels of subsections in ToC. default: 3 -->
<!-- control references -->
<?rfc symrefs="yes"?>
<!-- use symbolic references tags, i.e, [RFC2119] instead of [1] -->
<?rfc sortrefs="yes" ?>
<!-- sort the reference entries alphabetically -->
<!-- control vertical white space 
     (using these PIs as follows is recommended by the RFC Editor) -->
<?rfc compact="yes" ?>
<!-- do not start each main section on a new page -->
<?rfc subcompact="no" ?>
<!-- keep one blank line between list items -->
<!-- end of list of popular I-D processing instructions -->




<rfc category="bcp" docName="draft-irtf-asrg-bcp-blacklists-04" ipr="full3978">
  <!-- category values: std, bcp, info, exp, and historic
     ipr values: full3667, noModification3667, noDerivatives3667
     you can add the attributes updates="NNNN" and obsoletes="NNNN" 
     they will automatically be output with "(if approved)" -->

  <!-- ***** FRONT MATTER ***** -->

  <front>
    <!-- The abbreviated title is used in the page header - it is only necessary if the 
         full title is longer than 39 characters -->

    <title abbrev="DNSBL BCP">Guidelines for Management of DNSBLs for Email</title>

    <!-- add 'role="editor"' below for the editors if appropriate -->

    <!-- Another author who claims to be an editor -->

    <author fullname="Chris Lewis" initials="C.R." surname="Lewis">
      <organization>Nortel Networks</organization>

      <address>

        <email>clewis@nortel.com</email>

        <!-- uri and facsimile elements may also be added -->
      </address>
    </author>

    <author fullname="Matt Sergeant" initials="M." surname="Sergeant">
      <organization>MessageLabs, Inc</organization>

      <address>

        <email>msergeant@messagelabs.com</email>

      </address>
    </author>


    <date month="July" year="2008" />


    <area>General</area>

    <workgroup>Anti-Spam Research Group - IRTF</workgroup>

    <!-- WG name at the upperleft corner of the doc,
         IETF is fine for individual submissions.  
	 If this element is not present, the default is "Network Working Group",
         which is used by the RFC Editor as a nod to the history of the IETF. -->

    <keyword>DNSBL policy</keyword>

    <!-- Keywords will be incorporated into HTML output
         files in a meta tag but they have no effect on text or nroff
         output. If you submit your draft to the RFC Editor, the
         keywords will be used for the search engine. -->

    <abstract>
       <t>The rise of spam and other anti-social behavior on the Internet has
       led to the creation of shared DNS-based lists ("DNSBLs") of IP
       addresses or domain names intended to help guide email filtering.
       This memo discusses guidelines for the management of public DNSBLs by
       their operators as well as for the proper use of such lists by mail
       server administrators (DNSBL users), and it provides useful background for both
       parties.  It is not intended to advise on the utility or efficacy of
       particular DNSBLs or the DNSBL concept in general, nor to assist end
       users with questions about spam.</t>

      <t>The document will seek BCP status. Comments and discussion of this
      document should be addressed to the asrg@ietf.org mailing list.</t>
    </abstract>
  </front>

  <middle>
    <section title="Introduction">
      
      <section title="DNS-Based Reputation Systems">
        <t>Due to the rising amount of spam and other forms of network abuse on
        the Internet, many community members and companies began to
        create, publish and maintain DNS-based reputation systems (DNS-Based Lists)
        of IP addresses or domain names and make reputation suggestions or assertions
        about email sourced from these IP addresses or domain names.</t>

        <t>The first DNS-based Lists were almost exclusively
        intended to be used (by email administrators) as lists of abusive
        IP addresses
        to block, however the DNS publication method has proven to be so robust,
        popular and simple to use, that it has been extended for use in many
        different ways, far beyond the imaginings of the designers of DNS or DNS-based
        blocking IP lists.  For example, today, the same basic
        DNS-based listing technology is commonly used for:</t>

        <t>
           <list style='hanging'>
               <t hangText="DNSWL">
               listings of well-behaving email source IP/domain addresses (whitelist).</t>
               <t hangText="RHSBL">
               listings of well/ill behaving email source domain names (often
               applied against the domain name part of the originating email
               address or DNS PTR (reverse IP) lookups)</t>
               <t hangText="URIBL">
               listings of well/ill behaving web link domain names or host names
               used in email</t>
           </list>
        </t>

        <t>Further, the DNSBL user using the list doesn't have to use a listing
        as a pass/fail binary decision, it can use a listing as one factor in
        email filters that make decisions based on scoring multiple
        factors together.</t>

        <t>The DNS-based list technology has even been extended to purely
        informational purposes.  For example, there are implementations that return
        results based on what geographic region an IP/domain is putatively allocated
        in, implementations that translate an IP/domain address into a ASN number
        and/or
        allocation block, implementations that indicate whether the queried domain name
        is registered through a given Domain registrar, implementations that return
        aggregate numeric reputation for an IP address or domain name from another system's email
        system, and so on.  The possibilities are virtually endless.</t>
           
        <t>As well, DNS-based listing technology has also been used in areas
        other than email filtering, such as IRC, web access control, and
        transaction verification.</t>

        <t>As the terminology in this area has never been well formalized,
        often overlaps, and lacks precision, this document has been written
        to use the term "DNSBL" to refer to DNS-based lists generally, not just
        DNS-based block (or black) lists.  This document is not applicable
        to some DNSBLs in some areas, will be mentioned as
        appropriate, but it is the authors' belief that most of the practises
        are applicable to almost all DNSBLs.</t>

	<t>DNSBLs may be either public or
        private.  A public DNSBL makes its data available to any party
        seeking information about data on the list, while a private DNSBL
        is used solely by an organization for its own use and the data is 
        not made available publicly.  There are also commercial DNSBLs,
	  available for a fee.
	  Furthermore, some are free yet require a fee for higher numbers of 
	  queries or certain classes of DNSBL users.
	</t>

        <t>The first publicly available DNSBL using the Domain Name System (DNS)
        for distributing reputation data about email senders emerged in 1997,
        shortly after spam became a problem for network operators and email
        administrators.  This pioneer DNSBL focused on identifying known 
        spam sources situated at static (unchanging) IP/domain addresses.  Due to the broad 
        adoption of this DNSBL, it had a major impact on static spam 
        sources.  Consequently, abusers found other methods for distributing 
        their spam,
	  such as relaying messages through unsecured email servers 
        or flawed formmail scripts on web pages.  Additional DNSBLs were 
        developed by others in order to address these changing tactics, and 
        today more than 700 public DNSBLs are known to be in operation.</t>
   					
        <t>These DNSBLs vary widely in purpose for which the list was intended,
        the method the list uses to achieve the purpose, the integrity of those
        overseeing the method, and the stability of the technology used 
        to create and distribute the data.
        Listing criteria can sometimes be quite
        controversial, therefore this document deliberately does not discuss the
        rightness or wrongness of any criteria.   We assert that DNSBL
        operators are free to choose whatever listing criteria they
        wish, as long as those criteria are clearly and accurately communicated.
        It is the responsibility of the DNSBL user to ensure that
        the listing criteria and other aspects of a DNSBL meets their
        needs.</t>

        <t>This document is intended to provide guidance to DNSBL
        operators so that they may be able to identify what features
        users would be interested in seeing as part of a high-quality, well-managed
        DNSBL, for example, a clear listing and delisting policy to which
        the DNSBL operator adheres strictly.  This document is intended 
        to be normative rather than prescriptive: it seeks to characterize 
        the features of a well-managed DNSBL rather than setting out rules 
        for how DNSBLs should be operated.</t>

        <t>This document is not intended as a protocol specification of DNSBL
        queries.  (See <xref target="DNSBL-EMAIL" />.)</t>
    </section>
    <section title="Guidance for DNSBL Users">
      <t>When choosing to adopt a DNSBL, a DNSBL user SHOULD keep the
      following questions in mind:</t>
      <t>
      <list style="numbers">

         <t>What is the intended use of the list?</t>
         <t>Does the list have a web site?</t>
         <t>Are the list's policies stated on the web site?</t>
         <t>Are the policies stated clearly and understandably?</t>
         <t>Does the web site function properly, e.g., hyperlinks?</t>
         <t>Are web pages for removal requirements accessible and working properly?</t>
         <t>How long has the list been in operation?</t>
         <t>What are the demographics and quantity of the list's user base?
            In other words, do other sites like my own use this DNSBL?</t>
         <t>Are comparative evaluations of the list available?
            Note: all such evaluations depend on the mail mix used
            as well as local policy.  DNSBL users SHOULD consider trial
            periods and/or ongoing local monitoring of DNSBL suitability.</t>
         <t>What do your peers or members of the Internet community say
            about the list?  DNSBLs can sometimes be quite controversial and
            sometimes considerable misinformation is spread.  Ensure that the
            opinions are knowledgeable, and reflect similar goals to yours.</t>
         <t>Does the DNSBL have a mailing list for announcing
            changes, outages etc?</t>
      </list>
      </t>

      <t>The DNSBL user MUST ensure that they understand the intended use of
      the DNSBL.  For example, some IP address-based DNSBLs are appropriate
      only for assessment of 
      the peer IP address of the machine connecting to the DNSBL
      user's mail server, and not other IP addresses appearing in an email
      (such as header Received lines or web links), or IRC connections etc.
      While a DNSBL user may choose to ignore the intent of the DNSBL, they
      SHOULD implement any variance in compliance with the DNSBL usage
      instructions.</t>

      <t>For example, one of the requirements of some DNSBLs is that if the DNSBL is used
      contrary to the usage instructions, then the DNSBL user should not identify the
      DNSBL being used.  Furthermore, it is the DNSBL user's responsibility to
      mitigate the effect of the listing locally.</t>
   
      <t>It is the responsibility of the system administrators who adopt one 
      or more DNSBLs to evaluate, understand, and make a determination of
      which DNSBLs are appropriate for the sites they administer.  If you
      are going to allow a third party's information to guide your filtering
      decision-making process,
      you MUST understand the policies and practices of those third 
      parties because responsibility for filter decisions remains
      ultimately with you, the postmaster.
      </t>

      <t>A DNSBL without DNSBL users does not
      block (or otherwise impair) email or any other Internet service.
      A DNSBL user voluntarily uses the DNSBL data to guide their decisions, and the user
      therefore MUST assume responsibility for the consequences.</t>

      <t>DNSBL operators are expressing an opinion through the
      publication of a DNSBL.  However,
      it is through abiding by the guidelines set forth in this BCP that
      the operators of a DNSBL may gain the trust of their users.</t>

      <t>These guidelines only address public DNSBLs and do not apply to
      private access DNSBLs.</t>

    </section>

      <section title="Requirements Language">
        <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
        "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
        document are to be interpreted as described in <xref
        target="RFC2119"/>.</t>
      </section>
      <section title="Background">
      <t>The Anti Spam Research Group (ASRG) was chartered to address the
       spam problem. The ASRG charter includes:</t>

      <t>"codification of best current practices in spam management"</t>
	 
      <t>This note falls within that category by listing guidelines for
      management of public DNSBLs. This document will seek BCP status.</t>

      <t><list style='hanging'>
        <t hangText="NOTE:">
          This document is a product of the Anti-Spam Research Group
   	    (ASRG) of the IRTF. As per section 3 of <xref
        target="RFC2014"/> IRTF
   	    groups do not require consensus to publish documents.
   	    Therefore readers should be aware that this document
   	    does not necessarily represent the consensus of the
   	    entire ASRG.</t>
        <t hangText="NOTE:">
          This document is intended to evolve, based on comments from
          the Anti-Spam Research Group (ASRG) mailing list. It is
          certain that the current draft is incomplete and entirely
          possible that it is inaccurate. Hence, comments are eagerly
          sought, preferably in the form of suggested text changes,
          and preferably on the ASRG mailing list, at &lt;asrg@ietf.org>.</t>
       </list></t>
      </section>
     
      
    </section>
    <section title="DNSBL Policies">
    
    
      <section title="Transparency">

        <t>A DNSBL SHOULD carefully describe the criteria that are the cause
        for adding, and the criteria for removing an entry from the list.
        Such listing and delisting criteria SHOULD be 
        presented in a clear and readable manner easily accessible to the 
        public on the DNSBL's web site. A DNSBL 
        MUST abide by its stated listing and delisting criteria.  Entries 
        that do not meet the published criteria MUST NOT be added to the
        DNSBL.</t>
  
        <t>In other words, be direct and honest and clear about the listing 
        criteria, and make certain that only entries meeting the published
        criteria are added to the list.  For example, some DNSBL operators
        have been known to include "spite listings" in the lists they
        administer -- listings of IP addresses or domain names associated
	  with someone who has insulted them, rather than actually violating
	  the published criteria for inclusion in the list.
        There is nothing inherently wrong with this practice so long as it
        is clearly disclosed.  For example, a DNSBL described as only listing open
        relays MUST NOT include IP addresses for any other reason.  This
        transparency principle does not require DNSBL operators to
        disclose the precise algorithms and data involved in a listing,
	  but rather the intent behind choosing those algorithms
        and data.</t>

        <t>Furthermore, the DNSBL documentation SHOULD be clear on the intended
        use of the DNSBL - whether it be intended for peer addresses of
        email, IRC, etc.</t>

        <t>Availability of documentation concerning a DNSBL SHOULD NOT be
        dependent on the continued operation of DNS for DNSBL queries.
        </t>

        <t>In other words, if the DNSBL documentation is at "http://dnsbl.example.com",
        the documentation for the web site should not become unavailable if the
        DNSBL query name servers are not available (or shut down).
        See <xref target="limited" />.</t>
          
        <section anchor="listing_criteria" title="Listing/Delisting Criteria SHOULD Be Easily Available">
   
          <t>Listing and delisting criteria for DNSBLs SHOULD be easily
          available and SHOULD be located in a place clearly marked
          in its own section of the web site affiliated with the DNSBL.</t>
   
          <t>DNSBLs often publish their listing criteria along with
          additional technical information about using the DNSBL. This 
          additional technical information can confuse end users, so a 
          separate page, section or query function on its own SHOULD be dedicated to 
          detailing why a specific entry appears in the DNSBL.</t>
        </section>

      <section title="Audit Trail SHOULD be maintained">

        <t>A DNSBL SHOULD maintain an audit trail for all listings and it is
        RECOMMENDED that it is made publicly available in an easy to find
        location, preferably on the DNSBL's web site.  Please note that
        making an audit trail data public does not entail revealing all
        information in the DNSBL operator's possession relating to the
        listing; e.g., a DNSBL operator MAY make the audit trail data
        selectively accessible in such a way as to not disclose information
        that might assist spammers, such as the location or identity of
        a spam trap.</t>
      </section>

      <section title="The Scope and Aggressiveness of Listings SHOULD be Disclosed.">

        <t>Some DNSBLs have adopted policies of listing entries that are
        broader in scope than they have evidence of being involved in abuse.
        Similarly, some DNSBLs list entries that are "mixed", in that the
        entry may be behaving in a manner that is both abusive and non-abusive.
        This is inherent to the techniques that many DNSBLs use.</t>

        <t>Examples: Some DNSBLs will list IP address ranges if there is reason to believe
        that abusive behaviour seen from a few IP addresses within the range is (or will be)
        reflected in the rest of the range.  Some DNSBLs utilize scoring
        to list IP addresses, IP ranges or domain names that have abusive behaviour above some threshold -
        often meaning that some of the email corresponding to the listing is not
        abusive.  Even an entry demonstrably infected with email spam or virus
        emitting malware may emit non-abusive email.</t>

        <t>Inevitably, some of these listings may impact non-abusive
        email.  This has resulted in some labelling such
        practises by the emotionally loaded term "collateral damage".  No
        filtering technique is perfect, and that
        an occasional mistake is inevitable no matter what is used, DNSBLs or otherwise.</t>

        <t>There is nothing wrong with this practise, because mail server
        administrators may wish to implement such policies or use them
        in combination with other techniques (such as scoring).  However,
        a diligent administrator needs information about these policies
        in order to make an informed decision as to the risk and benefit
        of using any particularly DNSBL, and in many cases guide them in how to use it for
        results best reflecting the DNSBL user's requirements.</t>

        <t>Therefore, DNSBL listing policies MUST include statements as to the
        scope and aggressiveness of listings, and include, as appropriate,
        whether the DNSBL operator intends the listings to be used in scoring
        or other techniques.</t>
      </section>

      
    </section>
    <section title="Listings and Removals">   

      <section anchor="temporary" title="Listings SHOULD Be Temporary">

	<t>In many cases, listings can exist for long periods of time past the
        conditions leading to the listing's creation, and/or the listed
        entity has putatively changed ownership.</t>

        <t>Generally speaking, listings SHOULD be considered temporary, and
        should expire on their own at some point in the future unless reasons
        for listing still exist.</t>

        <t>Expiration intervals SHOULD be chosen to be reasonable for
        the type of listing.  For example:</t>
	<t>
	  <list style="numbers">

          <t>It does not make sense to remove entries from DNSBL where the
          existance of an entry is not of direct meaning.  In other words,
          DNSBLs that return information in addition to just existance/non-existance.
          For example: entries in DNSBLs that return geographic or assignment
          information on where the IP address or domain name is located or owned, or DNSBLs that
          return flow statistics from the DNSBL operator that are intended for
          the DNSBL user to interpret, need not ever be removed, just kept
          reasonably current.</t>

	    <t>DNSBLs based on relatively static information, such as
          block assignment or domain names of demonstrably bad actors MAY have
	    very long expiration intervals or only be removed upon request
          after verification that the removal criteria has been met.</t>

	    <t>Automated DNSBLs with highly effective detection and fast
	    listing mechanisms can benefit from very short expiration
	    intervals.  Many of the things that these DNSBLs look for are
	    of relatively short duration, and even if they do expire,
	    a resumption of the behaviour will be caught quickly by the
	    DNSBL's detection mechanisms and relisted.  By utilizing
          a short expiration interval, after reassignment/problem
	    correction, the listing will automatically expire in short
	    order without manual intervention.</t>

	    <t>Manually created DNSBL entries SHOULD be periodically reviewed in
	    some manner.</t>
	  </list>
	</t>

        <t>
      It is RECOMMENDED that DNSBL operators publish in general terms
      their expiration policy is, even if its only "delist on request" or
      no expiration is performed.  In information-only lists, a method for
      users requesting corrections to the information (if appropriate)
      SHOULD be published.

      Abusers may be able
	to "game" plicy that is too explicit;
      on the other hand, many DNSBL users wish to have an
	idea of how "current" the DNSBL is.  It is the authors' experience
	that some automated DNSBLs have increasingly higher error rates as
	the "last detection date" gets older.

      </t>

        <t>Note that listings being temporary does not mean that some
        listings will not remain after the initial timeout period. If the 
        DNSBL operator determines that the conditions triggering listing 
        still exist, then the timer for determining timeouts can be 
        renewed.</t>
      </section>
      <section title="A Direct Non-Public Way to Request Removal SHOULD Be Available">

        <t>Discussions about whether a DNSBL should remove an entry MAY 
        include activity in a public forum.  Methods for processing 
        removal requests through private, direct exchanges, such as 
        person-to-person email or a combination of web page requests 
        and email responses, SHOULD be available.  As a minimum, the 
        DNSBL SHOULD have a web page that has a removal request function 
        (separate from the page describing listing criteria as per 
        <xref target='listing_criteria' />).  The DNSBL SHOULD also 
        make available an email 
        address to handle issues other than blocking issues.</t>
   
        <t>The DNSBL operator MUST NOT use the list in question in such
        a way that removal requests would be blocked, and, moreover,
        SHOULD make mailboxes available in order to allow 
        affected users to submit their requests.  In some cases it is 
        impractical not to filter email to accounts due to the amount 
        of spam those mailboxes receive.  If filtering should be necessary
        in such circumstances, filtering methods with as low
        false positive rate as practical SHOULD be chosen.</t>
      </section>

      <section title="Removals SHOULD Be Prompt">

        <t>The response to removal requests (if the conditions
        for list removal are present) SHOULD be prompt.</t>
        
        <t>A DNSBL MAY impose restrictions on who (e.g. network operator's
        representative or domain name owner) may make valid removal requests.
        However, in many DNSBLs this is inadvisable because it requires
        impractical amounts of effort and is hence NOT RECOMMENDED in most
        cases.</t>
        
        <t>Many DNSBLs (especially those with highly effective detection and
        fast listing mechanisms) greatly benefit from a "no questions asked"
        removal policy.</t>
        
        <t>Although this approach allows people to submit a request and have any
        listed IP/domain address removed immediately, it does not prevent the DNSBL
        operator from re-listing the IP/domain address at a later time.</t>
        
        <t>Many DNSBLs can effectively use a "no questions asked" removal policy
        because by their very nature they will redetect or relist problems
        almost immediately.  They can mitigate more organized attempts to
        "game" the system by elementary checking and rate-limiting
        procedures, increasing lockout periods, rescans etc.  Furthermore, a
        few IP addresses more or less usually do not make a significant difference in
        the overall effectiveness of a DNSBL.  Moreover, a "no questions
        asked" removal policy provides the huge benefit of a swift reaction
        to incorrect listings.</t>
        
        <t>As an example, one popular DNSBL uses a "no questions asked" removal
        policy, but does perform rate-limiting and malicious removal
        detection and mitigation.</t>
        
        <t>Another important consideration supporting a "no questions asked"
        self-removal policy is that it forestalls many conflicts between
        DNSBL operators and organizations whose IP/domain addresses have been
        listed.  Such a policy may be an effective measure to prevent small
        issues from becoming big problems.</t>

      </section>

      <section title="SHOULD Have Similar Criteria for Listing and Delisting">

        <t>The criteria for being removed from a DNSBL SHOULD bear a reasonable
        relationship to the factors that were the cause of the addition to
        the
        DNSBL.  If a listed entity fulfills all published requirements for
        removal from a DNSBL, then the DNSBL operator SHOULD NOT 
        impose any additional obstacles to remove a given entry from
        the DNSBL.  There SHOULD NOT be any extra rules for de-listing
        other than the ones listed in the published listing criteria.</t>
   
      </section>


    </section>
    </section>
    <section title="Operational Issues">

      <section anchor='limited' title="DNSBL Query Root Domain SHOULD be a Subdomain">

        <t>By virtue of using domain names, a
        DNSBL is a hierarchy with a root anchored in the global Internet.

        The DNSBL "query root" SHOULD be below the registered domain name, so 
        that the DNSBL information is not conflated with domain name housekeeping 
        information (e.g., name server or MX records) for the domain name.

        By using this 
        approach, DNSBL queries would take the form of 
        "&lt;query>.dnsbl.example.com" rather than "&lt;query>.example.com".

        Further, this sub-tree should have its own name servers.  Thus,
        the DNSBL query root has its own zone file containing the DNSBL
	information, and the registered domain name has its own name servers containing
	the information (MX records etc.) for the domain name.

        This approach 
        facilitates clear delineation of function as well as
        orderly DNSBL shutdown because the DNSBL nameserver records can be
        specified separately from the domain name's principal nameservers.</t>

	<t>Many DNSBLs support more than one logical zone (DNSBL entries with
	different meanings) that DNSBL users may wish to treat differently
	(or even ignore).  It is RECOMMENDED that, even if there is a single
	DNSBL zone with entry type distinguished by return code, that
	separate subdomain names (of the query root) consist only of the
	corresponding entries.  For example, entry types "A" and "B" might
	return 127.0.0.2 and 127.0.0.3 from the consolidated zone (eg:
	dnsbl.example.com), but there should also be zones
	typeA.dnsbl.example.com and typeB.dnsbl.example.com that contain
	their respective types only.  See also <xref target="updown" />.</t>

      </section>

      <section anchor="provisioning" title="DNSBLs SHOULD be Adequately Provisioned">

        <t>The DNSBL SHOULD have sufficient name server capacity to
        handle the expected loading, and have sufficient redundancy to
        handle normal outages.</t>

        <t>Nameservers SHOULD provide appropriate glue records,
	  possibly in different TLDs to protect against single-TLD issues.</t>

	  <t>If the DNSBL offers zone transfers (in addition to or instead
	  of standard DNSBL query mechanisms), it SHOULD be sufficiently
        provisioned to handle the expected loading.</t>

	  <t>Note that some DNSBLs have been subject to distributed denial
	  of service attacks. Provisioning SHOULD take the likelyhood of
	  this into account, and include plans for dealing with it.</t>

      </section>

      <section anchor="updown" title="DNSBLs SHOULD Provide Operational Flags">

        <t>Most IP address-based DNSBLs follow a convention of query entries for IP addresses in  
         127.0.0.0/8 (127.0.0.0-127.255.255.255) to
         provide online indication of whether the DNSBL is operational.
	   Many, if not most, DNSBLs arrange to have a query
	   of 127.0.0.2 return an A record indicating that the IP address is listed.
         This appears to be a defacto standard.
         (See <xref target="DNSBL-EMAIL" />.)</t>

         <t>If this indicator is missing (query of 127.0.0.2 returns NXDOMAIN),
         the DNSBL should be considered non-functional.</t>

         <t>There does not appear to be a defacto standard for test entries
         within domain name-based DNSBLs.  A number use the same 127.0.0.2
         query test mechanism as IP address-based DNSBLs, and others use a variety of
         domain name-based
         test entries.  Due to the way many
         domain name-based DNSBLs are used (eg: hostname parts of URIs in email bodies),
         using anything likely to appear in a legitimate email is a bad idea (eg:
         http://example.com), especially considering that some email readers will
         transform bare IP addresses or domain name names appearing in the body of
         an email into links.  So, even 127.0.0.2 may be problemmatic.  But a common
         testing method is desirable.</t>

         <t>In the absence of new emerging standards, it is RECOMMENDED that domain name-based
         DNSBLs use a test entry of "_DNSBL_.test".  Test is chosen because it is a reserved
         top-level-domain, and the underscores because it they are generally prohibited in hostnames,
         and are highly unlikely to appear in valid URIs.
         </t>

         <t>Note: In <xref target="down" /> it is noted that some DNSBLs
         have shut down in such a way to list all of the Internet.  Further,
	   in <xref target="reserved" />, DNSBL operators MUST NOT list
         127.0.0.1.  Therefore, a positive listing for 127.0.0.1 SHOULD
         be interpretable as an indicator that the DNSBL has started listing the world and
         is non-functional.</t>
         
         <t>Other results, such as 127.0.0.3, may have different meanings.
	   This operational flag usage and meaning SHOULD be published on the DNSBL's
         web site, and the DNSBL user SHOULD periodically test the DNSBL.</t>

	   <t>Some mail systems are unable to differentiate between these various
	   results or flags, however, so a public DNSBL SHOULD NOT include
	   opposing or widely different meanings -- such as 127.0.0.23 for 
         "sends good mail" and 127.0.0.99 for "sends bad mail" -- within the
	   same DNS zone.
	   </t>

      </section>

      <section anchor="down" title="Shutdowns MUST Be Done Gracefully">
 
        <t>A number of DNSBLs have shut down operations in such a way as
	to list the entire Internet, sometimes without warning.  These
	were usually done this way to force DNSBL users (mail administrators)
	to adjust their DNSBL client configurations to omit the now inoperative
	DNSBL and to shed the DNS query load from the registered domain name
        servers for the DNSBL.  
        Popular DNSBLs are used by tens
        of thousands of sites, are not well monitored, and often not compliant
	with DNSBL query conventions (e.g.: will treat any A record return
	as being "listed",
        instead of specific 127/8 A record returns)
        hence shutdowns (or even ordinary domain name expiration)
        can be quite destructive to all email flow if not done
        properly.</t>

        <t>The DNSBL operator MUST issue impending shutdown
	warnings (on the DNSBL web site, appropriate mailing lists,
	newsgroups, vendor newsletters etc), and indicate that the DNSBL
	is inoperative using the signalling given in
        <xref target="updown" />.</t>

	<t>Only after these warnings have been issued for
	a significant period of time (RECOMMENDED: one or more months),
	should the DNSBL operator finally shutdown the DNSBL.</t>

        <t>The shutdown procedure should have the following properties:
	   <list style="numbers">

	     <t>MUST NOT list the entire Internet</t>

	     <t>SHOULD shed the DNSBL query load from the DNSBL name servers,
	     permitting the registered domain name to continue being
	     useable.</t>

	     <t>SHOULD, perhaps through increased delays, indicate to the
	     Mail administrator that the DNSBL is no longer functional.</t>     

             <t>Name server or query lookups MUST NOT be aimed at third
	     parties unrelated to DNSBL operation.  Such behaviour is
	     similar to inflicting a DDOS attack.</t>

	     <t>The base domain name SHOULD be registered indefinitely,
	     so as to ensure that the domain name doesn't represent a
	     "booby trap" for future owners, and/or provide a means by
	     which a new owner could maliciously list the entire Internet.</t>

        </list></t>

	<t>One way of satisfying the points 1-4 above is to change the
	DNS name servers for the DNSBL to point at "TEST-NET" addresses
	(see <xref target="RFC3330"/>). The below suggested
	<xref target="BIND" /> declarations will cause a DNSBL query
	to query non-existant name servers in TEST-NET addresses, which will result
	in a significant delay (usually more delay as the number of non-existant TEST-NET
      name servers is increased, but not return any A records except in
	very unusual circumstances.</t> 

        <figure>
             <preamble>BIND-equivalent DNS declarations for DNSBL shutdown.
	       </preamble>
             <artwork><![CDATA[
dnsbl.example.com.  604800  IN  NS  u1.example.com.
u1.example.com.     604800  IN  A   192.0.2.1

dnsbl.example.com.  604800  IN  NS  u2.example.com.
u2.example.com.     604800  IN  A   192.0.2.2

dnsbl.example.com.  604800  IN  NS  u3.example.com.
u3.example.com.     604800  IN  A   192.0.2.3

... [as many NS/A record pairs as you like]
             ]]></artwork>
             <postamble>This example assumes that the DNSBL is named "dnsbl.example.com".
	       Replace "example.com" and "dnsbl.example.com" as appropriate
		 for the DNSBL.</postamble>
        </figure>

      <t>
	<list style='hanging'>
	  <t hangText="NOTE:">Of course, the above shutdown procedure
	  cannot be implemented if <xref target="limited" /> is not
	  followed.</t>
        </list>
      </t>
       
      </section>

      <section anchor="reserved" title="Listing of Special and Reserved IP Addresses MUST be disclosed">

        <t>The DNSBL MAY list loopback, <xref
        target="RFC1918"/>, LINK-LOCAL class <xref target="RFC3927">RFC 3927</xref>,
        class D/E, and
        any other permanently reserved or special-use IP addresses <xref target="RFC3330">
	  RFC3330</xref>, <xref target="RFC5156">RFC 5156</xref>.  Such
        use MUST be disclosed in the documentation related to the DNSBL.</t>

        <t>As additional insurance against listings of space that should
        not be listed through testing or other unforeseen events, DNSBL operators SHOULD
        consider implementing facilities to prevent them.  At least one
        popular automated DNSBL has implemented permanent exclusions for
        such addresses.</t>

        <t>A functioning DNSBL MUST NOT list 127.0.0.1.  There are a number
        of mail server implementations that do not cope with this well,
        and many will use a positive response for 127.0.0.1 as an indication
        that the DNSBL is shut down and listing the entire Internet.</t>

      </section>

      <section title="Considerations for DNSBLs Listing Insecure Hosts">
        <t>Some DNSBLs list IP addresses of hosts that are insecure 
        in various ways (e.g. open relays, open proxies).  The following
        recommendations for such DNSBLs may not be relevant to 
        other types of DNSBLs.</t>

        <t>The practise of scanning for vulnerabilities can
        represent a risk in some jurisdictions.  The
	following recommendations for such DNSBLs MAY help
	alleviate this risk.</t>

        <section title="MUST NOT scan without provocation">

          <t>DNSBLs MUST NOT automatically probe for insecure hosts without
          provocation. There is little agreement in the community as to whether
          or not such activity should be allowed, so this BCP errs on the side 
          of caution.</t>
   
          <t>Therefore, scanning MUST be targeted, rather than broad-based, 
          where a given scan is motivated by a specific reason to have 
          concern about the address being scanned.  Examples of such reasons 
          include delivery of an email, delivery to a spam trap address,
          receipt of a user complaint, or periodic testing of an address
          that is already listed.</t>
        </section>

        <section title="Re-scan Periods SHOULD be Reasonable">

          <t>If the DNSBL operator re-scans a host in order to determine 
          whether the listing SHOULD timeout or not, the re-scan period 
          SHOULD be reasonable.  Automated scanning SHOULD NOT occur more
	  often than once every 24 hours.</t>

          <t>It is RECOMMENDED that automated re-scanning should
	  cease within a reasonable period of the vulnerability no
	  longer existing, and targetting conditions no longer being met.</t>

	</section>

        <section title="Scans MUST NOT be Destructive">

	  <t>In the past, some scanning mechanisms have proven to
	  adversely impact the scanned host, sometimes in severe fashion.
	  Scanning methodologies MUST be chosen to NOT negatively impact
	  the scanned host.</t>

        </section>

      
      </section>

      <section title="Removals SHOULD Be Possible in Absence of the DNSBL Operator">

        <t>If removals cannot be automated (e.g., via robot re-testing) then the
        DNSBL SHOULD have multiple administrators so that a removal 
        request can be processed if the principal list administrator is 
        on vacation or otherwise unavailable.</t>
      </section>

      
      <section title="Protect Against Misconfiguration/Outages">
        <t>It is not altogether uncommon for DNSBL users to configure their
        systems improperly for DNSBL queries.  The consequences of an error can
        range from undue (or even damaging) load on the DNSBL servers, to
        accidentally blocking all incoming email.</t>

        <t>DNSBL users MUST test their initial DNSBL configurations to ensure
        that they're working correctly, and SHOULD periodically recheck the
        status of the DNSBLs they use and adjust their configuration as
        necessary.</t>

        <t>Common types of misconfigurations include:</t>
        <t>
	  <list style="numbers">
	    <t>Using wrong (sub-)zones for querying (e.g. 4.3.2.1.example.com
	    or 4.3.2.1.dnsbl.exmple.cm instead of 4.3.2.1.dnsbl.example.com).</t>

	    <t>Downloading a local mirror of the data, but failing to set up
	    the local nameserver infrastructure appropriately, and thus
	    continuing to query the public nameservers.</t>

	    <t>Downloading a local mirror of the data, but misconfiguring
	    the local nameserver infrastructure to query a locally invented
	    zone name (4.3.2.1.dnsbl.local) at the public nameservers.</t>

            <t>Misconfigured local nameservers to not do meaningful
	    caching, thus heavily increasing load on the public nameservers.</t>

            <t>Using the DNSBL query root domain name as the name server
	    for queries.</t>
            <t>Using the DNSBL incorrectly.  E.g. Some DNSBLs are suitable
            only for certain types of filtering.  Improper use may result
            in excessive incorrect filtering.</t>
	  </list>
	</t>
      <t>While in many cases, it can be difficult to detect such situations, to
      protect against such misconfiguration it is RECOMMENDED that DNSBL
      operators make design decisions to mitigate the impact of such
      mistakes, and make efforts to contact administrative contacts to
      remedy the situation where appropriate.  But the DNSBL operator
      SHOULD also prepare to take appropriate steps to protect the
      operational infrastructure (e.g., have the ability to block abusive
      users from causing further damage).</t>

      <t>Appropriate use of the DNSBL (e.g. email, not IRC, not against
      authenticated local users) SHOULD be documented on the web site.</t>

      </section>
  </section>


  <section title="Security Considerations">

    <t>Any system manager that uses DNSBLs is entrusting part of his or her server
    management to the parties that run the lists.
    A DNSBL manager that decided to list 0/0 (which has actually happened) could cause every server
    that uses the DNSBL to reject all mail.
    Conversely, if a DNSBL manager removes all of the entries (which has
    also happened), systems that depend
    on the DNSBL will find that their filtering doesn't work as they want it to.</t>

    <t>If a registered domain name used for a DNSBL is allowed to lapse, or the DNSBL user
    spells the DNSBL domain name incorrectly, the system manager's server management
    is now subject to an entirely different party than was intended.  Further, even if
    there is no malicious intent, some DNSBL query clients will interpret any A record
    being returned as being listed.</t>

    <t>Like all DNS-based mechanisms, DNSBLs are subject to various 
    threats outlined in <xref
        target="RFC3833"/>.</t>

  </section>
  <section title="IANA Considerations">
    <t>This document has no actions for IANA. [This section may be
    removed before publishing as an RFC.]</t>
  </section> 

  </middle>

  <!--  *****BACK MATTER ***** -->

  <back>
    <!-- References split into informative and normative -->

    <!-- There are 2 ways to insert reference entries from the citation libraries:
     1. define an ENTITY at the top, and use "ampersand character"RFC2629; here (as shown)
     2. simply use a PI "less than character"?rfc include="reference.RFC.2119.xml"?> here
        (for I-Ds: include="reference.I-D.narten-iana-considerations-rfc2434bis.xml")

     Both are cited textually in the same manner: by using xref elements.
     If you use the PI option, xml2rfc will, by default, try to find included files in the same
     directory as the including file. You can also define the XML_LIBRARY environment variable
     with a value containing a set of directories to search.  These can be either in the local
     filing system or remote ones accessed by http (http://domain/dir/... ).-->

    <references title="Normative References">
      <!-- Here we use entities that we defined at the beginning. -->

      &RFC1918;
      &RFC2014;
      &RFC2119;
      &RFC3927;

    </references>

    <references title="Informative References">
      <reference anchor="DNSBL-EMAIL" target="http://www.ietf.org/internet-drafts/draft-irtf-asrg-dnsbl-05.txt" >
        <front>
          <title>DNS-based blacklists and Whitelists for E-Mail</title>

          <author initials="J.R." surname="Levine">
             <organization></organization>
          </author>
          
          <date month="May" year="2008" />
        </front>
        <format type='txt' target="http://www.ietf.org/internet-drafts/draft-irtf-asrg-dnsbl-04.txt" />
      </reference>
      <reference anchor="BIND" target = "http://www.isc.org/index.pl?/sw/bind/index.php">
        <front>
          <title>ISC BIND</title>
          <author>
            <organization>Internet Systems Corporation</organization>
          </author>
        </front>
      </reference>
      &RFC3330;
      &RFC3833;
      &RFC5156;


    </references>

    <section anchor="app-additional" title="Acknowledgements">

     <t>We would like to thank John R. Levine, Alan Murphy and Dave
     Crocker for their insightful comments.</t>

     <t>We would also like to thank Yakov Shafranovich and Nick Nicholas
     for editing previous versions of this document.</t>

    </section>

    <!-- Change Log

v02  CRL  Mar 2002Initial takeover from Nick  -->
  </back>
</rfc>
