Some suggestions for improvements to the TPE, to help increase transparency and make implementation easier. This is based on building the Chrome and Firefox browser extension Bouncer, which implements the full Javascript API, together with experience creating user consent systems based on the API on European ePrivacy compliant websites.

Scope

Introduction

Sites and third-party servers are not likely to implement DNT (either the W3C TCS or the EFF flavour) unless there is a scalable way to override the DNT general preference on particular domains i.e. to whitelist them. Moreover, the TCS scope section specifically requires browser support for the JavaScript Exceptions or Consent API. The API has proven to be implementable in browsers and browser extensions, but there has been no use of it by a high profile site such as a publisher or third-party advertiser. Until the API is more widely supported the out-of-band mechanism is the only way for domains to be white-listed, but it suffers from a lack of verifiability and transparency.

There have been a few implementations of the Tracking Status Resource or the Tk response. Moreover there also been very few declarations by sites of one of the valid EFF DNT policies at /.well-known/dnt-policy.txt

Some sites such as www.w3.org, pixel.piwik.org and gchq.gov.uk have the Tk response header, but with an incorrect value. This could partly to do with lack of clarity, but also hints at the problems for server side code detecting (in a scalable way) out-of-band consent using cookies. Server side code capable of changing HTTP headers is often maintained by a different team, even a different company, than the team responsible for the logic of a web application. A way for the application team to declare the header using a "meta tag" approach could help here, as it has for other recommendations such as the Content-Security-Policy header.

The GDPR and the principles laid out in the EU-US Privacy Shield arrangements both call for increased transparency. Embedded third-parties now have to be identified and declare their legal basis and purpose for personal data processing. This requirement is a good fit with facilities in browsers, or browser extensions, to present information clearly, which in turn relies on it being available in a standardised machine readable way. The Tracking Status Resource is perfect mechanism for that, but it would help if this could be deployed more flexibly. Giving sites the option to vary the location of the TSR, and to declare it within an HTML document using meta tags would make it easier for sites to deploy it.

This suggested changes are an attempt to address these issues:

• DNT:0 header deliverable using well-known named cookie.

  Origin specific user consent should be implementable by script libraries when the underlying JavaScript API is not implemented by a particular user-agent. Delivering the protocol elements, e.g. may not be possible in a DNT request header but would be possible as the value of an HTTP cookie with a standardised name. We propose using the cookie name __DNT for this. This will allow the user to override their general preference not to be tracked in a standardised and verifiable way. Although this would be simple method to replace the web-wide API, it would not be easy to use it for site-specific consent due to the lack of support for double-keyed cookies.

• New API functionality.

  The new confirmTrackingException and enhanced storeSiteSpecificTrackingException MAY return a Promise with a fulfilment value of TrackingStatusObject. This helps the API be more in line with the asynchronous nature of client side JavaScript, and creates callback functionality so web applications can respond dynamically in cases where the user agent confirms Consent with the user before granting DNT:0.

• useSameParty to replace cookie type subdomain rules

  The use of the domain parameter wastes unnecessary cycles on checking for potential grants for subdomains. The API dictionary now includes a useSameDomain property which, if set true, will signal the user agent that a particular Consent Grant should also apply to the domains referred to by the same-party TSR array property. The user-agent MAY check that the targeted domains contain TSRs that refer back to the host domain of the calling origin.

• Enhanced transparency through extending the Tracking Status Resource

  The new US-EU PrivacyShield requires companies to uphold certain privacy principles, including the right of users to access, amend or erase personal data, the duty to notify users on the purposes for personal data use and the identity of data controllers, and the ability of users to opt-out (or in some circumstances require them to opt-in) for processing. The GDPR (coming in 2 year)s will increase the likelihood these principles will be enforced, and made more robust.

• The current TPE document has the following minor bug

  https://www.w3.org/2011/tracking-protection/drafts/tracking-dnt.html#dnt-header-field "A user agent must not generate a DNT header field if the user's tracking preference is not enabled" Contradicts EU requirement for DNT:0 to override DNT unset.

Tracking Status Resource in the Tk Response Header field

Organisations sometimes have to manage several, perhaps thousands, of websites, often developed and maintained by marketing agencies and other contracted parties. Logistically it can be difficult to manage the multiple servers run by these disparate parties in a coordinated way.

An example of where this is required is delivering a Tk response header, especially when the out-of-band consent (OOBC) mechanism must be used. If a user's browser does not support the DNT Exception (Consent) API then the server must determine from the request if tracking consent has been given by that user, before inserting the correct Tk response header. This will in most cases involve looking up the value of an authentication cookie in an external database, and the code to do this would have to reside in the server, and it, or its underlying logic, would have to be shared among the different parties.

Another example is the delivery and maintenance of the Tracking Status Resource at the "well known" location. Every origin would have to contain a reference (at /.well-known/dnt) to the current resource, which may have to be maintained by the company's legal or data protection function.

While it is possible in the current TPE to target any URI as the declared Tracking Status Resource using a chain of HTTP Redirects, getting all a company's servers to initiate the correct server-side redirection code can be difficult.

Both of these issues could be addressed by explicitly allowing the delivery of Tk responses within the returned HTML content, and communicating an alternative TSR URI within the response. It is relatively easy to supply a JavaScript library reference to be inserted in every HTML head-section, and this is a well understood industry practice.

This mechanism would only take effect in situations where an origin server did not return a Tk header, and where it did not already return a well formed JSON resource to a GET to /.well-known/dnt.

This proposal calls for the ability to specify the Tracking Status Resource within the value of the Tk response header, and be able to declare any Tk response header in an HTML head section using a meta tag with the standard http-equiv attribute.

In future there is likely to be a standardised policy resource for an entire origin (e.g web site) where security and policy oriented headers can be declared. The motivation for this is managing the "header bloat" issue arising from new recommendations such as Content-Security-Policy. It would make sense to append the Tracking Status Resource to this, and incorporating a reference inside a header is a natural way to do it. If every site must have a set of cached origin-wide resources it would be best to combine their functionality so only one extra round-trip was needed, and to minimise the size of the cache.

A valid URI after a "%" character in the Tk header will be construed as a valid Tracking Status Resource, unless a valid resource is also located at {origin server domain}/.well-known/dnt, in whch case that resource is taken as the only valid one.

If a Tk response header exists the meta tag is ignored. If a Tracking Status Resource exists at {origin server domain}/.well-known/dnt then any "%" component of any Tk response , in a header or in the HTML, is ignored.

The new Tk response header is defined formally by the following ABNF

            Tk-field-name  = "TK"
            Tk-field-value = TSV [ ";" status-id] [ "%" URI-reference]
            status-id =  1*id-char
            id-char =  ALPHA / DIGIT / "_" / "-" / "+" / "=" / "/"
            URI-reference is defined as per [[RFC3986]]
         

Tk response header via meta tag

The server MAY supply a Tk response via an HTML meta elements with http-equiv attributes that are an ASCII case-insensitive match for the string "Tk". For example:

<meta http-equiv="Tk" content="N">

<meta http-equiv="Tk" content="C%https://corporate-site.com/.well-known/dnt">

Add the following entry to the pragma directives for the meta element:

Tracking response (http-equiv="Tk")

1. If a Tk response header exists abort these steps.
2. If the Document’s head element is not an ancestor of the meta element, abort these steps.
3. If the meta element lacks a content attribute, abort these steps.
4. Let trackingResponse be the value of the content attribute of the meta element.

Authors are strongly encouraged to place meta elements as early in the document as possible, because policies in meta elements are not applied to content which precedes them.

Note: Modifications to the content attribute of a meta element after the element has been parsed will be ignored.

well-known named cookie to indicate consent

If a user agent does not support the JavaScript API i.e. if navigator.confirmWebWideTrackingException is undefined, and so cannot indicate a DNT exception by sending DNT: 0 to the appropriate origin server, then there should be an alternative method to indicate user consent, i.e. that DNT: 1 has been overridden, by arranging for the user agent to send a cookie with a "well known" name. It is envisioned that a JavaScript library could be installed on a page to be activated if the required functionality is not implemented by a particular user agent.

For transparency and verifiability the cookie should have a standard name __DNT0, MUST have the Secure flag, and must only be placed if the user has given their affirmative consent for web-wide tracking by the origin server, i.e. as the result of the appropriate call to the JavaScript DNT Exception API.

If the server recognises the consent indication it MUST respond with a Tk: C header, or an HTML response containing a in the head section.

The recognition of web-wide consent for origin server is straight forward using this method, as the cookie would be present in every request. For site-specific consent the data needed could be encoded in the cookie's value, perhaps to be matched against the domain specified in a Referer header. If the recognition requires servers held in the server linked by singling-out via a UID this should be encoded here, and in that case the qualifier TSR property MUST include the i code.

Cookie:__DNT0=1a5b43ea7

Tracking Status Representation - Additional Properties

object {
    string tracking;                 // as per the Orderly schema in the TPE
    array { string; } compliance?;   //                 ''                   
    string qualifiers?;              // amended below 
    array { string; } controller?;   // as per the Orderly schema in the TPE
    array { string; } same-party?;   //                 '' 
    array { string; } audit?;        //                 '' 
    string policy?;                  //                 '' 
    string config?;                  //                 '' 
    array { object { string name; string uri} } controllerDescription; // Legal name of the data controller and a URI of a page providing the user with information about it;
    
    object { long overall_retention?; // A positive number of seconds indicating the overall maximum lifetime of the client storage.
             array object { string type ["C","L","I"]; // storage type C = a cookie, L = localStorage item, I = indexdedDB table, "E" = cache item e.g. ETag 
                            string match; // a regular expression used to match an item of storage when applied to the item name or cookie name and value pair
                            array { string ["n","t","i","f","a","u","O","!f","!l","!s","!d"]* purpose;};
                            array { string sharedWith; }; // the main domains names managed by the entities the data may be shared with.
                            long retention?; // A positive number of seconds indicating the maximum lifetime of the storage item.
                          } 
           } storageUse; // matches any client data item e.g. a cookie, item in localStorage etc., used by the origin server
    string deleteDataUri?; // URI of resource that causes all tracking data on this origin to be deleted
    string optInUri?; // URI of resource that causes the origin server or its JavaScript browsing context to register tracking consent
    string optOutUri?; // URI of resource that causes the origin server or its JavaScript browsing context to revoke any tracking consent
    string dataHeldUri; // URI of resource that will return HTML content identifying tracking data held by the origin server,  
                        // giving the user the ability to access, amend or delete it.
    object {integer {,48} bitLength?; integer maxAge}} trackingIdConstraint?; // declared tracking identifier entropy and expiry
    array { string; } currentAudienceSelector?; // array of audience categories derived from the current DNT-identifier
}*;

User-Agent header dntapi client property information

Many browsers have yet to support the JavaScript API and this makes it difficult for servers who want to register user consent to decide on whether to support the API or fall back to the out-of-band mechanism. It is for server side code to insert the Tk header and this is a necessity if OOBC must b used. JavaScript can be used to detect the shortfall, but this creates the necessity for another round-trip to communicate the result. At this time even this is problematic in the case of the Microsoft Edge browser because navigator.confirmSiteSpecificTrackingException exists, and it is not until a call to storeSiteSpecificTrackingException is made, which throws an Access is denied exception, that the client code can detect that the API is not supported.

We propose that Browsers or browser extensions that support the API should indicate this to servers by including the string dntapi in the User-Agent request header. Servers or client code can easily detect this string and immediately implement the appropriate procedures.

API

The [[tracking-dnt]]] JavaScript API is enhanced in the following ways:

• If the userSameParty boolean is true then an exception will also be requested for any domains specified in the origin server's [TSR] same-party property, as long as the reciprocal same-party also references the origin server requesting the exception. The domain property will then no longer be necessary.
• All functions will return a Promise to guarantee asynchronous completion when the Promise is resolved. The Promise has a TrackingStatusObject fulfilment value.
• A TrackingStatusObject contains a DOMString Status indicating the success or failure of the call. It MAY also contain a unique DOMString value GrantID identifying the consent instance. The GrantId will be associated in the user agent with the other information representing a consent instance, and MAY be communicated to an origin server so that a proof-of-consent system can be implemented.
• There is a new confirmTrackingException function to replace confirmSiteSpecificTrackingException and confirmWebWideTrackingException. This will return a promise resolving to an object indicating more information about the exception, e.g. whether it was WebWide or SiteSpecfic, the version level of the API, whether the user has been prompted for confirmation etc.

            partial interface Navigator {
                Promise<TrackingStatusObject> 
                    storeSiteSpecificTrackingException (TrackingPermission properties);
                Promise<TrackingStatusObject> 
                    confirmTrackingException(TrackingPermission properties);
            };
            dictionary TrackingStatusObject {
                DomString Status; // string value indicating "OK" for success or any other value for failure
                DOMString GrantId; // a unique value registered for this granted exception, 
                                   // or consent receipt. 
                                   // This can be communicated to a server so the grant  can be matched to one stored 
                                   // within the user agent for proof-of-consent purposes.
            };
            dictionary StoreExceptionPropertyBag {
            DOMString? domain;
            DOMString? siteName;
            DOMString? explanationString;
            DOMString? detailURI;
            DOMString? expires;
            long? maxAge;
            };
            dictionary TrackingPermission : StoreExceptionPropertyBag {
                boolean? useSameParty; // true to specify UGE is also for same-parties
                boolean? resetTargets; // clears any pre-existing arrayOfDomainNames property from a previous granted exception
                DOMString? DNTVal; // the DNT field-name to be sent in future requests to this origin server, optionally its same-parties, and any targets. 
                                   // if the property is present the first character must be either "0" or "1"
                                   // The user agent will strip any characters after the field value which defaults to "0".
                DOMString? optIn; // the URI of a resource to communicate a user opt-in (explicit consent) to a server. When a user opts-in using a user agent
                                  // based UI, an HTTP GET MAY be sent to this URI.
                DOMString? optOut; // the URI of a resource to communicate a user opt-out (revoked consent) to a server. When a user opts-out using a user agent
                                  // based UI, or the user agent detects that the Consent Grant has expired, an HTTP GET MAY be sent to this URI.
            };