plugin/forward using pkg/up (#1493)

* plugin/forward: on demand healtchecking

Only start doing health checks when we encouner an error (any error).
This uses the new pluing/pkg/up package to abstract away the actual
checking. This reduces the LOC quite a bit; does need more testing, unit
testing and tcpdumping a bit.

* fix tests

* Fix readme

* Use pkg/up for healthchecks

* remove unused channel

* more cleanups

* update readme

* * Again do go generate and go build; still referencing the wrong forward
  repo? Anyway fixed.
* Use pkg/up for doing the healtchecks to cut back on unwanted queries
  * Change up.Func to return an error instead of a boolean.
  * Drop the string target argument as it doesn't make sense.
* Add healthcheck test on failing to get an upstream answer.

TODO(miek): double check Forward and Lookup and how they interact with
HC, and if we correctly call close() on those

* actual test

* Tests here

* more tests

* try getting rid of host

* Get rid of the host indirection

* Finish removing hosts

* moar testing

* import fmt

* field is not used

* docs

* move some stuff

* bring back health_check

* maxfails=0 test

* git and merging, bah

* review

plugin/forward/connect.go

@@ -21,9 +21,6 @@ func (p *Proxy) connect(ctx context.Context, state request.Request, forceTCP, me
 	if forceTCP {
 		proto = "tcp"
 	}
-	if p.host.tlsConfig != nil {
-		proto = "tcp-tls"
-	}
 
 	conn, err := p.Dial(proto)
 	if err != nil {
@@ -57,9 +54,9 @@ func (p *Proxy) connect(ctx context.Context, state request.Request, forceTCP, me
 			rc = strconv.Itoa(ret.Rcode)
 		}
 
-		RequestCount.WithLabelValues(p.host.addr).Add(1)
-		RcodeCount.WithLabelValues(rc, p.host.addr).Add(1)
-		RequestDuration.WithLabelValues(p.host.addr).Observe(time.Since(start).Seconds())
+		RequestCount.WithLabelValues(p.addr).Add(1)
+		RcodeCount.WithLabelValues(rc, p.addr).Add(1)
+		RequestDuration.WithLabelValues(p.addr).Observe(time.Since(start).Seconds())
 	}
 
 	return ret, nil