Commit Graph

50 Commits

Author SHA1 Message Date
Michael Richmond
4a3b57d81b Adding test cases for Corefile parsing (#193)
Adding test cases for Corefile parsing.
Some code refactoring to allow test reuse.
2016-07-22 16:07:27 -07:00
Michael Richmond
3ba86f2421 Move to logging package (#191)
* Updating comment to remove references to Caddy

* Updating README to mark TODO items complete

* Changing all debug print statements over to use logging package
2016-07-18 18:47:36 +01:00
Michael Richmond
289f53d386 k8s middleware cleanup, testcases, basic SRV (#181)
* Removing unnecessary gitignore pattern

* Updating Makefile to run unittests for subpackages

* Adding Corefile validation to ignore overlapping zones

* Fixing SRV query handling

* Updating README.md now that SRV works

* Fixing debug message, adding code comment

* Clarifying implementation of zone normalization

* "Overlapping zones" is ill-defined. Reimplemented zone overlap/subzone
  checking to contain these functions in k8s middleware and provide
  better code comments explaining the normalization.

* Separate build verbosity from test verbosity

* Cleaning up comments to match repo code style

* Merging warning messages into single message
* Moving function docs to before function declaration

* Adding test cases for k8sclient connector

* Tests cover connector create and setting base url
* Fixed bugs in connector create and setting base url functions

* Updaing README to group and order development work

* Priority focused on achieving functional parity with SkyDNS.

* Adding work items to README and cleaning up formatting

* More README format cleaning

* List formating

* Refactoring k8s API call to allow dependency injection

* Add test cases for data parsing from k8s into dataobject structures

* URL is dependency-injected to allow replacement with a mock http
  server during test execution

* Adding more data validation for JSON parsing tests

* Adding test case for GetResourceList()

* Adding notes about SkyDNS embedded IP and port record names

* Marked test case implemented.

* Fixing formatting for example command.

* Fixing formatting

* Adding notes about Docker image building.

* Adding SkyDNS work item

* Updating TODO list

* Adding name template to Corefile to specify how k8s record names are assembled

* Adding template support for multi-segment zones

* Updating example CoreFile for k8s with template comment

* Misc whitespace cleanup

* Adding SkyDNS naming notes

* Adding namespace filtering to CoreFile config

* Updating example k8sCoreFile to specify namespaces

* Removing unused codepath

* Adding check for valid namespace

* More README TODO restructuring to focus effort

* Adding template validation while parsing CoreFile

* Record name template is considered invalid if it contains a symbol of the form ${bar} where the symbol
  "${bar}" is not an accepted template symbol.

* Refactoring generation of answer records

* Parse typeName out of query string
* Refactor answer record creation as operation over list of ServiceItems

* Moving k8s API caching into SkyDNS equivalency segment

* Adding function to assemble record names from template

* Warning: This commit may be broken. Syncing to get laptop code over to dev machine.

* More todo notes

* Adding comment describing sample test data.

* Update k8sCorefile

* Adding comment

* Adding filtering support for kubernetes "type"

* Required refactoring to support reuse of the StringInSlice function.

* Cleaning up formatting

* Adding note about SkyDNS supporting word "any".

* baseUrl -> baseURL

* Also removed debug statement from core/setup/kubernetes.go

* Fixing test breaking from Url -> URL naming changes

* Changing record name template language ${...} -> {...}

* Fix formatting with go fmt

* Updating all k8sclient data getters to return error value

* Adding error message to k8sclient data accessors

* Cleaning up setup for kubernetes

* Removed verbose nils in initial k8s middleware instance
* Set reasonable defaults if CoreFile has no parameters in the
kubernetes block. (k8s endpoint, and name template)

* Formatting cleanup -- go fmt
2016-07-07 09:40:58 +01:00
Michael Richmond
d04abdf422 BK8s datasource middleware -- PoC for A records (#153)
* Laying down kubernetes middleware foundation

* Duplicated a bunch of code form etcd middleware
* Duplicated code hacked to compile and load as a separate middleware

* Adding verbose build option to Makefile

* Removing stubzone and tls support

tls and stubzone support was carried over from base etcd middleware code.
Removing to simplify the kube middleware implementation. (For now.)

* Adding conf directory for sample conf files

* Removing stubzone support from query handler

* Remove upstream and proxy from k8s corefile.

Not sure that upstream or proxy makes sense for a k8s backed zone.

* Comment out use of singleflight serialization

* Removing parsing support for "upstream" directive from k8s

* Removing upstream directive parsing code

* Removing CNAME and TXT lookup implementation

* Create README.md

Brain-dump of DNS record name assembly and open work items.

* Adding notes about wildcard handling

* Adding basic k8s API client

* Fleshing out methods on k8s connector

* Remove PathPrefix from middleware init

* Removing incorrect plural

* Adding brute-force k8s service lookup functions

* Initializing k8s API connector during startup

* Hacking around to call k8s connector

* Parsing incoming domain name into serviceName and namespace

* Improving and simplifying k8s zone matching and label segmentation

* Removing unused functions carried over from etcd middleware

* Adding basic return of k8s data to DNS client

* updated debugging println statements to flag with "[debug]"
* removed code in kubernetes.go::Records that was a hold-over from etcd middleware.
* Removed some random exploratory hacking.

* Minior README.md updates

* Updating with demo instructions

* Updating README.md with CoreFile and removing completed TODO items

* Updating conf file and README to reflect DNS response cache works

* Disabling DNS response caching

* Adding debug statement on entry to Records()

* Changing port number in exampes to port 53.

* Misc style and clarity changes

* Removing empty function definitions

* Adding comment to track future cleanup

* Refactoring README to follow style of other middleware

* Exposing dataobject field (typo)
2016-06-06 20:49:53 +01:00
Miek Gieben
c30671f4c0 Allow debug queries to etcd middleware (#150)
With this you can retreive the raw data that the etcd middleware
used to create the reply. The debug data is put in TXT records
that are stuffed in the CH classs. This is only enabled if you
specify `debug` in the etcd stanza.

You can retrieve it by prefixing your query with 'o-o.debug.'
For instance:

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @localhost -p 1053 SRV o-o.debug.production.*.skydns.local
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47798
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;o-o.debug.production.*.skydns.local. IN	SRV

;; ANSWER SECTION:
production.*.skydns.local. 154	IN	SRV	10 50 8080 service1.example.com.
production.*.skydns.local. 154	IN	SRV	10 50 8080 service2.example.com.

;; ADDITIONAL SECTION:
skydns.local.skydns.east.production.rails.1. 154 CH TXT	"service1.example.com:8080(10,0,,false)[0,]"
skydns.local.skydns.west.production.rails.2. 154 CH TXT	"service2.example.com:8080(10,0,,false)[0,]"
2016-05-22 21:16:26 +01:00
Miek Gieben
14b84ce02b pprof does not need to inject middleware 2016-04-29 22:04:22 +01:00
Miek Gieben
9e9d72655d Make middleware survive a restart (#142)
Make middleware that sets up a (http) handler survive a graceful
restart. We calls the middleware's Shutdown function(s). If restart
fails the Start function is called again.

* middleware/health: OK
* middleware/pprof: OK
* middleware/metrics: OK

All restart OK.
2016-04-29 07:28:35 +01:00
Michael Richmond
e34280e7af Clean up remove caddy refs (#139)
* Changed reference to Caddy over to CoreDNS

* Removing references to caddy

* Fixed misleading error message to reference coredns

* Cleaning up references to caddy

* Adding clean and deps targets

Muscle memory is resulting in "make clean" commands.

* Adding test target to makefile

* More "Caddy" cleanup
2016-04-28 19:07:44 +01:00
Miek Gieben
2d71f94aef metrics port should be 9153 2016-04-28 09:43:18 +00:00
Miek Gieben
2700eece2e pprof middleware (#138)
Add pprof middleware, enabled by pprof directive.
2016-04-28 10:26:58 +01:00
Miek Gieben
7a8d943bcc Bail out on failure when starting up
Don't silently hide failures, barf on startup. Also add more integration
tests that should catch some of these things.
2016-04-27 10:48:22 +00:00
Miek Gieben
1aa1a92198 Add middleware/dnssec (#133)
This adds an online dnssec middleware. The middleware will sign
responses on the fly. Negative responses are signed with NSEC black
lies.
2016-04-26 17:57:11 +01:00
Miek Gieben
e5e0cde08f Fix travis.yml (#129)
Also fix failing test in setup/error.
2016-04-21 14:26:02 +01:00
Miek Gieben
e726dca2ee Fix error reporting (#128)
Put error back in the correct place in the directives.go. Also don't
make it a pointer. If it *is* a pointer the buildstack function does
not correctly set the Next Handler. Don't understand *why* this is
different from Caddy. Anyway this fixes it, with the caveat that
the error log file is now openend earlier in the startup.

Fixes #127
2016-04-19 22:51:23 +01:00
Miek Gieben
a1e6cb7cf4 middleware/etcd: Use the correct endpoint
The endpoint for etcd as wrongly set if a proxy was configured.
Clean up some documentation in the process as well.
2016-04-19 12:52:05 +00:00
Miek Gieben
10db2a80df Cache (#126)
* Add middleware/cache

Add a caching middleware that caches nxdomain, nodata and successful
responses. It differentiates between DNSSEC on normal DNS replies.

Each reply is compress and scrubbed so it will fit the specific client
asking for it.

* first simple test, less exporting of stuff

* more

* Add middleware/cache

Add a caching middleware that caches nxdomain, nodata and successful
responses. It differentiates between DNSSEC on normal DNS replies.

Each reply is compressed and scrubbed so it will fit the specific client
asking for it. The TTL is decremented with the time spend in the cache.
There is syntax that allows you to cap the TTL for all records, no
matter what. This allows for a shortlived cache, just to absorb query
peaks.

+Tests

* cache test infrastructure

* Testing
2016-04-19 11:13:24 +01:00
Miek Gieben
9b82cca9e2 Fix parsing of transfer lines 2016-04-15 17:08:31 +01:00
Miek Gieben
c9d8a57ed6 Reload zone when a write is detected (#122)
Zone reloading
2016-04-15 14:26:27 +01:00
Miek Gieben
29ad957a9d Fix build 2016-04-14 21:29:01 +01:00
Miek Gieben
fb3286983b Do the notifies only once during startup 2016-04-14 21:22:12 +01:00
Miek Gieben
885e6e8246 Allow more than 1 address for transfer (#121)
No reason why not to allow more then one address:
`transfer to 127.0.0.1 10.240.20.1`.

Fix startup as well, as it turned out to be broken...
2016-04-14 19:57:39 +01:00
Miek Gieben
982377516b Only transfer a zone once (#117)
Use sync.Once to only transfer a zone once.

Fixes #115
2016-04-13 20:14:03 +01:00
Miek Gieben
9f651a397b All (non etcd) tests are now local (#105)
We don't need to network to do tests, we up enough local servers
to we don't need to forward to,s say 8.8.8.8
2016-04-11 15:56:22 +01:00
Miek Gieben
ad221f4b2a correct EDNS responses (#96)
Tests updated as well and all the middleware. And Prometheus renamed to
metrics (directive is still prometheus).
2016-04-09 16:17:53 +01:00
Miek Gieben
68171c7a63 A health middleware
Start http handler on port 8080 and return OK. Also add some

documentation fixes for the prometheus middleware.
2016-04-06 09:21:46 +01:00
Miek Gieben
ecb53addd6 Normalize names before putting it in the handler 2016-04-05 16:36:27 +01:00
Miek Gieben
e0c4f94d67 return failure with CH class hits file or etcd, monitoring portis 9135 2016-04-05 16:23:03 +01:00
Miek Gieben
8892a1b490 Middleware chaining fixes
For prometheus use the plain value, not a pointer and change all usages.
Allow AXFR to be requested over udp as well and some other more log
printed when commencing an AXFR.
2016-04-05 15:54:06 +01:00
Miek Gieben
c961acbb6e Add complete secondary support
Respond to notifies and allow a secondary to follow the SOA parameters

to update a zone from a primary. Also sprinkle it with logging.



Also extend monitoring to include qtype in more metrics.
2016-04-05 10:53:23 +01:00
Miek Gieben
8d0d324f27 Dont normalize * 2016-04-03 16:08:10 +01:00
Miek Gieben
3b25bfd7ae Fix Corefile parsing
Fix some file/secondary issues when parsing a Corefile, also allow

for multiple origins to be specified. Also don't fail on startup when

a zonetransfer fails.



Fixes: #54
2016-04-03 15:52:23 +01:00
Miek Gieben
f58f1e4285 Add secondary support
Allow specifying a primary server and retrieve the zone's content.

Add tests and an Expired bool to zone struct, to stop server zones

that are expired. The zone is retrieved on Startup, no updates of

changed content are done. We also don't respond to notifies yet.
2016-04-03 09:02:34 +01:00
Miek Gieben
6ecbdef112 Add notifies to master servers
This adds a bunch of supporting code to send notifies to a primary

name server.
2016-04-03 07:37:41 +01:00
Miek Gieben
e56d206542 Support outgoing zone transfers
These can be enabled by adding "transfer out" to the Corefile. Without
it no AXFR is allowed.

For now only AXFR and no IXFR. No TSIG and no ACLs.
2016-03-28 18:23:17 +01:00
Miek Gieben
5a919198ac Merge pull request #45 from miekg/etcd-stub
Etcd stub
2016-03-28 10:18:08 +01:00
Miek Gieben
5387c162c9 Implement a DNS zone
Full implementation, DNS (and in the future DNSSEC). Returns answer in a
hopefully standards compliant way.
Testing with my miek.nl zone are included as well.
This should correctly handle nodata, nxdomain and cnames.
2016-03-28 10:15:05 +01:00
Miek Gieben
ebef64280a Support SkyDNS' stubzones
This implements stubzones in the same way as SkyDNS. This
also works with multiple configured domains and has tests.
Also add more configuration parameters for TLS and path prefix and
enabling stubzones.  Run StubUpdates as a startup command to keep up to
date with the list in etcd.
2016-03-26 16:29:35 +00:00
Miek Gieben
5d70567f1c Implement chaos middleware
This allows for CH TXT queries that return some information about
the server and/or the authors (or whatever you put in there).
2016-03-25 08:36:32 +00:00
Miek Gieben
163e5d8e71 fixed 2016-03-23 23:20:50 +00:00
Miek Gieben
b167411bba Add roundrobin middleware
This middleware allows you to round robin a/aaaa records in a reply and
maybe more in the future (i.e.) sort a packet?
2016-03-23 21:38:35 +00:00
Miek Gieben
7d834ce037 Remove reflect middleware - it was only a toy 2016-03-23 10:54:50 +00:00
Miek Gieben
8c707c8031 more 2016-03-22 22:44:50 +00:00
Miek Gieben
b6341e8b63 more etcd stuff 2016-03-20 21:36:55 +00:00
Miek Gieben
0c94de4f71 More stuff copied from SkyDNS 2016-03-20 18:17:07 +00:00
Miek Gieben
e00e002fc2 more 2016-03-20 17:54:21 +00:00
Miek Gieben
2454d23cab only rewrite types when they are given in uppercase 2016-03-19 22:12:07 +00:00
Miek Gieben
ae5783b7c4 Errors directive testing and fixing
Drop a few tests and make it work and compile.
Also add the documentation: errors.md
2016-03-19 13:37:20 +00:00
Miek Gieben
39dbd447b5 Fix middleware log 2016-03-19 11:16:08 +00:00
Miek Gieben
f907311cdf Use context.Context
Rename the old Context to State and use context.Context in the
middleware for intra-middleware communication and more.
2016-03-19 07:32:50 +00:00
Miek Gieben
3ec0d9fe6b First commit 2016-03-18 20:57:35 +00:00