Mirrors/coredns
1
0
Fork 0
mirror of https://github.com/coredns/coredns.git synced 2025-11-01 02:33:14 -04:00
Code Issues Packages Projects Releases Wiki Activity

New cache implementation and prefetch handing in mw/cache (#731)

Browse Source 
* cache: add sharded cache implementation

Add Cache impl and a few tests. This cache is 256-way sharded, mainly
so each shard has it's own lock. The main cache structure is a readonly
jump plane into the right shard.

This should remove the single lock contention on the main lock and
provide more concurrent throughput - Obviously this hasn't been tested
or measured.

The key into the cache was made a uint32 (hash.fnv) and the hashing op
is not using strings.ToLower anymore remove any GC in that code path.

* here too

* Minimum shard size

* typos

* blurp

* small cleanups no defer

* typo

* Add freq based on Johns idea

* cherry-pick conflict resolv

* typo

* update from early code review from john

* add prefetch to the cache

* mw/cache: add prefetch

* remove println

* remove comment

* Fix tests

* Test prefetch in setup

* Add start of cache

* try add diff cache options

* Add hacky testcase

* not needed

* allow the use of a percentage for prefetch

If the TTL falls below xx% do a prefetch, if the record was popular.
Some other fixes and correctly prefetch only popular records.
This commit is contained in:
Miek Gieben
2017-06-13 12:39:10 -07:00
committed by GitHub
parent b1efd3736e
commit e9eda7e7c8
23 changed files with 595 additions and 142 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
middleware/etcd/etcd.go
View File

@@ -9,6 +9,7 @@ import (
"github.com/coredns/coredns/middleware"
"github.com/coredns/coredns/middleware/etcd/msg"
"github.com/coredns/coredns/middleware/pkg/cache"
"github.com/coredns/coredns/middleware/pkg/singleflight"
"github.com/coredns/coredns/middleware/proxy"
"github.com/coredns/coredns/request"
@@ -90,7 +91,10 @@ func (e *Etcd) Records(name string, exact bool) ([]msg.Service, error) {
// get is a wrapper for client.Get that uses SingleInflight to suppress multiple outstanding queries.
func (e *Etcd) get(path string, recursive bool) (*etcdc.Response, error) {
resp, err := e.Inflight.Do(path, func() (interface{}, error) {
hash := cache.Hash([]byte(path))
resp, err := e.Inflight.Do(hash, func() (interface{}, error) {
ctx, cancel := context.WithTimeout(e.Ctx, etcdTimeout)
defer cancel()
r, e := e.Client.Get(ctx, path, &etcdc.GetOptions{Sort: false, Recursive: recursive})