X Tutup
The Wayback Machine - https://web.archive.org/web/20201004114422/https://github.com/cortexproject/cortex/issues/2380
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make all the caching components use snappy #2380

Open
gouthamve opened this issue Apr 1, 2020 · 7 comments
Open

Make all the caching components use snappy #2380

gouthamve opened this issue Apr 1, 2020 · 7 comments

Comments

@gouthamve
Copy link
Member

@gouthamve gouthamve commented Apr 1, 2020

We have an easy way to add snappy compression to caches, and we do it for the index-read-cache. But we should add it to other caches, specially the frontend-results cache.

We had it in the frontend-cache, but we lost it in this refactoring by mistake: #1437

@RajatVaryani
Copy link

@RajatVaryani RajatVaryani commented Apr 3, 2020

@gouthamve Can I work on this?

@codesome
Copy link
Contributor

@codesome codesome commented Apr 6, 2020

@RajatVaryani sure go ahead!

RajatVaryani added a commit to RajatVaryani/cortex that referenced this issue Apr 7, 2020
Signed-off-by: Rajat Varyani <rajat.varyani@outlook.com>
@RajatVaryani RajatVaryani mentioned this issue Apr 7, 2020
0 of 3 tasks complete
@yeya24
Copy link
Contributor

@yeya24 yeya24 commented Sep 18, 2020

Hi, I want to work on supporting snappy cache in the query-frontend, since Thanos can also benefit from it.

Now my idea is to add a EnableSnappy config to ResultsCacheConfig and then modify the NewResultsCacheMiddleware function.

func NewResultsCacheMiddleware(
	logger log.Logger,
	cfg ResultsCacheConfig,
	splitter CacheSplitter,
	limits Limits,
	merger Merger,
	extractor Extractor,
	cacheGenNumberLoader CacheGenNumberLoader,
	shouldCache ShouldCacheFn,
	reg prometheus.Registerer,
) (Middleware, cache.Cache, error) {
	c, err := cache.New(cfg.CacheConfig, reg, logger)
	if err != nil {
		return nil, nil, err
	}
	if cfg.EnableSnappy {
		c = cache.NewSnappy(c, logger)
	}
        ...
}

If you think this way looks ok, I can do some performance testing and open a pr.

@pracucci
Copy link
Contributor

@pracucci pracucci commented Sep 21, 2020

Why not directly passing cache.NewSnappy() to NewResultsCacheMiddleware()?

@yeya24
Copy link
Contributor

@yeya24 yeya24 commented Sep 21, 2020

Hi @pracucci, did you mean to always use cache.NewSnappy in NewResultsCacheMiddleware()? My idea is to make snappy configurable since if only fifo cache is enabled, then it is not necessary to use snappy.

@pracucci
Copy link
Contributor

@pracucci pracucci commented Sep 21, 2020

We should allow to configure which compression should be used, but I'm wondering if we should do it in the parent caller to NewResultsCacheMiddleware() and not in the NewResultsCacheMiddleware() itself.

@yeya24
Copy link
Contributor

@yeya24 yeya24 commented Sep 21, 2020

I still think that adding a EnableSnappy config to ResultsCacheConfig is better since we don't need to update the function signatures. I will create a pr and we can talk there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants
You can’t perform that action at this time.
X Tutup