Implement plugins for logging drivers by cpuguy83 · Pull Request #28403 · moby/moby

cpuguy83 · 2016-11-14T19:07:43Z

This is intended as PoC code that as of yet untested other than ensuring compilation.

Logging plugins use the same HTTP interface as other plugins for basic
command operations meanwhile actual logging operations are handled (on
Unix) via a fifo. I believe this is much cleaner than dealing with go1.8 style plugins.

The plugin interface looks like so:

type loggingPlugin interface {
  StartLogging(fifoPath string, loggingContext Context) error
  StopLogging(fifoPath)
}

This means a plugin must implement LoggingDriver.StartLogging and
LoggingDriver.StopLogging endpoints and be able to consume the passed
in fifo.

Logs are sent via stream encoder to the fifo encoded with json.
Ideally this would be protobuf, but I did not implement this yet.

ping @tiborvass @anusha-ragunathan

cpuguy83 · 2016-11-15T20:23:36Z

The other alternative here would be to, instead of using fifo's, either hijack the HTTP stream (getting rid of the HTTP overhead), or potentially force an upgrade to HTTP2 (which is likely simpler for implementers than hijack).

tianon · 2016-11-17T19:22:39Z

Just to note for those looking for something in the interim, Docker does currently support GELF (which is a standardized protocol for logging), so anything which can consume that (and/or translate that, like logstash input/output plugins) can be connected currently that way while a proper plugin interface is designed/developed. 👍

AkihiroSuda · 2016-11-21T06:50:25Z

Thank you for working on this 👍

Some questions about FIFO, from my previous similar PR (#23198)

What happens if the plugin doesn't read(2) from the FIFO? Probably the container gets stuck because its write(2) to stdout blocks?
What happens if the plugin crashes?

cpuguy83 · 2016-11-22T18:06:29Z

What happens if the plugin doesn't read(2) from the FIFO? Probably the container gets stuck because its write(2) to stdout blocks?

Correct, write will be blocked, which is the behavior for all existing drivers except UDP based ones.

What happens if the plugin crashes?

writes will be blocked until the plugin comes back

cpuguy83 · 2016-11-23T15:50:54Z

@AkihiroSuda For non-blocking behavior, I implemented a ring buffer that can be used with any driver in #28762

cpuguy83 · 2016-12-14T18:11:50Z

Since we are already going to be communicating over a unix socket, I'm going to ditch the fifo. These offer similar performance, but with the unix socket I don't have to setup anything else.

cpuguy83 · 2017-01-02T15:03:28Z

Just an update on this:

writing logs works
Kept usage of fifo since it is actually much simpler than dealing with the http connection, can support named pipes on Windows later when there is platform support for this
Reading logs is supported if the log driver reports that it supports reading

To think about (need input from others):

Should we just use a single stream for all log messages (from all containers) to the log driver? This would require encoding the log context with each message (easy enough). This would likely simplify plugin implementations, but make the docker side a bit more complex.
Do we need some cleanup call for log drivers? I'm thinking likely not but throwing it out there. As is I have a plugin which wraps the jsonfile driver which would need a way cleanup logs for removed containers... but I think this is just because of the nature of the jsonfile logger, a real log driver wouldn't just remove logs in this manner.

Here's my plugin implementation: https://github.com/cpuguy83/docker-log-driver-test
I do have a (simple) integration test that uses this driver to both write and read back container logs.

cpuguy83 · 2017-04-08T19:12:01Z

This should be good to go.

anusha-ragunathan · 2017-04-10T16:47:51Z

docs/extend/config.md has the list of drivers currently supported. It needs to be updated.

cpuguy83 · 2017-04-10T16:52:09Z

Added.

anusha-ragunathan · 2017-04-10T16:53:56Z

api/types/plugins/logdriver/io.go

Is this call to io.ReadFull necessary, given the real read at L 83?

Yes, this gets the size of the message.
83 gets the actual message, which we need the size of to know where it ends.

anusha-ragunathan · 2017-04-10T16:55:44Z

api/types/plugins/logdriver/io.go

shouldnt uint32(n) -> uint32(total) ?

Here we are putting writing the size of the log entry to the buffer.
But the buffer needs to be the total size otherwise we'll be 4-bytes short.

Thanks, I just looked at the logEntry's Size() implementation. Looks good.

anusha-ragunathan · 2017-04-10T16:56:57Z

api/types/plugins/logdriver/entry.pb.go

TimeNano is not copied into the dAtA marshal output. Looks incorrect.

I think this is an protobuf optimization since TimeNano is an int64 already there is nothing extra to copy.

tgermain · 2017-04-10T17:12:56Z

docs/extend/plugins_logging.md

typo, josn should be json I think.

josn, it's the same as json, but all the keys are scrambled up.

:) Thanks I do this all the time.

Logging plugins use the same HTTP interface as other plugins for basic command operations meanwhile actual logging operations are handled (on Unix) via a fifo. The plugin interface looks like so: ```go type loggingPlugin interface { StartLogging(fifoPath string, loggingContext Context) error StopLogging(fifoPath) ``` This means a plugin must implement `LoggingDriver.StartLogging` and `LoggingDriver.StopLogging` endpoints and be able to consume the passed in fifo. Logs are sent via stream encoder to the fifo encoded with protobuf. Signed-off-by: Brian Goff <cpuguy83@gmail.com>

anusha-ragunathan · 2017-04-10T19:12:10Z

@cpuguy83: I'm okay with a follow up PR to address

api/types reorg
swarm services plumbing for scheduling where logging plugins are available. Thenwe can also add swarm tests in https://github.com/docker/docker/blob/master/integration-cli/docker_cli_swarm_unix_test.go

anusha-ragunathan · 2017-04-10T19:12:29Z

LGTM

cpuguy83 · 2017-04-10T19:39:28Z

Cool, going to move to docs-review then, thanks!

thaJeztah · 2017-04-10T19:57:49Z

Discussed with @cpuguy83 and we'll do a follow up for docs due to code-freeze today

GordonTheTurtle added the status/0-triage label Nov 14, 2016

cpuguy83 force-pushed the logging_plugins branch from 3bcfb39 to 20beb61 Compare November 14, 2016 19:08

cpuguy83 added status/1-design-review and removed status/0-triage labels Nov 14, 2016

cpuguy83 force-pushed the logging_plugins branch from 20beb61 to def27ad Compare November 14, 2016 19:14

anusha-ragunathan added the area/plugins label Nov 17, 2016

thaJeztah mentioned this pull request Nov 17, 2016

add AVRO/Flume logger driver #26215

Closed

This was referenced Nov 28, 2016

Feature: Log Driver for Kafka #21271

Closed

Proposal: add aliyun logdriver #21992

Closed

cpuguy83 mentioned this pull request Dec 21, 2016

Add NATS client as option for publishing container logs #27841

Closed

cpuguy83 force-pushed the logging_plugins branch 8 times, most recently from 90e6407 to a277a32 Compare December 31, 2016 19:10

cpuguy83 added rebuild/windowsRS1 and removed status/1-design-review labels Jan 2, 2017

GordonTheTurtle removed the rebuild/windowsRS1 label Jan 2, 2017

cpuguy83 added the status/1-design-review label Jan 2, 2017

cpuguy83 mentioned this pull request Jan 2, 2017

Add log-driver for logstash #14949

Closed

cpuguy83 added the rebuild/windowsRS1 label Apr 8, 2017

GordonTheTurtle removed the rebuild/windowsRS1 label Apr 8, 2017

cpuguy83 force-pushed the logging_plugins branch from b62cd28 to d13e742 Compare April 10, 2017 16:51

anusha-ragunathan reviewed Apr 10, 2017

View reviewed changes

tgermain reviewed Apr 10, 2017

View reviewed changes

cpuguy83 force-pushed the logging_plugins branch from d13e742 to 27bd684 Compare April 10, 2017 17:17

cpuguy83 added status/3-docs-review docs/revisit impact/changelog and removed status/2-code-review labels Apr 10, 2017

thaJeztah merged commit 28334c1 into moby:master Apr 10, 2017

GordonTheTurtle added this to the 17.05.0 milestone Apr 10, 2017

cpuguy83 deleted the logging_plugins branch April 10, 2017 19:59

cpuguy83 mentioned this pull request Apr 10, 2017

Logging driver plugins #18604

Closed

gquintana mentioned this pull request Apr 11, 2017

Getting Logs from Docker Containers elastic/beats#918

Closed

mdlinville mentioned this pull request Apr 21, 2017

Document logging driver plugins docker/docs#2997

Closed

naag mentioned this pull request Jun 7, 2017

Inconsistent file path passed to logging plugins on StartLogging/StopLogging #33569

Closed

subfuzion mentioned this pull request Aug 7, 2017

logs appcelerator-archive/ampnext-discussion#7

Open

thaJeztah mentioned this pull request Sep 14, 2017

Proposal: automatic plugin installation at startup #34850

Open

thaJeztah mentioned this pull request Jul 22, 2025

API: types/plugins/logdriver must be removed from API module #50473

Closed

Conversation

cpuguy83 commented Nov 14, 2016

Uh oh!

cpuguy83 commented Nov 15, 2016

Uh oh!

tianon commented Nov 17, 2016

Uh oh!

AkihiroSuda commented Nov 21, 2016

Uh oh!

cpuguy83 commented Nov 22, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpuguy83 commented Nov 23, 2016

Uh oh!

cpuguy83 commented Dec 14, 2016

Uh oh!

cpuguy83 commented Jan 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpuguy83 commented Apr 8, 2017

Uh oh!

anusha-ragunathan commented Apr 10, 2017

Uh oh!

cpuguy83 commented Apr 10, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tgermain Apr 10, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anusha-ragunathan commented Apr 10, 2017

Uh oh!

anusha-ragunathan commented Apr 10, 2017

Uh oh!

cpuguy83 commented Apr 10, 2017

Uh oh!

thaJeztah commented Apr 10, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

cpuguy83 commented Nov 22, 2016 •

edited

Loading

cpuguy83 commented Jan 2, 2017 •

edited

Loading

tgermain Apr 10, 2017 •

edited

Loading