Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better documentation for logging #731

Merged
merged 25 commits into from
Sep 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:8.11.4
FROM node:10.0.0

WORKDIR /app/website

Expand Down
5 changes: 3 additions & 2 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,8 @@ lazy val doobieLogging = project
lazy val examples = project
.in(file("examples"))
.settings(
libraryDependencies ++= List(doobieCore, doobieH2, derevo, monix, groovy),
libraryDependencies ++= List(doobieCore, doobieH2, derevo, monix, groovy, derevoCirce),
libraryDependencies ++= http4s,
defaultSettings,
name := "tofu-examples",
noPublishSettings,
Expand Down Expand Up @@ -412,7 +413,7 @@ lazy val scalacWarningConfig = scalacOptions += {
// }.mkString(",")

// print warning category for fine-grained suppressing, e.g. @nowarn("cat=unused-params")
val contextDeprecationInfo = "cat=deprecation&msg=^(.*((Has)|(With)).*)$:silent"
val contextDeprecationInfo = "cat=deprecation&msg=^(.*((Has)|(With)|(Logging)).*)$:silent"
val verboseWarnings = "any:wv"

s"-Wconf:$contextDeprecationInfo,$verboseWarnings"
Expand Down
37 changes: 3 additions & 34 deletions docs/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
id: logging
title: Logging
---

#
This page is obsolete. Head on to the [new docs on logging](logging/home-page.md).
## Installation

Logging are available as a separate project
Expand Down Expand Up @@ -291,39 +292,7 @@ automagically. That is natural — describe how we can get a context from a comp
context in log, and we will ensure it will be logged anytime you use `Logging` operations. You can log that context to
string as well as use it as a part of structured logging.

### Layouts

Tofu is built upon [Logback](http://logback.qos.ch/) so it needs a custom `logback.xml` file with contexual logging
support. Tofu uses mechanism called markers to store context in logs, so it won't work with existing Layouts e.g.
with [Logstash-encoder](https://github.com/logstash/logstash-logback-encoder).

Luckily for us, tofu has two special Layouts:

* [ELKLayout](https://github.com/tofu-tf/tofu/blob/master/logging/layout/src/main/scala/tofu/logging/ELKLayout.scala)
that outputs structured logs in JSON format. Example appender looks like that:

```xml

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="tofu.logging.ELKLayout"/>
</encoder>
</appender>
```

* [ConsoleContextLayout](https://github.com/tofu-tf/tofu/blob/master/logging/layout/src/main/scala/tofu/logging/ConsoleContextLayout.scala)
that outputs simple text logs. Example appender looks like that:

```xml

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="tofu.logging.logback.ConsoleContextLayout">
<pattern>%d{HH:mm:ss} %-5level %logger{36} - %msg%n [%mdc]%n</pattern>
</layout>
</encoder>
</appender>
```

### Example

Expand Down Expand Up @@ -512,7 +481,7 @@ your `LogRendered`, so your structured logging will work as expected.

## Integration with logs4cats

There is a library for effectful logging named [log4cats](https://github.com/typelevel/log4cats) which shares
There is a library for effectual logging named [log4cats](https://github.com/typelevel/log4cats) which shares
the goal of representing logging as an effect and providing the ability to log context values. It is used by some of the
open-source libraries which may require you to pass an instance of `org.typelevel.log4cats.Logger` to it in order to
log something. Module `tofu-logging-log4cats` contains a helper that can create an instance of log4cats `Logger` from
Expand Down
58 changes: 58 additions & 0 deletions docs/logging/home-page.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
id: tofu.logging.home title: Tofu Logging
---

# Welcome to the Tofu Logging Home page

This is a set of tools for logging. Configurable, concise, handy.

## Installation

Add these dependencies into your project:

```sbt
libraryDependencies += "tf.tofu" %% "tofu-logging" % "<latest version in the badge in README>"
libraryDependencies += "tf.tofu" %% "tofu-logging-derivation" % "<latest version in the badge in README>"

```

For ZIO users the following is enough:

```sbt
libraryDependencies += "tf.tofu" %% "tofu-logging-zio" % "<latest version in the badge in README>"

```

## Quick demo

```scala
import tofu.syntax.logging._
import tofu.logging.Logging
import derevo.derive
import tofu.logging.derivation.loggable

type CardNumber = String //could be a newtype

@derive(loggable)
case class Client(name: String, @hidden cardNumber: CardNumber, id: UUID)

def processPayment[F[_]: Monad: Logging](client: Client, amount: Long): F[Result] =
for {
_ <- info"Processing payment for $client"
_ <- warn"Amount $amount is lower than zero!".whenA(amount < 0)
result <- processData(client, amount, "USD")
.onError(errorCause"Got error on processing payment for $client"(_))
} yield result
```

## What's next

- Discover [the key features](./key-features.md)
- Get to know [the core concepts](./main-entities.md)
- Learn how to use [the syntax](./syntax.md)
- Find a way to use `logging` suitable for you in the [recipes](recipes/recipes.md)
- Check out [the examples](https://github.com/tofu-tf/tofu/tree/better-doobie-example/examples)

## Old documentation

You can also visit obsolete documentation [here](../logging.md).
117 changes: 117 additions & 0 deletions docs/logging/key-features.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
id: tofu.logging.key-features title: Tofu Logging
---

# Simple logging for your services

## Key features

### Supported for every popular style

You can use `tofu.logging` in any style you want:

- Tagless Final style;
- Simple `cats.effect.IO` or `(monix|zio).Task` and `ReaderT`;
- `ZIO` and ZLayers — with the help of dedicated module `tofu-zio-logging` and `ZLogs`;
- Even with `Future`s (although it's kinda in the past).

### Concise syntax

With the implicit instance of `Logging` one can log messages with ease:

```scala
import tofu.logging._
import tofu.syntax.logging._

val error = new Throwable("Oh no.")

def log[F[_] : Logging : Monad] = for {
_ <- info"Hi! I'm logging"
_ <- warn"Hello again!"
_ <- error"It's me, error!"
_ <- errorCause"So sad, I've got an error"(error)
} yield ()
```

### Structured and controllable

You can easily put any necessary values into the structure of the log message with `Loggable`; also you can fully
control what is logged and what is not:

```scala
import derevo._
import tofu.logging._

@derive(Loggable)
case class Payment(id: String, @hidden cardNumber: String, amount: Long)

def log[F[_] : Logging : Monad](payment: Payment) = info"Got payment $payment"
```

Given that there's a defined `Loggable` instance for this context and provided logger is context aware (e.g. created
with `contextual` method), every logged message will contain information about context, for example:

```json
{
"message": "Got payment payment",
"level": "INFO",
"payment": {
"id": "131234234",
"amount": 432
}
}
```

Note that card number is not present at all as it was `@hidden`.

### Context support

Let's say your effect type has some context — it could be trace id or some domain info:

```scala
import cats._
import cats.effect._

case class TraceId(id: Long)

type TracedIO[A] = ReaderT[TraceId, IO, A]
```

tofu.logging can extract it and automatically add into the structure of every log message:

```scala
class MyService[F[_] : Logging] {
def sayHello: F[Unit] = info"Hello!"
}

val ioservice = new MyService[IO]
val tracedService = new MyService[TracedIO]
```

(_Note the absence of anything related to context in MyService, logging doest it all itself._)

Now if we run `tracedService.sayHello` the log message structure will contain the trace id:

```json
{
"message": "Hello!",
"level": "INFO",
"trace.id": 64534
}
```

and if we run `ioService.sayHello` the message will be clean:

```json
{
"message": "Hello!",
"level": "INFO"
}
```

More on that can be found on the dedicated [recipe page](./recipes/context.md).

## What's next

- You can read about core concepts [here](./main-entities.md)
- You can see the recipes and discover what you need to use `logging` [here](recipes/recipes.md)
33 changes: 33 additions & 0 deletions docs/logging/layouts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Layouts

Tofu is built upon [Logback](http://logback.qos.ch/) so it needs a custom `logback.xml` file with contexual logging
support. Tofu uses mechanism called markers to store context in logs, so it won't work with existing Layouts e.g.
with [Logstash-encoder](https://github.com/logstash/logstash-logback-encoder).

Luckily for us, tofu has two special Layouts:

* [ELKLayout](https://github.com/tofu-tf/tofu/blob/master/logging/layout/src/main/scala/tofu/logging/ELKLayout.scala)
that outputs structured logs in JSON format. Example appender looks like that:

```xml

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="tofu.logging.ELKLayout"/>
</encoder>
</appender>
```

* [ConsoleContextLayout](https://github.com/tofu-tf/tofu/blob/master/logging/layout/src/main/scala/tofu/logging/ConsoleContextLayout.scala)
that outputs simple text logs. Example appender looks like that:

```xml

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="tofu.logging.logback.ConsoleContextLayout">
<pattern>%d{HH:mm:ss} %-5level %logger{36} - %msg%n [%mdc]%n</pattern>
</layout>
</encoder>
</appender>
```
99 changes: 99 additions & 0 deletions docs/logging/main-entities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
id: tofu.logging.main.entities title: tofu.logging
---

# tofu.logging core concepts

`tofu.logging` consists of three main things:

## Typeclass `Loggable[A]`

This is a type class that describes how an arbitrary instance of type A should be presented in the log messages.

## Derivation

Loggable instances could be derived automatically using `derevo`:

```scala
import tofu.logging.derivation._
import derevo.derive

@derive(loggable)
case class Data(id: Long, weight: Int, name: String)

```

This annotation puts the instance of `Loggable[Data]` into the (generated if not present) companion object `Data`. When
the message is logged, the fields of `Data` will be put into the result (e.g. JSON).

### Configured derivation

This derivation can be configured with annotations `hidden`, `masked` and `unembed`:

```scala
import tofu.logging.derivation._
import derevo.derive

@derive(loggable)
case class ClientData(name: String, surname: String)

@derive(loggable)
case class Payment(id: Long, @masked(MaskMode.Erase) cardNumber: String, @unembed name: ClientData)

```

A message `info"This is $payment"` would look like that:

```json
{
"@timestamp": "2021-08-20T17:13:39.787Z",
"loggerName": "some-logger",
"threadName": "ioapp-compute-0",
"level": "INFO",
"message": "This is Payment: Payment{id=3234,cardNumber=...,name=ClientData{name=foo,surname=bar}}",
"id": 3234,
"cardNumber": "...",
"name": "foo",
"surname": "bar"
}
```

## Logging

`Logging[F]` is a trait that describes logging capabilities of `F`.

```scala
trait Logging[F[_]] {
def info(message: String, values: LoggedValue*): F[Unit]
//...
}

```

It has a tagged version `ServiceLogging[F, Service]` which can be used for some `Service` and carry this information on
the typelevel.

## Logs

Trait `Logs[I, F]` is a factory of the `Logging[F]` instances which decides how created Logging will behave.

As the creation of arbitrary logging instance could potentially have some side effects, operations of this trait are
effectual:

```scala
trait Logs[I[_], F[_]] {
def byName(name: String): I[Logging[F]]

def forService[Svc]: I[Logging[F]]
//...
}

```

The name method parameter (or type tag for `Svc` type parameter) is used in the underlying logger and then displayed in
the log messages.

### Logging.Make

Nevertheless, some Logging instances can be created safely with no side effects, so one could use `Logging.Make`
which creates plain `Logging[F]`. It uses the default backend by `Slf4j` and `Delay` typeclass.
Loading