Logarithmic Loss Solver

Reverse engineers the prediction or the positive sample ratio to provide to achieve a known loss.

metrics.logloss.solve(to_solve, known_loss = NULL, known_pred = NULL,
  known_ratio = NULL)

Arguments

to_solve	Type: character. What to solve. "pred" solves for the best constant prediction value provided the known_loss and known_ratio. Use known_loss as 0 if you want to minimize the loss function. Provide another value when reverse-engineering the prediction is required. Although the solving equation can be expressed as an exact formula (with an exact solution), the solver uses an approximate method (Brent) to fit rare cases such as loss minimization provided a single ratio. "ratio" solves for the best constant ratio of positives over (positives + negatives), provided the known_loss and known_pred. Use known_loss as 0 if you want to minimize the loss function. Provide another value when reverse-engineering the ratio is required. It uses an approximate solving method (Brent).
known_loss	Type: numeric. The known loss issued from the logartihmic loss.
known_pred	Type: numeric. The prediction value which must be fixed. Must be provided when to_solve == "ratio" Defaults to NULL.
known_ratio	Type: numeric. The positive ratio which must be fixed. Must be provided when to_solve == "pred" Defaults to NULL.

Value

The solved value.

Examples

# Note: this example unexpectedly fails when using pkgdown.
# Example from https://www.kaggle.com/opanichev/mean-baseline-lb-0-30786/code
# WSDM - KKBox's Churn Prediction Challenge (public score: 0.17689)

# Reverse engineeer ratio of positives in Public Leaderboard
print(metrics.logloss.solve(to_solve = "ratio",
                            known_loss = 0.17695680071494552,
                            known_pred = 0.08994191315811156), digits = 17)
#> [1] 0.035737961921635884

# Reverse engineer the prediction value used in Public Leaderboard
print(metrics.logloss.solve(to_solve = "pred",
                            known_loss = 0.17695680071494552,
                            known_ratio = 29650 / (800000 + 29650)), digits = 17)
#> [1] 0.089941912818003567

# Find better prediction value for the Public Leaderboard
print(metrics.logloss.solve(to_solve = "pred",
                            known_loss = 0,
                            known_ratio = 29650 / (800000 + 29650)), digits = 17)
#> [1] 0.035737961195227909
cat("My better logloss: ",
    -1 * ((0.03573796) * log(0.03573796) + ((1 - 0.03573796) * log(1 - 0.03573796))),
    sep = "")
#> My better logloss: 0.1541541