Skip to content

Add ScoringRules.jl as scoring backend #32

Description

@jc-macdonald

Summary

The scoring module is being developed as a standalone package at jcm-sci/ScoringRules.jl since proper scoring rules have broad utility beyond model criticism.

TradeStudy.jl should depend on ScoringRules.jl and wire it into the AbstractScorer dispatch system from the Protocols module (#25).

Proposed Design

struct ProperScoringRuleScorer <: AbstractScorer
    metric::Symbol  # :crps, :energy, :wis, :brier, etc.
end

function score(s::ProperScoringRuleScorer, obs, pred)
    ScoringRules.score(s.metric, obs, pred)
end

Depends On

Acceptance Criteria

  • ScoringRules added to [deps] in Project.toml
  • ProperScoringRuleScorer exported and documented
  • Integration test using CRPS on synthetic data

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions