HuberLossDeltaComparison

public static void DeltaTuningGuide()

Language: C#

Documentation: Delta parameter tuning guide

Recommended delta values for different scenarios:

Scenario 1: Normalized data (μ=0, σ=1)

  • Start with delta = 1.0

  • Error > 1σ is considered outlier-like

Scenario 2: Un-normalized data (e.g., house prices in dollars)

  • Set delta based on acceptable error magnitude

  • delta = 0.1 * typical_error (e.g., 10% for price prediction)

Scenario 3: Data with known outlier rate

  • Estimate outlier threshold from data

  • Set delta to that threshold

Tuning Strategy (GPU-Friendly):

  1. Start with delta = σ (standard deviation of errors)

  2. Use HuberLoss with gpuOps for fast evaluation

  3. Plot loss values for different deltas

  4. Choose delta where validation loss is minimized

Benefits of GPU-Aware Comparison:

  • All delta values can be tested in parallel

  • Each evaluation stays on GPU

  • No data transfer overhead

  • Results consistent across runs (no randomness in comparison)