Jev Can't Be Calibrated

Jev, TypeSafe's new classification model that provides typed decisions with probability scores, is useful for solving classification problems without requiring training data. However, the author argues that Jev's claimed calibrated probabilities cannot hold in general because calibration depends on data distribution—a model calibrated on TypeSafe's training data may be miscalibrated on users' different production data, and evidence suggests the model performs worse than expected even on simple tasks like predicting a fair coin flip.