Comparison with baseline methods

Do sophisticated algorithms always outperform simple baseline methods?

Finding: Many algorithms are beaten by the IDENTITY baseline at large scales, in both 1D and 2D. At low scales, many algorithms result in error rates that are comparable to, or worse than, the Uniform baseline.

The above heatmap shows the ratio of the error of the primary algorithm to the error of a baseline algorithm (Identity or Uniform), across all datasets and various scales. The magnitude of the ratio is represented by the color in each cell.

Blue cells show the primary algorithm outperforming the baseline.
Red cells show the primary algorithm underperforming the baseline.

At small scale, there are some datasets, (e.g. BIDS-FJ) on which all algorithms are beaten by UNIFORM.