Segmentation Validation Engine

This web resource provides an online framework for evaluating the results of skull-stripping algorithms. Users may browse the results produced by others, or they may download the test data, segment it with their own algorithm, and upload it to have it evaluated.

Visit Website

Features

  • SVE provides fast automated comparison of uploaded segmentation results with manually delineated MRI.
  • SVE produces visual maps of segmentation errors mapped into a common reference space.
  • SVE archives test results so they may be browsed and sorted to compare different algorithms.
  • SVE provides graphs for comparing segmentation results.

Description

We provide a set of 40 T1 weighted scans which were used in the LONI Probabilistic Brain Atlas (LPBA40) project (Shattuck et al., 2008). These data were initially skull-stripped using BET, then hand-corrected to fix segmentation errors and remove the cerebellum. The volumes were then labeled with 56 anatomical areas, mostly describing major cortical areas. We then processed the labels and cerebellum masks to produce a whole-brain mask for each volume.

These masks allow us to evaluate the results of other segmentation approaches, and we can also assess performance relative to specific areas of the brain. Users can download data from this site, perform segmentation on it, and then upload their results. This server will then compute a series of error metrics and archive the results. For a full description of the metrics used, please see the metrics page. The archive can then be explored to examine the attributes of individual algorithms or the effects of changing parameter settings.

For information on how to use this site, please see the instructions.

Previous results can be found in our results archive.

A complete description can be found in (Shattuck et al., 2008B).

Usage

SVE does not currently support concurrent computation sessions.

Installation

None required; the resource is available through a web-browser. Java and Flash are required for full functionality.

Purpose

SVE provides an online resource to test and evaluate segmentation algorithms. Users can download data from this site, perform segmentation on it locally, and then upload their results. This server will then compute a series of error metrics and archive the results. Visitors to the site may browse the archived results and compare them using the different measures.