FastSurfer Quality Assurance for Manual Edits

[julii-a]

General Support Request

Description

I am performing quality assurance (QA/QC) on FastSurfer outputs and trying to determine which segmentation or surface reconstruction artifacts should be considered critical and require intervention (editing, rerunning, or exclusion).

According to the FastSurfer documentation(https://deep-mi.org/FastSurfer/dev/overview/EDITING.html#pipeline-overview), manual editing is generally discouraged and rarely required. However, during QC using Freeview, I encountered several ambiguous cases. I would like guidance on how to interpret these cases, especially in the context of downstream analyses using volumetric measures, cortical thickness, and eTIV.

Specifically, I want to understand:

1. Which artifacts are acceptable variations,

2. Which artifacts are critical and invalidate the subject,

3. Why FastSurfer discourages manual edits more strongly than FreeSurfer.

=> I would appreciate it if you could comment on each screenshot case.

Steps that lead to your Issue

I observed several cases during visual QC that are not clearly classified as acceptable or critical.

Steps performed:

1. Run FastSurfer using the standard pipeline (run_fastsurfer.sh)

2. Perform visual QC in Freeview

3. Inspect:
   brainmask.mgz / mask.mgz
   aparc.DKTatlas+aseg.deep.mgz
   aseg.auto_noCCseg.mgz
   white and pial surfaces

4. Observed cases:

4-1) The brain mask (or mask) appears hollow in the central region.

4-2) Regions near the skull are misclassified as white matter in the segmentation.

4-3) Mild underestimation of gray matter appears present.

4-4) Due to skull-adjacent white matter misclassification, the white and pial surfaces follow incorrect trajectories.

4-5) Due to skull-adjacent white matter misclassification, the white and pial surfaces form circular artifacts.

4-6) When the colormap is set to heat, white matter is displayed in red. In another subject, the same region was displayed in yellow.

I understand that QC does not have a single “correct” answer and that manual editing does not always improve results. However, I would like to know which of these cases should be considered critical and require mandatory action.

Log Files / Screenshots

I will attach representative screenshots illustrating each of the cases listed above.
Each image shows a different anatomical view: the left image is the sagittal view, the upper-right image is the coronal view, and the lower-right image is the axial (horizontal) view.

4-1) The brain mask (or mask) appears hollow in the central region.

4-2) Regions near the skull are misclassified as white matter in the segmentation.

4-3) Underestimation of gray matter appears present.

4-4) Due to skull-adjacent white matter misclassification, the white and pial surfaces follow incorrect trajectories.

4-5) Due to skull-adjacent white matter misclassification, the white and pial surfaces form circular artifacts.

4-6) When the colormap is set to heat, white matter is displayed in red. In another subject, the same region was displayed in yellow.

Environment

• FastSurfer Verions: 2.4.2
• Installation type: Docker
• OS: Windows

Additional Context

[dkuegler]

I am not sure where the statement "FastSurfer discourages manual edits" is coming from...

Generally, they are rarely required as the Deep Learning segmentation is quite reliable, but rarely, issues might pop up (usually things that might also happen in FreeSurfer). Like dura inclusion or errors for participants with very large ventricles.

When you are going through the cases for QC (as is recommended), you will sometimes find incorrect segmentation that needs to be fixed (see below). In general, fixing those should probably be preferred, but exclusion is also an option. In general, you have to weigh which of these two options aligns more with your goals and what the issues are arising from.

Some examples:

1. Bad MRI image quality, e.g. due to motion: motion can severely affect a lot of downstream measures including cortical thickness. [1,2] Therefore, exclusion should be considered.
2. Biases: some anatomical features (and therefore issues) may be correlated with specific groups, so if similar errors are found repeatedly, any correlation to such features should probably be analyzed. Also, we would love to hear about them so we can address them.
3. Dataset size: Sometimes exclusion is not an option, because the dataset size really requires every cases, especially if motion and therefore image quality is correlated due to a medical condition, but be careful about causality, some effects may be explained by the motion (or artifacts in the image) themselves.
4. Biases by different methods: If you edit/correct segmentations/surfaces, this means the method to process the image changes as well, which is a possible source of biases. So you should consider if the effect you are measuring could also have been caused by you editing, i.e. whether your editing is introducing a bias itself.

Some of these considerations are obviously very advanced, but it always good to be cognizant of assumptions and biases.

As for your examples and questions

1. Which artifacts are acceptable variations
This very much depends on your analysis and data, the more subtle the effects are, the more your should question your practice. In effect, you should ask whether an observed effect is explainable by segmentation/processing biases or not.

2. Which artifacts are critical and invalidate the subject,
Usually, only very bad image quality requires full exclusion. Again, the more subtle the effect, the more strongly you should consider exclusion/editing/etc.

3. Why FastSurfer discourages manual edits more strongly than FreeSurfer.
As stated above, manual edits are not discouraged, we are trying to establish a culture of understanding what impacts results and biases statistical evaluations, as explained above and researchers often have to understand, document and analyze issues in the results to verify statistical analyses are not biased.

4-1) The brain mask (or mask) appears hollow in the central region.
This is an intended outcome. What happens here is that there is a gap between cerebellum and cortex/cortical white matter. This gap of non-brain tissue is in fact so large that the filling algorithm does not fill it. It would only be an issue, if you actually detected brain tissue in this "cavity" (i.e. the hole in the brain mask).

If you want to fill this cavity, you can easily do so with (biary_fill_holes)[https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.binary_fill_holes.html#scipy.ndimage.binary_fill_holes]. See also #733 .

4-2) Regions near the skull are misclassified as white matter in the segmentation.
This should not happen. If you find multiple such cases, we would welcome you reporting such issues.
I would probably attempt to fix this via editing the asegDKT segfile.

4-3) Underestimation of gray matter appears present.
This one is hard for me to judge without having a look at the image.
It might be this is actually down to limited/coil sensitivity in the frontal region.
It might be addressable by Bias field correction, but the effect is so local, the biasfield correction tools cannot actually pick up this detail.
An interesting question would definitely be, whether this type of issue happens often on the scanner, of whether this particular participant had a particularly large head.
Alternatively, this might be addressable by editing the asegDKT segfile.

4-4) Due to skull-adjacent white matter misclassification, the white and pial surfaces follow incorrect trajectories.
and
4-5) Due to skull-adjacent white matter misclassification, the white and pial surfaces form circular artifacts.
Essentially, these are the same issues as/consequences of 4-2, so fixing 4-2 should also address this problem.

4-6) When the colormap is set to heat, white matter is displayed in red. In another subject, the same region was displayed in yellow.
Segmentations should be viewed with a Lookup Table and probably the FreeSurferColorLUT, this ensures consistent displaying.

In Heatmap mode, Freeview automatically selects window and offset, which can lead to inconsistent/unreproducible views.

[1] Clemens Pollak, David Kügler, Monique M.B. Breteler, Martin Reuter, Quantifying MR head motion in the Rhineland Study – A robust method for population cohorts, NeuroImage, Volume 275, 2023, 120176, ISSN 1053-8119, https://doi.org/10.1016/j.neuroimage.2023.120176.
[2] Martin Reuter, Nicholas J. Schmansky, H. Diana Rosas, Bruce Fischl, Within-subject template estimation for unbiased longitudinal image analysis, NeuroImage, Volume 61, Issue 4, 2012, Pages 1402-1418, ISSN 1053-8119, https://doi.org/10.1016/j.neuroimage.2012.02.084.

[julii-a]

I think I may have misunderstood the statement that “FastSurfer discourages manual edits.” Thank you for clarifying this point in detail.

That said, I am still wondering to what extent manual edits are expected or recommended. In my dataset, issues corresponding to cases like 4-4 and 4-5 are fairly common.
I am unsure whether it is appropriate to manually correct all of these cases, or whether such issues should instead be considered systematic bias and treated as acceptable to some degree.

In contrast, for cases like 4-2, it seems more appropriate to apply manual corrections. However, I am uncertain whether every instance—such as situations where the segmentation itself appears reasonable, but the pial and white matter surfaces show inaccuracies near the parietal or occipital boundaries—needs to be corrected exhaustively.

Additionally, regarding volumetric measures, is it sufficient for the segmentation to be correct?
Or are volumetric results also influenced by the accuracy of the pial and white matter surfaces, as cortical thickness is?

[m-reuter]

Hi, from your screenshots it is difficult to tell what is going on. WM should not be labeled outside the brain in the spongy bone. It looks like this is happening frequently in your images and we do not see much of that in our data. You could try a prior bias-field correction step and then run FastSurfer on that.

What images are these (scanner, field strength, acquisition resolution, sequence)?

4-3 looks to me like there are motion artefacts and generally very blurry low contrasts. Are these diseased cases? What disease?
And as David said, don't use heat colormap, we would need to look at the original image next to the segmentation (using the LUT for colouring) to see what is what.

About your last question, if you look at volumetric results after the segmentation module, then surfaces have no influence. Fro the results in the final aseg.stats etc after the surface stream, segmentations are edited with surfaces, but this affects mainly cortical WM and GM estimates. Same as FreeSurfer.

[julii-a]

Hi, from your screenshots it is difficult to tell what is going on. WM should not be labeled outside the brain in the spongy bone. It looks like this is happening frequently in your images and we do not see much of that in our data. You could try a prior bias-field correction step and then run FastSurfer on that.

What images are these (scanner, field strength, acquisition resolution, sequence)?

4-3 looks to me like there are motion artefacts and generally very blurry low contrasts. Are these diseased cases? What disease? And as David said, don't use heat colormap, we would need to look at the original image next to the segmentation (using the LUT for colouring) to see what is what.

About your last question, if you look at volumetric results after the segmentation module, then surfaces have no influence. Fro the results in the final aseg.stats etc after the surface stream, segmentations are edited with surfaces, but this affects mainly cortical WM and GM estimates. Same as FreeSurfer.

Thank you for the clarification.

1. We will try applying a prior bias-field correction and rerun FastSurfer as suggested.
2. The data were acquired on a 3T Siemens MRI scanner using a T1-weighted MPRAGE sequence with a 1×1×1 mm³ voxel size.
3. For case 4-3, the subject is a healthy older adult.
4. If WM is still labeled outside the brain after applying bias-field correction, would you recommend performing manual corrections systematically, or would it be more appropriate to treat this as a systematic bias and handle it at the analysis level (e.g., through exclusion criteria or sensitivity analyses) rather than through exhaustive manual editing?

[m-reuter]

Hmm, surprising that the segmentation produces WM in the dura and spongy bone for 1mm MPRAGE on a 3T Siemens. Those images should be covered by our training really well. I do see motion artefacts and that can lead to issues in individual cases, but should not be the majority.

If this is in individual cases and your dataset is small (say 10-30 cases) I would recommend to edit it, especially the incorrect WM at the dura and bone. If you have hundreds of cases and this occurs in a few only, I would not go through the effort, unless this region is of specific interest for your research question.

[julii-a]

Hmm, surprising that the segmentation produces WM in the dura and spongy bone for 1mm MPRAGE on a 3T Siemens. Those images should be covered by our training really well. I do see motion artefacts and that can lead to issues in individual cases, but should not be the majority.

If this is in individual cases and your dataset is small (say 10-30 cases) I would recommend to edit it, especially the incorrect WM at the dura and bone. If you have hundreds of cases and this occurs in a few only, I would not go through the effort, unless this region is of specific interest for your research question.

Thank you for your reply. I have one further question.
When performing bias field correction, should the pipeline be re-run with the --edits flag, or is it correct to run FastSurfer again without using the --edits flag?

[m-reuter]

When you bias-correct the input, you would run the pipeline from scratch into a new output directory with the bias field corrected image. There you would not specify the --edits flag.

[julii-a]

When you bias-correct the input, you would run the pipeline from scratch into a new output directory with the bias field corrected image. There you would not specify the --edits flag.

I have tried both bias-field correction and manual edits.

Based on my observations, it seems that the aseg itself is not problematic. Rather, the issue appears to be white matter leaking into the dura/bone region. I therefore edited the white matter segmentation and re-ran surf_only, but even then I confirmed that the pial surface still extends beyond the aseg and into the dura region where no white matter should be present.

I would like to ask the following questions:

1. Is editing wm.mgz not considered the standard or recommended approach?
   Would it have been more appropriate to edit wm.filled.mgz instead?

2. I assumed that editing the aseg was the primary correction step.
   However, in this case, the dura region does not have any aseg labels assigned (and visually, the aseg does not extend into the dura), while white matter misclassification is still present there.
   In such a situation, is there a recommended way to address the issue by editing the aseg, or is aseg editing not applicable here?

3. At this point, it seems that editing brain.finalsurf may be the last remaining option.
   However, there are more cases than expected that would require such edits, which makes this approach quite labor-intensive.
   Would it be acceptable to leave the dura-adjacent white matter unedited and only correct brain.finalsurf, or would that introduce inconsistencies or biases?

Alternatively, if my overall approach to correcting this issue is flawed, I would greatly appreciate your guidance on what the correct workflow should be.

1. before

2. after: editing wm.mgz and re-ran surf_only
   This is not the exact same location, but it is shown here as an example.

Thank you very much for your time and help.

[m-reuter]

I see. So the segmentations look good, but the pial surface extends too far outside. Aseg edits do not make sense if the aseg is already good.

Also you say that WM goes too far, but in these screenshots it is only the Pial that goes a little too far (like around 1-2 voxels). Those are two very different things with different ways to edit them. Our edits are described here:
https://deep-mi.org/FastSurfer/dev/overview/EDITING.html

Generally for Pial edits you would edit brain.finalsurf and for wm the filled.mgz if the edits to aparc.DKTatlas+aseg.deep.manedit.mgz do not help.

When comparing what an edit actually helped, you need to look at exactly the same region. comparing one region with another does not give us any information.

Generally FreeSurfer often goes out too far with the pial surface, if this is just 1-2 voxels on a small portion of the cortex, people usually don't edit that.

[julii-a]

I see. So the segmentations look good, but the pial surface extends too far outside. Aseg edits do not make sense if the aseg is already good.

Also you say that WM goes too far, but in these screenshots it is only the Pial that goes a little too far (like around 1-2 voxels). Those are two very different things with different ways to edit them. Our edits are described here: https://deep-mi.org/FastSurfer/dev/overview/EDITING.html

Generally for Pial edits you would edit brain.finalsurf and for wm the filled.mgz if the edits to aparc.DKTatlas+aseg.deep.manedit.mgz do not help.

When comparing what an edit actually helped, you need to look at exactly the same region. comparing one region with another does not give us any information.

Generally FreeSurfer often goes out too far with the pial surface, if this is just 1-2 voxels on a small portion of the cortex, people usually don't edit that.

Thank you for your quick response.

Since I have not previously performed manual edits in FreeSurfer, I realize that I am still unclear about the exact and correct editing procedures. I would appreciate your guidance on the following points.

As mentioned earlier, in my case the white matter extends into the dura region, and the pial surface also intrudes into the dura, while the segmentation (aseg/aparc) itself appears to be correct.

1. In this situation, is it sufficient to edit only the pial placement using brain.finalsurfs(.manedit).mgz, or should both the white matter (wm.mgz) and the pial surface be edited?

2. How should white matter that incorrectly extends into the dura be handled at the aseg level?
   Since the aseg appeared correct in my case, I removed the dura-side voxels directly in wm.mgz using Shift + left click in Freeview and saved the edited wm.mgz.
   Is there a recommended way to address this kind of white matter error through asegdkt_segfile editing?

3. Could the issue be that I edited and saved the changes directly in wm.mgz
   Alternatively, should I have edited wm.filled.mgz instead of wm.mgz?
   In such cases, should the edits be saved as 'wm.auto.mgz' or as 'wm.filled.auto.mgz'?

4. Is there a reliable way to visually assess whether an edit corresponds to approximately 1–2 voxels?
   Because I am using WSL, I have difficulty interacting with the Freeview GUI (mouse clicks are not reliable), and therefore I cannot use the measurement tools. As a result, it is hard for me to judge what constitutes a 1–2 voxel edit.

5. exmaple subject

5-1) Original files (wm.filled.mgz and aseg)

5-2) Files after re-running the surface reconstruction following edits (wm.filled.mgz and aseg)

5-3) Files after re-running the surface reconstruction following edits (wm.mgz and aseg; equivalent to the subject in case 2 above)

It seems possible that the issue arose because I did not fully remove the white matter voxels from wm.mgz. In that case, I would like to ask whether sufficiently correcting the white matter alone would generally be enough, such that no direct pial edits are required.

Alternatively, I am wondering whether this situation should instead have been addressed by directly editing the pial placement (rather than removing white matter voxels), and under what circumstances that would be the preferred approach.

Thank you very much for your time and assistance.

[m-reuter]

Hi, since your segmentation looks good, you would not need to edit that. The issue happens later when we use FreeSurfer tools to create the wm and filled. For some reason it labels wm too far outside. As described on the edit pages, you should edit the wm.mgz . Our documentation is not very clear there whether to edit wm or filled or both. I think wm.mgz should be enough, make sure you use the manual labels 1 for deletion and 255 for inclusion.

Also you mention wm.filled which I think does not exist. Just use wm.mgz or filled.mgz, no need to create an auto or manedit file.

( @dkuegler can you confirm this ? we should be more clear in the edits.md file. Maybe we should mention there that the label IDs are important so that the file can be recognised as having edits, and no manedit file is needed).

So for WM edits I think you did not remove some of the voxels and therefore the blue surface still goes out too far. Once you delete all non-wm voxels (setting to 1) you should see a good WM surface. Check then if the pial still goes out too far. If so you can edit brain.finalsurf.mgz . I think here you need to create a copy named ...manedit... and edit that one. (@dkuegler also here the documentation is not really very clear on how to "open" that file if it does not exist).

so

1. wm.mgz (or filled.mgz not totally sure, either one seems to work in your example above)
2. not in aseg if that looks already good. The problem starts later in your case.
3. Directly wm.mgz or filled.mgz see 1.
4. Freeview is the best way. WSL can be tricky, a Mac or linux would work better. In order to edit it is good to see both the surface and the image at the same time and make the edits, for this FreeView is the best tool.