# Probabilistic Atlas-Based Segmentation

A probabilistic atlas represents the existence probability of an organ at each voxel and was originally explored in brain segmentation followed by organ segmentation of a human torso. Details of a probabilistic atlas can be found in Sect. 2.3.5.1. This section focuses on segmentation algorithms of organs in a human torso based on a probabilistic atlas. A typical probabilistic atlas-based segmentation algorithm is MAP estimation of an organ [151], in which a prior probability is defined by a probabilistic atlas. Given a feature vector *x **2* R^{d}, posterior probability is represented by the following equation using Bayesâ€™ theorem:

where**p***(x***
***)* shows the likelihood of a vector*x* of organ n, which is widely assumed to be a mixture of Gaussian distributions as follows:

where * ^_{n m}* and

*E*

*are an m-th average vector and an m-th covariance matrix of an organ*

_{nm}*in an unseen image which can be estimated by an EM algorithm or variational Bayes [33]. An atlas-guided EM algorithm is an optional choice to achieve low computational cost and high accuracy [124]. Parzen window estimation is an alternative choice to define a vector*

**n***of organ*

**x***(refer details of nonparametric probability density function estimation to [34]). A probability (density) function is defined by the following equation:*

**n**

where * N* is equal to number of organs to be segmented plus one that corresponds to the background.

As mentioned earlier in this section, prior probability is given by a probabilistic atlas which makes the segmentation results more accurate especially from the point of view of anatomy. Figure 2.23a, b are an axial section of an original abdominal CT volume dataset (a) with prior probability of liver, or a probabilistic atlas, in which whiter color represents higher probability of existence of liver (b). Parts (c) and (d) illustrate the likelihood of liver and that of background, in which numbers of Gaussian of the mixture distributions are 2 for liver and 3 for background. Parts (e)

Fig. 2.23 Example of maximum a posterior-based liver segmentation using a probabilistic atlas of liver. (a) Original CT image. (b) Probabilistic atlas of liver. (c) Likelihood of liver. (d) Likelihood of background. (e) Posterior probability of liver. (f) Posterior probability of background. (g) Segmentation result with a probabilistic atlas. (h) Segmentation result without a probabilistic atlas and (f) present posterior probability of liver and that of background, respectively, and parts (g) and (h) show segmentation results with and without the probabilistic atlas of liver, respectively. The likelihood parts (c) and (d) tell us that the liver is algorithmically enhanced but parts of surrounding tissues and organs, such as muscle and pancreas, are also enhanced, which might lead to false positives in segmentation. In contrast, the probabilistic atlas of part (b) appropriately restricts the existence area of liver, resulting in better segmentation results of part (g) compared with the results of part (h) (performed without the probabilistic atlas). The JIs are

0.851 for part (g) and 0.536 for (h), respectively.

Extensions of a probabilistic atlas of organs in a human torso can be found in several papers. For example, a multi-organ probabilistic atlas can be easily derived by normalizing probabilities of multiple organs at each voxel under constraints that the summation of probabilities of whole organs/tissues is equal to one. Figure 2.24 presents a result of twelve-organ segmentation using a multi-organ probabilistic atlas from a noncontrast CT volume [124, 152], in which MAP-based segmentation using a multi-organ probabilistic atlas was followed by a multiple level-set method with interaction mechanism between neighboring level-set functions. In [153], probabilistic atlases of multiple organs were constructed and used in graph-cuts- based fine segmentation, in which an energy term is defined using probabilities of multiple organs.

A modification of a probabilistic atlas was reported in [151], where MRF was adopted to improve prior probability. In the iterative scheme of multi-organ segmentation, a prior probability of each voxel of interest was updated at each step so as to reduce unnatural non-smooth boundaries, holes, and over-extracted regions by referring to the regions of the organs neighboring to each target organ segmented at the previous step.

Alternative important improvement for organ segmentation was a patient-specific probabilistic atlas [154-156]. Since a conventional atlas accounts for a whole distribution of existence probability, it is effective in describing shapes around a mean shape but not for atypical shapes located in marginal areas of the distribution. To improve the segmentation performance for an organ with atypical shape, patient- specific atlases were constructed using an SSM [154], multi-atlas [155], and sparse modeling-based [156] approaches. It was proven that a patient-specific probabilistic atlas was effective in segmenting an organ with a typical shape to a statistically significant degree [156].

In general, a probabilistic atlas is constructed using healthy organs or mostly healthy organs, such as an organ with small lesions. Consequently this might not account for an organ with large pathological lesions, which frequently change the appearance as well as the shape of an organ radically, resulting in low posterior probability, even though a probabilistic atlas indicates high existence probability of a target organ. In such cases, the probability of pathological lesions might be helpful to recover the low posterior probability of a voxel in a lesion and segment an organ with large lesions. An algorithm was proposed [156] to construct a probabilistic atlas of pathological lesions and a probabilistic atlas of a liver simultaneously using sparse modeling with lesion bases. The result of applying the algorithm to livers

Fig. 2.24 MAP-based segmentation using a multi-organ probabilistic atlas was followed by a multiple level-set method. (a) An input noncontrast upper abdominal CT volume. (b) Multi-organ segmentation result. The twelve target organs are the esophagus, heart, stomach, liver, gallbladder, pancreas, spleen, left and right kidneys, inferior vena cava, aorta, and splenic vein between the liver and spleen with large pathological lesions in CT volumes proved its statistical effectiveness comparing with a conventional probabilistic atlas-based algorithm.