# Multiple Organs, Anomaly, and Lesions

In this section, two important tasks are described: multiple organ segmentation and anomaly recognition. The methods described in the previous section segmented one target organ by using its single model. Such methods would be insufficient for the solutions because they cannot handle the two problems. First, any information on the dependencies between the locations and shapes of the multiple organs cannot be used if each organ is segmented independently. For example, some of the resultant regions of the organs can overlap each other and be inconsistent with human anatomy. Second, the models described in the previous sections cannot represent some anatomical anomalies, such as variations in the number of some anatomical entities. In the following section, some approaches to these issues are introduced.

## Multiple Organs

A CA model (SSM) for multiple organs can be constructed by simply extending an SSM for a single organ, such as a PDM [69] or an LSDM [180]. For example, a collection of SSMs, each of which corresponds to an organ and is constructed individually, might be a solution for an SSM for multiple organs. This simple solution, however, suffers from an overlap problem between neighboring organs. The relationship between neighboring organs should be incorporated to reduce the overlap and can be modeled by connecting feature vectors, each of which corresponds to an organ, followed by statistical analysis of the concatenated vectors [181]. Other modeling approaches include logarithm of the odds (LogOdds) [128], label space [182], and isometric log-ratio mapping [183], which were originally designed for multiple objects (organs) and thus are unaffected by overlap of neighboring organs. This section describes an SSM for multiple organs based on PDM or LSDM followed by LogOdds and label space-based models. Note that all of the models mentioned above are SSMs in which all shape features are simultaneously exploited without any weight, or priority, in the modeling process. In contrast, if features highly correlated with target organs are available, a conditional SSM [184-188] will work, which will be also explained in this section.

1. SSM for multiple organs based on a point distribution model and an LSDM Let *x* be a feature vector of *M* organs, which consists of feature vectors xi,x_{2},... *,x*_{M}, each of which corresponds to each organ. Specifically, two of the vectors consist of point coordinates of multiple organs when based on a PDM and a signed distance function (SDF) when based on an LSDM. An SSM for multiple organs is a statistical model of a probability distribution of *x* or *p(x).* Since the original feature vectors are defined in a very highdimensional space and might be contaminated by noise, statistical analysis is applied to reduce dimension and noise. Let v be a shape feature vector whose length is L and element is v_{1}, *v*_{2}*,...,v _{L},* which are shape features extracted by statistical analysis of a training data set of

*x*, e.g., principal component scores (PCS) extracted by principal component analysis (PCA). Consequently, the probability density function

*p(x)*is approximated by

*p(v).*The following equation is a probability density function if the distribution of v can be assumed to be Gaussian:

where *p.* is an average vector and *E* is a covariance matrix of shape feature vector v.

Variation in location and shape of multiple organs is an accumulation of individual differences of an organ, resulting in a larger variation in pose and shape. This makes model construction difficult and results in low performance. One possible solution to handle such large variation is to separate variations in

Fig. 2.28 **An SSM of 14 organs: heart, esophagus, stomach, liver, gallbladder, pancreas, spleen, left and right kidneys, inferior vena cava, aorta, splenic vein, portal vein, superior mesenteric vein**

pose from those in shape in a statistical modeling process. One group [181] proposed an SSM, or an LSDM, for multiple organs, in which Procrustes alignment was applied to a training label dataset of each organ individually to separate pose, or rotation and translation parameters, from shape. The extracted parameter sets of rotation and translation as well as shape were analyzed by PCA to build a rotation model, a translation model, and a shape model for each organ. The final step was to integrate all models of multiple organs by concatenating PCS vectors of organs into a vector followed by PCA of the concatenated vectors. Figure 2.28 shows fourteen organs generated from nine different parameter sets in a subspace of the proposed SSM, in which the horizontal axis corresponds to the first PCS and the vertical axis corresponds to the second PCS. In terms of generalization, specificity, and overlap between neighboring organs, the performance of the proposed algorithm was proven to be superior to that of a multi-organ model without separation of pose and shape. The SSM for multiple organs will be incorporated into a multi-organ segmentation algorithm [124] to boost segmentation performance.

2. LogOdds and Label space

LogOdds is an example of a class of functions that map the space of discrete distributions to Euclidean space and is employed for multi-organ shape representation [128]. The multinomial LogOdds function logit P_{M} ! R^{M} and the generalized logistic operations are used to bridge between the manifold of signed distance maps (SDMs) and the linear space of LogOdds:

where *pi(e* [0,1]) is the i-th probability and *p _{M}*(e [0,1]) is the last entry of a discrete distribution

*p(e*P

_{M}), where P

_{M}is an open probability simplex, or the space of discrete distributions for

*M*labels which correspond to background label and

*M โ 1*object (organs) labels.

The inverse of the logit function is the generalized logistic function:

Note that logit {logit(p)|p e P_{M}g is the *(M โ* 1)-dimensional space of LogOdds induced from P*M* and is equivalent to an *M* โ 1 -dimensional real vector space that provides closed operations for addition and scalar multiplication, which is not the case for an SDM-based model or LSDM. Pohl et al. [128] showed that the LogOdds variant was superior to the SDM model in an experiment segmenting 20 subject brains into subcortical structures.

Label space [182] is a multi-organ shape representation that maps *M* organ labels to the vertices of a regular simplex, in which a scalar label value is changed to a vertex coordinate position in an *M* dimensional space. As demonstrated in Fig. 2.29, the regular simplex is a hyperdimensional analogue of an equilateral

Fig. 2.30 **An example of organ assignment to each vertices of a triangle**

triangle with * M + 1* vertices capable of being represented in M dimensions. Figure 2.30 is an example of an assignment of two organs and background to vertices of a triangle. Lying in a linear vector space, this space has several desirable properties: all labels are equally separated in space, addition and scalar multiplication are natural, label uncertainty is expressed as a weighted combination of label vertices, and interpolation is unbiased toward any label including the background. Malcolm et al. demonstrated that algebraic operations may be done directly [182]. Label uncertainty is expressed naturally as a mixture of labels, interpolation is unbiased toward any label or the background, and registration may be performed directly.

An alternative probabilistic multi-organ shape representation is the isometric Log-Ratio, which forms a vector space, isometric, and thus isomorphic to the probability simplex, and results in a nonsingular covariance [183]. These authors claimed that these properties did not exist together in any previously offered probabilistic CA work. They demonstrated how the lack of some of these properties degraded the results, e.g., statistical analysis using linear PCA.

The main advantages of the above multi-organ modelings are that they do not suffer from overlap between neighboring organs. However, they sometimes result in unnatural shapes, in particular, boundaries contacting neighboring tissues or organs.

3. Conditional SSMfor multiple organs

When features highly correlating with a feature vector * x* of a target organ are available, it is effective to model the distribution of

*using a conditional SSM [184-188], for example, a conditional SSM of gallbladder given a liver. Let organ*

**x**1 and organ 2 be a target organ and a conditional organ, respectively. Feature vectors of both organs are denoted by x^ and *x*_{2}*i (i =* 1,2 , N), where *N*

is the number of training labels. Average vectors are given by */i** _{1}* and ะด

_{2}, and covariance matrices

*S*1,2) are computed using the following equation:

_{kl}(k,l =

Average *m* and covariance matrix *K* of conditional distribution of x_{1} given x_{2}, *p(x** _{1}* |x

_{2}) are given below:

A conditional SSM can be derived by statistical analysis of *p(x** _{1}* |x

_{2}), for example, eigenvalue decomposition of

*K*. Variance and covariance of a conditional probability distribution of x

_{1}given x

_{2}, or p(x

_{1}|x

_{2}), are smaller than those of a probability distribution of x

_{1}or p(x

_{1}). Therefore, performance of a conditional SSM of organ 1 given organ 2 is better than performance of an SSM of organ 1 without any condition. Figure 2.31 shows probabilistic atlases of a gallbladder generated by a non-conditional SSM, or a model of p(x

_{1}), and a conditional SSM of a gallbladder given a liver or a model of

*p(x*

*|x*

_{1}_{2}). As shown in this figure, an area with high probability of existence in panel (a) merged with the liver, but did not do so in panel (b), which means that the conditional SSM appropriately restricted the area of the gallbladder with high probability.

**Fig. 2.31 **Probabilistic atlases generated by two SSMs of a gallbladder. The atlases were computed from labels reconstructed by normal random numbers generated in subspaces of SSMs or (**a**) nonconditional SSM and (**b**) conditional SSM given a liver

*liver*

*kidney*

*Uvera^j*

*aorta*

*liver*i. *liveri*

HtierjQ , *liver _{2}*

*liver*

*L-kidney*

*R-kidney*

Type 1

Vi

Type 2 V_{2}

1>pe3

*V,*

**Fig. 2.32 **Organ correlation graph (OCG) and OCG-based multi-organ segmentation. (**a**, **b**) OCG. *Blue* and *red edges* indicate the directed edges from a node in Types 1 and 2, respectively. (**c**)-(**f**) Sequential segmentation steps based on OCG. A *red border* indicates the nodes to be segmented. *Green color* indicates segmented nodes

The selection of a condition organ given a target organ is an important task. Okada et al. [188] proposed an organ correlation graph (OCG) which encodes the spatial correlations among organs inherent in human anatomy as presented in Fig. 2.32. Panels (a) and (b) show OCGs, and panels (c)-(f) present sequential segmentation steps based on the OCG of (b) (see [189] for details of segmentation steps). Panel (b) indicates an OCG of eight abdominal organs (liver, spleen, left and right kidney, pancreas, gallbladder, aorta, and inferior vena cava). This was constructed automatically based on shape predictability by partial least squares regression (PLSR) [189] under the constraints on the three types of organs in panel (b). A set of PDM-based conditional SSMs were constructed from 86 CT datasets obtained with four imaging conditions. These were used in the OCG-based segmentation, resulting in high segmentation accuracy of eight organs.

Finally, we discuss the relationship between an SSM of *p(x _{1}, x_{2})* and a conditional SSM of

*p(x*

*|x*

_{1}_{2}). The following is an equation that connects

*p(x*

_{1},

*x*and

_{2})*p(x*

*|x*

_{1}_{2}).

*p(x*x

_{1},_{2}) =

*p(x*

*|x*

_{1}_{2})p(x

_{2}) where

*p(x*is a marginal distribution of x

_{2})_{2}. As indicated by this equation,

*p(x*is composed of not only

_{1}, x_{2})*p(x*

*|x*

_{1}_{2}) but also

*p(x*which means that, in principle, an SSM of

_{2}),*p(x*is able to describe shapes that can be represented by a conditional SSM of

_{1}, x_{2})*p(x*

*|x*

_{1}_{2}) and an SSM of

*p(x*It is, however, more difficult to construct precisely and use effectively an SSM of

_{2}).*p(x*than an SSM of p(x

_{1}, x_{2})_{1}|x

_{2}), because of the large variance of distribution in a subspace. It should be noted that variances of

*p(x*and

_{1}, x_{2}), p(x_{1}),*p(x*

*|x*

_{1}_{2}) are decreased in the following order:

where Var(x) means variances of x. The higher the correlation between x_{1} and x_{2}, the smaller the variance of p(x_{1}|x_{2}), which means that a conditional SSM will be more effective for image analysis, such as segmentation. When using a conditional SSM, attention must be given to the reliability of the condition, which might be contaminated by noise and measurement error, in particular when the conditional features are measured by an automated process. Tomoshige et al. [190] proposed a relaxation scheme of condition with an error model in measurement, which is applicable to multiple organ modeling and segmentation.