Yoshinobu Sato, Akinobu Shimizu, Kensaku Mori and Takayuki Kitasaka
This section describes algorithms for analysis of the abdominal organs using CT and MR data. Segmentation algorithms of the organs are described, and then extraction and analysis of internal structures, that is, vessels and abnormal regions, are discussed for the purpose of diagnosis and surgical planning assistance.
The liver is one of the most-studied abdominal organs with respect to segmentation and further analysis. Early investigations of liver segmentation were based on slice- by-slice 2D image processing [24, 98]. Since the development of multi-detector row CT (MDCT) in the late 1990s, however, volume imaging of the abdomen, that is, image acquisition with a thin-section thickness, regarded as 3D isotropic imaging, has become popular. Therefore, liver segmentation methods based on fully 3D image processing were developed [186, 264]. They demonstrated potential clinical utility in tumor detection and surgical planning. Later on, two noteworthy papers on liver segmentation were published [168, 229], in which computational anatomy (CA) models, that is, a probabilistic atlas  and an SSM , were used as prior knowledge of the liver shape and location to formulate the segmentation problem as a MAP estimation based on Bayesian theorem. Representative methods for liver segmentation typically use an SSM or a probabilistic atlas [118, 141, 331]. The remaining part of this section describes automated liver segmentation algorithm using CA models from upper abdominal CT data and then mentions applications to diagnosis and surgical planning assistance.
Anatomy of Organs and Tissues Adjacent to the Liver The liver is the largest abdominal organ, located in the right upper quadrant of the abdominal cavity, just under the diaphragm, which separates the thoracic and abdominal cavities. The shapes of the liver dome and right lung base are strongly constrained by the diaphragm’s variations in contour during the respiratory cycle. The liver consists of large right and left lobes and small caudate and quadrate lobes. The shape of the right liver dome correlates closely with the right lung base via the intervening diaphragm. The contour of the inferior surface of the liver contains impressions from the right kidney, duodenum, gallbladder, and right colon. The liver is also surrounded by the lower ribs and abdominal musculature. Although CA models should deal with the above multi-organ relations systematically by mathematical models, we focus on the liver and utilize these relations in a manually specified manner in this subsection. The topic of modeling multi-organ relations will be addressed in a subsequent subsection.
CA Models of the Liver and Their Application to Segmentation from CT
Images CA models, typically probabilistic atlases and SSMs, are represented in the reference frame (coordinate system), which needs to be determined from input CT volume when they are utilized. There are two main approaches to define the reference frame, that is, an organ-centered frame and an external frame. The organ- centered frame is defined by features inherent in the target organ, and shape priors are represented by probabilistic atlas and the SSM. The external frame is defined by features of external structures (such as the lungs and ribs), and both shape and location priors are represented, where the locations of the target organ are represented relative to the external structures.
Fig. 3.68 Example of external reference frame for spatial normalization 
Figure 3.68 shows the determination processes of an example of the external reference frame [221, 222]. The reference space is defined based on the upper abdominal cavity, whose shape and location are constrained by the diaphragm, bones, and abdominal musculature. To determine the reference space, an approximated bounding box of the abdominal cavity is utilized, whose top plane is constrained by the diaphragm and side planes by the bones. More specifically, the upper plane corresponds to the axial plane tangential to the right dome of the diaphragm (which covers the liver surface), and the four side planes correspond to the sagittal right- and leftmost planes and the coronal front and back planes of the musculature and ribs. The lateral and anteroposterior dimensions of the reference space are normalized to the mean dimensions of all the patients. The right dome of the diaphragm separates the dome of the right lobe of the liver and the base of the right lung. Thus, it can be extracted by locating the right lung base. The lung and bone regions are not affected by contrast agents and are well delineated from other tissues in CT data because of their very low and very high densities, respectively. Therefore, these regions can be segmented in a stable manner from CT data irrespective of differences in contrast enhancement protocols. All the patient CT volumes are translated and scaled so as to be aligned to the normalized reference space, which defines the reference frame. This initialization process is called spatial normalization or spatial standardization.
Once spatial normalization has been completed, CA models such as probabilistic atlases and SSMs can be applied to the target CT data to perform segmentation. Figure 3.69 shows a probabilistic atlas and SSM of the liver, which are represented in the reference frame based on the bounding box approximating the upper abdominal cavity. Segmentation using CA models is formulated as a MAP estimation by combining data fidelity terms, which are typically derived from intensity models
Fig. 3.69 Computational anatomy (CA) models of the liver. (a) Probabilistic atlas. Volume rendering of the probabilistic atlas is displayed by assigning opacities proportional to probabilities to voxels. (b) SSM. The shape located at the center is the mean shape, and the shape variations of the first and second modes are indicated
or edge localization error models, as discussed in Chap. 2. Segmentation using a probabilistic atlas is regarded as voxel-wise MAP estimation, which does not require any initialization and nonlinear optimization (see Fig. 2.17 for typical segmentation processes in Chap. 2). Conversely, SSM requires initial values for the shape parameters for their nonlinear optimization. Although the average shape is often used for the initial values for an SSM, it is desirable to combine some methods to avoid stacking in poor local minima. Typically, edges to be fitted to SSMs are searched for near SSM surfaces while SSMs are being fitted to input images. The initial SSM needs to be sufficiently close to the true organ surfaces for successful edge search. The mean shape, however, may sometimes largely deviate from the true shape and fail to capture the true edges. To overcome this problem, probabilistic atlas-based segmentation results can be used for initial values for SSM- based segmentation, instead of the mean shape. Then these parameters are used for the initial values for subsequent SSM-based segmentation .
The criteria for evaluating the performance of SSM are known as specificity (property of maintaining the characteristics specific to the organ shape) and generality (property of representing any shapes accurately). Specificity and generality are a trade-off. To overcome this, a hierarchical SSM (H-SSM) has been developed [221, 222]. A coarse-to-fine strategy is adopted for the H-SSM, in which the top level of SSM is first fitted, and then its estimated shape parameters are used as initial values for subsequent fitting of sub-shape SSMs. Figure 3.70 shows an H-SSM of the liver, in which the whole liver shape is gradually decomposed into sub-shapes and SSMs of the sub-shapes, in addition to the whole liver shape, are constructed. By using hierarchical SSMs, generality and specificity are better balanced. While the top level of SSM is more specific to the liver, the lower levels of SSMs are more accurate in terms of generality.
Fig. 3.70 Shape decomposition for hierarchical SSMs (H-SSMs) 
Figure 3.71a shows the result of probabilistic atlas-based segmentation, in which initialization was not required after the spatial normalization. Figure 3.71b shows the result of subsequent H-SSM-based segmentation, in which the initialization was provided by the probabilistic atlas-based segmentation result. Figure 3.71c shows a result of graph cut refinement (described in Chap. 2) for the H-SSM-based segmentation result. In Fig. 3.71, clear improvements of segmentation accuracy are observed as the segmentation progresses from (a) to (c).
So far, the CA models of the liver have been generic; that is, they are assumed to cover the variabilities in various patients except for spatial normalization. If additional information on a patient is provided, however, we can make the CA models to represent a patient-specific variability, in which generality and specificity will improve by assuming conditions specific to the patient of interest. One way to construct a patient-specific CA model is to use intermediate results during the segmentation process . In the previously mentioned method, probabilistic atlas-based segmentation was first performed, and its result was used as an initial state for subsequent SSM-based segmentation. In this method , the probabilistic atlas-based segmentation result provides conditions specific to the patient, which are used to generate a conditional SSM  (described in the previous section) adaptive to the patient. The feature parameters are calculated on the gross shape of the liver observed from the probabilistic atlas-based segmentation result, i.e., the object volume, the area of the projected object in the coronal plane, the 50th percentile point of the x-coordinate, and so on. Figure 3.72 shows a schematic diagram of the approach. Given the observed conditions as features obtained from the target CT data, the conditional SSM is generated specifically to the given conditions. The right bottom frame of Fig. 3.72 shows a simplified example of a
Fig. 3.71 Segmentation of the liver from CT volume. Left: 3D shape of segmented liver region. Distance error from the manual trace (ground truth) surface is color-coded according to the indicated color bar. Right: Yellow and green contours denote manual trace and automated segmentation, respectively. (a) Probabilistic atlas-based segmentation. (b) H-SSM-based segmentation. (c) Graph cut refinement
conditional SSM of the liver, where the feature x0 is the lateral dimension of the bounding box of the liver region obtained by probabilistic atlas-based segmentation. Because the error from the width of the true liver shape is unavoidable in x0, the error model of x0 is combined with the conventional conditional SSM so as to
Fig. 3.72 Schematic diagram of conditional SSM generation from observed condition 
incorporate specificity inherent in the patient while maintaining sufficient generality. Therefore, this conditional SSM represents the remaining ambiguity and expected error after probabilistic atlas-based segmentation rather than inter-patient variability. These generated conditional SSMs are shown to be particularly useful for accurate segmentation of livers with largely deformed shapes .
Role in Diagnosis and Therapy of Liver Diseases Computational anatomy approaches including machine learning are useful for characterizing diffuse and focal liver abnormalities such as cirrhosis and tumors. Surgical planning for tumor resection needs precise patient-specific anatomy, including relations of tumors to vessels, and locations in anatomical segments of the liver. In the following, modeling and application for diagnostic and therapeutic assistance are described.
Computer assistance is most commonly used to target tumors. There are two main approaches for tumor detection and segmentation. One approach assumes presegmentation of the liver region and the other does not. The former fails if the pre-segmented liver misses the tumor region(s), which often occurs because tumors usually have different intensity properties compared with normal tissues. The nonsegmented approach may suffer from more false positives due to larger search areas. In the former approach , inaccurate segmentation areas in the initial conventional liver segmentation are detected by incorporating the proposed shape ambiguity measure in subsequent level set segmentation. Figure 3.73 shows a typical
Fig. 3.73 Segmentation of the liver and tumors . Blue and yellow regions are manually and automatically traced regions, respectively, and green regions are the overlapped regions. Red regions denote false-positive detections by the automatic tumor segmentation
result of segmentation of the liver that includes tumors. The tumor segmentation is performed inside the segmented liver regions. In one representative method of the latter, ensemble learning was used for segmentation of tumor regions . Recently, a robust statistics mechanism was incorporated in ensemble learning to significantly improve tumor segmentation accuracy .
Liver cirrhosis/fibrosis is one of the important diseases of the liver. While biopsy is still regarded as the method for definitive diagnosis, some noninvasive diagnostic methods, such as ultrasound/MR elastography and blood tests, are showing increased progress in diagnostic accuracy. Liver morphology can provide useful diagnostic information because cirrhotic livers, which may initially enlarge, subsequently shrink in size and are known to show characteristic shape deformations  as shown in Fig. 3.74, which shows 3D visualizations of healthy (fibrosis stage 0) and cirrhotic (fibrosis stage 4) livers. The traditional quantitative imaging method based on the ratio of the left to right lobe volumes  was successful to some extent. SSMs are expected to well capture the characteristics of the organ shape, and some successful results are reported in the brain . One method to use shape deformations for fibrosis quantification is to relate the shape parameters of SSMs
Fig. 3.74 3D visualizations of liver shape (a) side view, (b) bottom view . Top: Typical cirrhotic liver (fibrosis stage 4). Bottom: Healthy liver (fibrosis stage 0)
with the fibrosis stage by using support vector regression (SVR) . Accuracy in fibrosis stage estimation improved by adopting the method using SSMs and SVR (approximately 90% in sensitivity and specificity)  in comparison with the traditional method based on the left and right lobe volumes (approximately 60-80% in sensitivity and specificity).
Understanding patient-specific segmental anatomy of the liver is important, especially for surgical planning for tumor resection. The computational approach to approximating patient-specific segmental anatomy from CT images was addressed by two seminal papers in the early 2000s [36, 252]. Selle et al.  demonstrated that accurate and detailed vessel extraction and classification were critical for accurate approximation. Several efforts for virtual reality systematization  and automating the segmental anatomy approximation  according to the Couinaud nomenclature have been made since these papers were published. Figure 3.75 shows examples of automatically classified vessels and generated Couinaud liver segments based on them .
Fig. 3.75 Vessel extraction and classification of Couinaud liver segments . (a) Vessel enhancement (left) and labeling. (b) Anterior (left) and posterior (right) views of resulted segmented anatomy