Home Political science
In this section, we will present the findings of the topic modeling results. First, to interpret the topic, we observe the words associated with each topic by calculating the probability and frequency. We then examine content and correlations among topics to ensure that the labels we assign to topics are accurate and reliable. In comparison to past literature, we show that titles cannot predict the variation in issues among partnerships. Finally, we provide a test of China’s leadership as an alternative, more accurate explanation of issues distribution between partnerships.
Figure 1.2 Model diagnostics by number of topics
Words associated with a topic
As discussed above, the output from the model is ten topics, each classified by a series of words. All texts are analyzed in Simplified Chinese but presented in English via Google Translate. STM does not provide labels but assigns only a numeric index, including Probability, FREX, Lift, and Score to each topic, each of which represents one way to understand the topics from the STM model. As Roberts et al. (2014) discuss, these indicators provide different perspectives of the words that are associated with each topic. Probability is the most intuitive to understand and refers to words with the highest frequency in each topic. FREX is the weight of words by their overall frequency and their relative exclusivity to a given topic. Similar to FREX, Lift uses the frequency of words in other topics as a weighting method; thus, words that appear less frequently in other topics are given higher values. Score divides the log frequency of the word in the topic by the log frequency of the word
Figure 1.3 Words associated with each topic
in other topics. With these indices, our next goal is to interpret those words in each topic. Figure 1.3 shows the most relevant terms for each topic by Probability and FREX, which can be helpful to understanding the model.
From the derived topics, we identify eight major topics covering (1) Industrial and Infrastructural Investment, (2) Cultural and Civil Exchange, (3) Sovereignty, (4) International Organizations and Global Issues, (5) United Nations, (6) Trade and Economy, (7) Regional Stability, and (9) Security. We also identify two minor topics—(8) Diplomatic Pleasantries and (10) Political Visits—that mainly serve as descriptive words and provide limited information for our analysis. The associated words and a representative quote for each of the ten topics are described below.
We label Topic 1 Industrial and Infrastructural Investment because words in this cluster mostly consist of trade, infrastructure, development, transportation, rail, technology, energy, and the Belt-Road Project. In Figure 1.4, words with the highest frequency include China’s overseas investment in industries
Figure 1.4 Words and a representative paragraph from Topic 1
and infrastructure, another key focus of China’s foreign policy strategy. The quote in Figure 1.4 is an example paragraph randomly selected from Topic 1, which is focused on industrial and infrastructure investments between China and Bangladesh in a 2016 SP.
The second topic, Cultural and Civil Exchange, consists of various forms of people-to-people communication. In Figure 1.5, representative words include culture, tourism, education, sports, scientific research, media, medicine, and arts. The Confucius Institute and cultural cooperation agreements are typical examples. The quote in this topic is a section from the CSP with Pakistan in 2018 that declares that both countries will engage in new cultural cooperation on “culture, arts, radio, film, television, publishing, and sports.”
Sovereignty is the third topic, which touches on a critical concern of China— sovereignty—and particularly focuses on cross-strait relations. Words in this topic include the One-China policy, Taiwan, unification, territory integrity, internal affairs, and interference. Figure 1.6 presents a quote from the SP with Ukraine in 2011, demonstrating that Ukraine reaffirms its stance on the One- China policy and denies Taiwan’s participation, in any form, in international organizations.
We define the fourth topic as International Organizations and Global Issues,
which includes information regarding climate change, oceans, Afghanistan, governance, and multilateral agreements. Words in Topic 4 focus more on
Figure 1.5 Words and a representative paragraph from Topic 2
Figure 1.7 Words and a representative paragraph from Topic 4
comprehensive global issues such as G20, WTO, and Asia-Pacific Economic Cooperation (APEC) compared to Topic 5, which is more specifically concerned with the United Nations. The quote in Figure 1.7 is a statement from the SP with Germany in 2010, encouraging more economic cooperation via G20 on the basis of equality and representation.
Whereas Topic 4 focuses mainly on universal global issues, Topic 5 more specifically focuses on the United Nations and mainly refers to issues and agencies specifically connected to the United Nations. In Figure 1.8, terms associated with the topic are climate change, Security Council, effective, reform, Charter, convention, resolution, peace, multilateral, and globalization. The request to reform the UN Security Council is an issue frequently brought up in this topic. For instance, the quote in Figure 1.8 is from the CSP with Russia in 2019, demanding the reform of the UN Security Council to guarantee more participation from developing countries.
We classify words associated with the sixth topic as Trade and Economy, for primarily mentioning China’s bilateral trade and economic benefits, especially in regard to the Belt-Road Project. In Figure 1.9, words with the highest frequency in this topic are economy and trade, the Belt-Road Project, mutual exchange, mutually complementary, and commodity. An example quoted from the SP with Mongolia in 2014 encourages more high value-added products in bilateral trade. This topic is also highly associated with Topic 1,
Figure 1.8 Words and a representative paragraph from Topic 5
Figure 1.10 Words and a representative paragraph from Topic 7
but Topic 6 is more focused on bilateral trade over Topic 1 ’s primary focus on financial investments.
The label for Topic 7 is Regional Stability, which consists of words usually involving regional affairs and security such as norms, fundamental, basis, generally accepted, region, peace, stability, and security. The quote in Figure 1.10 is from the SP with Russia in 2010, stating that both countries should work together to maintain the security and stability of the Asia-Pacific region.
We describe Topic 8 as Diplomatic Pleasantries. As shown in Figure 1.11, this topic usually contains words like relations, mutual benefits, equal, and consolidate. The quote in Figure 1.11 is the opening section from the SP with Indonesia in 2011. As these words and paragraphs often serve as the opening of treaties and are too general to convey any valuable information, we consider Topic 8 a relatively meaningless topic that has theoretically little variance across partnerships. However, given that we do not hold assumptions before conducting topic modeling, we have no reason to exclude those sections from the analysis manually.
Security is the label for Topic 9, which includes terms associated with a wide range of security problems, both conventional and non-conventional. It includes issues of military interaction, defense, and non-proliferation, which have always been the key focus of Security. Additionally, Security covers challenges that are gaining greater attention in recent years, such as terrorism,
Figure 1.11 Words and a representative paragraph from Topic 8
extremism, separatism, crime, and drugs. The quote in Figure 1.12 is the statement from the CSP with Egypt in 2016, which denounces terror attacks and supports domestic counter-terrorism actions.
We define the tenth and final topic as Political Visits. Topic 10 comprises words such as government, invitation, engagement, consensus, state visit, meetings, and exchange of views. A quote in Figure 1.13 is a statement from the SP with Ukraine in 2013 that aims to improve bilateral high-level government official visits and exchanges. Topic 10, just like Topic 8, is less relevant to our analysis due to the wording serving as a relatively standard opening of most treaties.
Labeling and interpreting are the most challenging tasks in topic modeling. For instance, words and quotes from Topic 4 (International Organizations and Global Issues) and Topic 7 (Regional Stability) discuss similar issues under different topics. Figure 1.14 presents topic correlations estimated by a model that can help us further understand the topics: It connects topics that are likely to be discussed together within a document by word correlations. We plot a line between topics if their correlation is positive. For instance, Topic 2 (Cultural and Civil Exchange) stands isolated from all other topics, with a negative correlation with all other topics. It is not surprising that the highest correlation, at 0.31, is the one between Topic 8 (Diplomatic Pleasantries) and Topic 10 (Political Visits), as sections within both topics are the opening of
Figure 1.12 Words and a representative paragraph from Topic 9
Figure 1.14 Correlations between topics the treaties. Topic 1 (Industrial and Infrastructural Investment) and Topic 6 (Trade and Economy) are correlated with each other since investment requires funds, intuitively leading to the need to discuss financial institutions. The lowest correlation is between Topic 1 (Industrial and Infrastructural Investment) and Topic 8 (Diplomatic Pleasantries), at about -0.32. This indicates our topic classification is strong, since these topics are unlikely to be brought up together. Although certain issues within different topics may sometimes be related to each other, most of our major topics have low correlation values. In other words, our model successfully divides the content of partnerships into distinct and mostly exclusive categories.
Figure 1.15 shows the estimated topic proportions by different partnership titles. Investment (Topic 1) and Cultural Exchange (Topic 2) make up the highest proportion of text in the partnership documents, although, with each having an average topic proportion of less than 15%, neither dominates the content. These two topics have the highest proportion because China intends to expand economic and security ties with neighboring countries through trade, investment, and development of infrastructure underpinned by the Belt-Road Project. Overall, issues involving high politics (e.g., Security and Sovereignty) were mentioned less frequently than issues regarding low
Figure 1.15 Topic proportions by partnership titles
politics (e.g., Cultural and Civil Exchange). However, it is unclear whether partnerships with different titles affect issue distributions. As we cannot tell the differences among the three forms of partnerships based on topic proportions, we examine the confidence intervals in the following section to estimate the effects of titles on issue distributions.