GOF
How Do You Know Whether You Have a Good Model?
To say a model fits our data is to argue that the combination of such network structures specified in the model is a good representation of how this particular network could have been formed. GOF therefore allows us to know whether the specified model for our observed data represents particular network structures or graph features well. The GOF procedure gives us an indication of whether our model is able to represent important graph features.
To illustrate the utility of GOF in ERGM, let us examine the results of two GOFs (models A and C) for the communication network for The Corporation. These GOFs are presented in Table 13.3. What we see is that model A does not capture many of the structural network features because it does not include parameters for network selforganization (e.g., reciprocity, transitivity). However, when we included Markov parameters (model B), we were unable to obtain convergence for the model. This lack of convergence of model B, as shown in Table 13.2, demonstrates that the model cannot find a stable solution to the data, and as such, it is pointless to conduct a GOF. Finally, model C, which includes actor relation and higherorder structural effects, successfully converges, and we see that GOF is considerably improved. Taken together, we argue that model C is the best of the three models for this communication network.
Specifically looking at Table 13.3, there are four columns that refer to the GOF. These are the count (or statistic), which is simply the relevant statistic for our data (often, simply the count of the relevant
180 Exponential Random Graph Models for Social Networks
Table 13.3. Selected goodnessoffit (GOF) details for communication network (n = 38) for models A and C
parameter [PNet name] 
GOF 

count 
mean 
standard deviation (SD) 
GOF ?ratio 

Model A 

Arc 
146 
146.14 
10.54 
0.01 
Reciprocity 
44 
12.60 
3.48 
9.01 
2path 
559 
541.68 
83.38 
0.21 
2instar 
313 
280.92 
43.49 
0.74 
2outstar 
283 
276.30 
42.44 
0.16 
Transitive triad [030T] 
212 
77.11 
19.11 
7.06 
Popularity spread [AinS] 
167.51 
162.61 
18.30 
0.27 
Activity spread [AoutS] 
166.39 
162.22 
18.20 
0.23 
Path closure [ATT] 
153.97 
67.72 
15.23 
5.67 
Cyclic closure [ATC] 
120.97 
63.85 
17.19 
3.32 
Multiple 2paths [A2PT] 
452.47 
485.67 
67.43 
0.49 
Sender (seniority) 
73 
73.30 
7.68 
0.04 
Sender (projects) 
993 
994.63 
96.12 
0.02 
Receiver (seniority) 
84 
84.21 
7.94 
0.03 
Receiver (projects) 
1005 
1007.57 
96.66 
0.03 
Homophily (seniority) [interaction] 
60 
60.30 
6.90 
0.04 
Heterophily (projects) [difference] 
584 
585.91 
62.66 
0.03 
Homophily (office) [matching] 
51 
50.99 
6.16 
0.00 
Advice entrainment [covariate arc] 
24 
24.17 
3.57 
0.05 
SD indegree distribution 
2.39 
1.95 
0.23 
1.89 
Skew indegree distribution 
0.37 
0.40 
0.34 
0.08 
SD outdegree distribution 
2.02 
1.89 
0.21 
0.61 
Skew outdegree distribution 
0.17 
0.29 
0.34 
1.38 
Model C 

Arc 
146 
146.60 
14.80 
0.04 
Reciprocity 
44 
44.07 
6.79 
0.01 
2path 
559 
566.36 
116.35 
0.06 
2instar 
313 
310.17 
66.83 
0.04 
2outstar 
283 
301.34 
62.80 
0.29 
Transitive triad [030T] 
212 
212.42 
54.18 
0.01 
Popularity spread [AinS[ 
167.51 
168.56 
25.91 
0.04 
Activity spread [AoutS] 
166.39 
167.26 
25.61 
0.03 
Path closure [ATT] 
153.97 
154.85 
31.16 
0.03 
Cyclic closure [ATC] 
120.97 
121.92 
29.30 
0.03 
Multiple 2paths (A2PT) 
452.47 
457.20 
79.75 
0.06 
Sender (seniority) 
73 
73.83 
11.06 
0.08 
Sender (projects) 
993 
994.47 
142.35 
0.01 
Receiver (seniority) 
84 
84.59 
12.18 
0.05 
Receiver (projects) 
1005 
1008.53 
141.21 
0.03 
Homophily (seniority) [interaction] 
60 
61.11 
11.33 
0.10 
Heterophily (projects) [difference] 
584 
584.80 
88.84 
0.01 
Homophily (office) [matching] 
51 
51.20 
7.92 
0.03 
Advice entrainment [covariate arc] 
24 
23.86 
4.06 
0.03 
SD indegree distribution 
2.39 
2.28 
0.31 
0.35 
Skew indegree distribution 
0.37 
0.40 
0.35 
0.08 
SD outdegree distribution 
2.02 
2.18 
0.28 
0.56 
Skew outdegree distribution 
0.17 
0.35 
0.34 
1.56 
configuration). The mean is the estimated statistic produced by our model, and the next column is the associated SDs. For a model to represent a particular graph feature well, we want the observed data to be plausible (i.e., not too far away from the mean). For example, in the first row of Table 13.3 for the arc effect, we observed 146 ties, whereas the sample mean from the model was 146.14. The final column is the tratio of the GOF, which is a measure of fit of the model.
Recall (see also Section 12.9) that GOF is carried out on the premise that the estimation has converged; in other words, the difference between simulated statistics and observed statistics for “fitted effects,” as measured by the convergence statistic, is smaller than 0.1 in absolute value (e.g., in phase 3 of the RobbinsMonro procedure in PNet). Because the GOF also compares simulated average statistics under the fitted model with the observed statistics, the GOF tratios for the fitted statistics should be smaller than 0.1 (in absolute value). Of course, the estimation is a stochastic procedure, so we do not expect to obtain the exact same values for GOF and estimation for these tratios for fitted statistics. Thus, if the GOF statistics are a little over 0.1, there should be no need for concern. However, sometimes the GOF statistics can be substantially over 0.1. The first step is to run a much longer GOF simulation to determine whether the problem is resolved. If not, you may not have a fully converged model, and there may be a need to reestimate the model.
In detail, when the GOF statistic for a fitted effect is substantially over 0.1, the precision of either the samples in phase 3 of the Method of Moments estimation or the GOF is insufficient. Of these two, the sample with most iterations and lowest autocorrelation should be most trusted. If the number of iterations is the same, a discrepancy means that either the “number of iterations in each step” (given in the PNet output) for phase 3 is too small or the ratio of the number of iterations in the GOF to the “number of samples to be picked up” is too low (this is comparing the thinning; see Section 12.2.2). In case of the former, you will probably find that some of the autocorrelations are too big (in which case the basic premise of converged estimates is likely to be violated and the model needs refitting, with a larger multiplication factor; see Section 12.4.2). In the second case, we need to increase the number of iterations relative to the number of samples picked up for the GOF sample.
The “gof() ” routine in statnet works both as an independent convergence check (like phase 3) and GOF (but note that the SEs are calculated in the basic fitting routine “ergm() ”). The properties of the GOF sample may likewise have to be improved by increasing the burnin, thinning, or sample length. If there is a need to do this, it is assessed using a combination of numerical criteria and graphic inspections of the sample. In general, it is a good idea to graph the sequences of statistics produced by ergm and PNet (see Section 12.2 on simulation).
For statistics not specifically included in our model, however, we are interested to see if their values are extreme. A graph statistic with a tratio greater than 2.0 can be regarded as extreme, in which case we infer that it is not plausible that a statistic of that value could have arisen from the model (the extreme value 2.0 is chosen with reference to the approximate critical value of a standard normal variate; for more on this, see Section 12.9). In other words, the model does not represent that graph feature well. In model A, we have a tratio of 0.74 for the number of 2instars, which is clearly less than 2.0. Thus, the model reasonably captures the number of 2instars in our observed network, even without including a parameter for it in the model. However, the number of reciprocated ties and the number of transitive triads are not well represented by the model. Because these are rather fundamental features of a directed social network, we conclude that model A is not a very good model for the data.
Table 13.3 lists only a small number of possible graph features that can be examined in a GOF as illustrations. For directed networks, Robins, Pattison, and Wang (2009) suggested examining GOF on the following graph features:
 • Graph counts and associated statistics:
  Dyadic statistics: numbers of arcs, reciprocated arcs
  Markov graph configuration counts: numbers of 2instars, 2 outstars, 3instars, 3outstars, mixed 2stars (2paths), cyclic triads, and transitive triads
  Statistics for social circuit parameters: AinS, AoutS, ATT, ATU, ATD, ATC, A2PT, A2PD, A2PU
 • Degree distributions:
  SDs and skewness of both in and outdegree distributions
  Correlation between in and outdegrees across nodes
 • Closure:
  For single triangle closure, global clustering coefficients, that is, 3 x (no. of triangles)/(no. of 2stars). For directed graphs, there are four ways to complete a single triangle: from a 2 path to complete a 3cycle (clusteringC); from a 2path to complete a transitive triad (clusteringT); from a 2instar to complete a transitive triad (clusteringU); and from a 2outstar to complete a transitive triad (clusteringD).
  Clustering coefficients based on social circuit parameters: the ratio of the alternating ktriangle statistic to the analogous alternating k2path statistic (i.e., this coefficient is a measure of the proportion of alternating k2paths that have a base present to complete an alternating ktriangle). This gives an additional four higherorder clustering coefficients, ACT, AC D, ACU, and ACC, where “AC” stands for “alternating k cluster.”
 • Geodesic distributions:
  Quartiles of the geodesic distribution (labeled G25, G50, G75, and G100, respectively  G50 as the median can be taken as a measure of average geodesic length for the graph). Because some geodesics may be infinite, rather than use means and SDs, and hence tratios, Robins, Pattison, and Wang (2009) proposed to examine the percentile of these statistics for the observed graph in the distribution of statistics from the simulated distribution.
 • Triad census:
  Counts of the sixteen triad types defined by Holland and Lein hardt(1970)
These features, along with some others, are included in GOF in PNet and most of them in statnet.
Let’s now look more specifically at the GOF for models A and C in Table 13.3. Model A does not reproduce many of the features of the data. For example, there are 44 reciprocated ties in the data, yet the model suggests a mean of 12.60. The GOF tratio for this reciprocity count is 9.01, much higher than 2. Model A substantially underestimates reciprocated ties for our observed network, and the same occurs for transitive triads. However, the count of Markov 2outstars is 283, and our model produces a mean of 276.30. The associated GOF tratio of 0.16, which is much less than 2, suggested that our model is able to capture the presence of 2outstars sufficiently even without specifically including a 2outstar parameter.
For model C, reciprocity is modeled explicitly as a parameter in the model, and we also see that model C represents the number of transitive triads well (not surprising, given that there is an alternating triangle parameter in the model).
The inclusion of effects for network selforganization (i.e., purely structural effects) alters some of the actorrelation estimates (Table 13.2). For example, the negative sender effect for seniority that was significant for model A is no longer significant in model C. However, the inclusion of the purely structural effects does not “wash away” the actor relation effects entirely. Although estimates differ slightly, there are still homophily effects for seniority and number of projects in model C as there were in model A. The attributes of the actors and network selforganization tendencies both have a role in explaining the presence of ties in this communication network.
GOF may be helpful in choosing which network configurations to include for model specification. If a feature is not well fitted by a model, a judicious choice of an additional parameter in a new model may help. However, we stress that GOF should not in any way be seen as a replacement to theoretical motivation.