Is more information always a good thing?
Keynes seemed tentatively to be drawn to the conclusion that the (gross) weight of an argument and its probability are two different properties of it (TP, pp. 82-3). But this led to a serious worry. In making decisions, Keynes thought, "we ought to take into account the weight as well as the probability of different expectations." Although Keynes reiterated this thought later on in his Treatise, he did not propose anything more than a sketch of a positive account of his own of weight of argument or how it bore on decision making or inquiry.
To cut through the metaphors of balancing and weighing, what is the problem to be considered? Here is an illustrative urn model example.
An urn contains 100 black or white balls. If we invoke Insufficient Reason in the manner condoned by Keynes, each possible constitution of black balls and white balls carries equal prior probability. The prior probability of a selection of one ball turning up a black is 0.5, and that should be the betting rate in evaluating a bet on black.
Suppose a sample with replacement of 100 balls is taken at random and 50 per cent are observed to be black. Bayes's theorem gives back a posterior probability of 0.5 for obtaining a black on the next draw. Betting on obtaining a black on the next draw with equal odds for and against is once more the favorite recommendation.
Keynes thought that the decision maker evaluates the balance of arguments in the same way in both cases considered and, hence, this leads to a similar evaluation of the gambles in terms of expectation. Nonetheless, like many decision makers he would prefer to make the decision when the information about the outcome of sampling is available. Keynes saw this preference in a more general setting than the one just described.
For in deciding on a course of action, it seems plausible to suppose that we ought to take account of the weight as well as the probability of different expectations. But it is difficult to think of any clear example of this, and I do not feel sure that the theory of "evidential weight" has much practical significance.
Bernoulli's second maxim, that we must take into account all the information we have, amounts to an injunction that we should be guided by the probability of that argument, among those of which we know the premises, of which the evidential weight is the greatest.
But should not this be re-enforced by a further maxim, according to which we ought to make the weight of our arguments as great as possible by getting all the information we can? It is difficult to see, however, to what point the strengthening of an argument's weight by increasing the evidence ought to be pushed. We may argue that, when our knowledge is slight but capable of increase, the course of action, which will, relative to such knowledge, probably produce the greatest amount of good, will often consist in the acquisition of more knowledge. But there clearly comes a point when it is no longer worth while to spend trouble before acting, in acquisition of further information, and there is no evident principle by which to determine how far we ought to carry our maxim of strengthening the weight of our argument. A little reflection will probably convince the reader that this is a very confusing problem.
(Keynes, TP, pp. 83-4)
Sometimes choice is peremptory. Sometimes inquirers have the option to delay making a final decision and collecting more information on which to base a decision where exercising this option is costly. And sometimes the option of postponing choice pending the acquisition of additional information is without cost or nearly so. In the latter case, the question then arises: Would delaying choice and obtaining more information always serve the aims of the decision maker better than making a "terminal" choice without further ado? If not, what considerations determine when we should stop inquiring? This is the question about weight of argument that Keynes found confusing.
Suppose that agent X confronting a decision problem faces a choice at some initial time t between (a) choosing among a set of terminal options and (b) postponing that choice to some later time t' and acquiring additional information in the interim. Keynes's vision seems to have been that the new information would be the result of observation or experimentation of some sort. In that case, the option of postponing choice and acquiring new information via observation or experimentation is contemplated in a setting where X does not know what the new information to be acquired will be.