As I have only recently began to explore Bayesian philosophy, I am perhaps unqualified to be giving lessons. However, I have found that what little I understand has been quite beneficial, and I think I am with others' help capable of highlighting common errors in probabilistic and evidential thinking. As I find myself directing others to resources quite often, it would be handy to have a reference, both for myself and for others. So I intend this primer to be just that: a small, workable start useful for ordering future in-depth reading. The criterion of my success or failure in this endeavor is this: after working through these posts, can you or can you not begin to apply Bayesian tools or know where to look in order to learn them?
To some extent, Bayesian philosophy can be thought of as both extending deductive logic and as formalizing everyday reasoning. In dealing with an uncertain world and comparing uncertain hypotheses via evidence, there is a need for the sort of rigor which probability theory provides. Regardless of whether one accepts a `Dutch Book defense' of Bayesian epistemology, there are fantastic pragmatic reasons for understanding it. `To some extent' is an important caveat; thinking of probability solely as an extension of deductive logic can lead to errors. It also helps one be humble: it makes clear how difficult it can be to convict a coherent and consistent person of being `unreasonable'.
The difficulty with Bayesianism is that it takes serious work for the mathematically or philosophically uninitiated. For those lacking the math, this is obvious. But the vindication of this work is that as a skeptic and a person interested in science, you should put in the effort. If you lack the mathematical prerequisites for Bayesianism, you lack the mathematical prerequisites for everyday science and statistics. In so doing you will fail to understand the shortcomings of studies, among so many other things. I do not think I should have to convince an interested reader of the import of set theory and mathematical logic, or of axiomatic methodology. For those lacking the philosophy but in some possession of the math, you might come to feel betrayed by your probability textbook, as I have come to feel.
I thought of including a post introducing abstract mathematics, but this would be too much. I have to assume this on the reader, asking her to consult other resources. If you understand naive sets, de Morgan's laws, basic quantifiers, mathematical induction, and Cantor's diagonalization theorem, you are probably ready to tackle the first post. Do not feel the need to memorize a Junior-level course before entering; unless you are well-studied in mathematics, be content with regularly consulting the material as you go.
I organize my approach as follows:
1. Basic axiomatic probability
2. Bayesianism outlined
3. Arguments for and against Bayesianism
4. Interpretation and subjectivity, part 1 and part 2
5. Other outstanding issues and some resources
6. Applications: deductive logic extended?
For (1), the goal is to introduce readers to Kolmogorov's axioms and key theorems, including e.g. the Law of Total Probability and Bayes' Theorem. Naturally, those familiar with the maths can skip this section. (2) extends the discussion in (1), listing probabilistic assumptions vital to Bayesianism, for example conditionalization and rigidity. (3) will focus on the standard defense of Bayesian epistemology: the Dutch Book argument. (4) explores the subjective/objective divides in probabilistic philosophy, esp. frequentism, propensity theories, and equivocation. (5) allows me to retain some comprehensiveness while keeping other sections clean; the title is self-explanatory. The title of (6) is similarly explanatory. Once you have an idea of formalism, seeing it in practice is vital to absorbing it, just as those who are attempting to learn mathematics should focus on the exercise section of the text.
Again, my approach will be simplistic and designed to prompt further reading. As many of the topics discussed are matters of professional dispute, I can do no other.
You know what I find interesting here? That my Ivy League education of 20+ years ago has left me unequipped to move forward in this series without further study on the topics you mention. So I've ordered (and just begun) Stoll's "Set Theory and Logic" and I've also ordered but haven't yet received "Scientific Reasoning: The Bayesian Approach" by Howson and Urbach. [Let me know if you are familiar with either of these, and if you think I'm wasting my time with either.] I've been playing with Bayes tutorials and such over the years, and have love what I've learned, but I think you're right that a more basic, fundamental background is necessary as well.
ReplyDeleteWhat I truly find so surprising is how I would be so unfamiliar with what appears to be fundamental to mathematical understanding, despite having completed Math through the pre-med requirement of Derivative and Integral Calculus.
Education is no substitute for learning. Hm... Maybe I could scribble that across the transcripts I send to graduate admissions departments :D
ReplyDeleteI am unfamiliar with Stoll's book, but I have been consulting the Howson/Urbach text steadily (even in this primer, but not the latest edition). It is recommended; further, I think the content should be accessible to the mathematically uninitiated with relatively little effort. At least, it is about as mathematically simple an introduction as one might hope for.
A classic text on set theory is Kamke's Theory of Sets. It is older, so the notation is somewhat wonky, but it has `classic' status. It also assumes virtually nothing of the reader, and is very short. And of that short text, most of it is unnecessary to absorb. (I would have to check it again to remember how to do/prove arithmetic of infinite cardinals. That being a thing you should not need, thankfully.)
And of course, I recommend Richard Jeffrey's Subjective Probability: The Real Thing. It and other goodies of varying accessibility are available for download here.
For a free introduction to probability theory itself by someone with both the mathematical and relevant philosophical credentials (one must be careful about the latter in finding a good probability text), Choice, Chance, and Inference works. For an introduction to combinatorics, Basic Combinatorics is available at the same link. But if you follow the examples closely in the other works, you should be happy to consult these only for reference purposes and as sources for exercises. Learning to do `combinatorial proofs' is always helpful, as they are (often) succinct and nearly always useful.
A proviso: the materials I recommend come from the `subjective Bayesian' tradition rather than the `objective Bayesian' tradition. This may be a mercy, since objective Bayesians make more assumptions, and thus generate very complicated work. (Basically: add some higher algebra and information theory to your reading list). If you are interested in that, you may nevertheless wish to acquaint yourself with the subjective Bayesian material first.
But I am glad that you're intrigued! If you have any questions, I would be happy to answer them to the best of my abilities. I'm sure that I will learn through the process as well.