Although the concepts of information and probability are prevalent in statistics, the better part of statistics can also be understood in terms of signal and noise.
Explicitly and implicitly, Tukey (1915-2000) has written a lot about this idea. I would also like to recommend Bookstein's more recent book "Measuring and Reasoning" (Cambridge Univ. Press, 2014).
Here is a recent application to a common illness:
I have decided to protect the Daily English and Weekly English pages with a password after all. If you want to read on, please contact me.