Skippy Records » Physics and Mathematics

R, for example… Moving here

Dr. Skippy — Thu, 03 May 2012 13:30:09 +0000

R is a statistical computation language. As a programming language, I am not too fond of it, but for many quick analysis and visualization problems, R is just the tool. The ggplot package from Hadley Wickham produces very good looking plots quickly.

observations about R:

I thought I would master the basics in about 20% of the time it has actually taken. It seems R has a steep learning curve as they say. May this is only true when you are coming from other object/procedural languages (my experience is with C, Java, Python, C++…)?
Nearly every frustration I have experience with R comes from not understanding the required form the data and not knowing the tools for transform data. When you get the data in the right structure, many powerful packages will give sophisticated answers in a line or two.
When I start writing functions and loop in R, it is type to move to Python/NumPy/SciPy/Matplotlib.
(There is an art to searching for things related to “R” on the web–almost as bad as searching for the game of “go”.)

I started posting R examples on Tumblr. The idea behind “R, for example…” is to provide input data, code and output to quickly walk through a short analysis illustrating new (to me) or useful R. But the Tumblr platform really isn’t very friendly to math, data and plots. Going forward, R examples will appear here on Skippy Records.

BBC’s “Joy of Stats”

Dr. Skippy — Fri, 31 Dec 2010 21:42:55 +0000

Awhile back, I argued here that we should be teaching more statistics in high school and college, especially over Calculus. When I wrote before, I was thinking of skills and knowledge that includes:

Ability to choose what statistical tools to use. For example, it would be great to have students who could apply significance measures, evaluate probability and its relationship to risk and know when the average is meaningful (Normal Distribution) or nearly useless (Power Laws).
Practical skills with data collection and manipulation tools such as text editors and spreadsheets, statistics packages (maybe something like R?) and moving data around. Because bigger data sets are so common and rich, it is not enough to know how to calculate the average of a few numbers with a calculator.
Experience and confidence with quantitative analysis as a basis for consistent qualitative reasoning and decision making.

I was delighted to see that the BBC production “The Joy of Stats” with Hans Rosling is now available on YouTube. This piece does a great job of showing why and what to explore with statistics. And Hans is entertaining.

Tagged: statistics

Drawing lusona

Dr. Skippy — Sun, 05 Sep 2010 17:28:15 +0000

A Lusona is a Eulerian Path mandala that shows up in the traditions of African and Indian cultures. I made a short demonstration video this morning showing the drawing of one traditional path. Ron Eglash has a nice write-up and gives some additional references in his great book African Fractals.

Tagged: lusona, mandala, traditional art

Stewart, Taleb, Mandlebrot…

Dr. Skippy — Mon, 17 May 2010 17:09:22 +0000

N. N. Taleb and many other mathematicians (e.g. Mandelbrot) have been trying to explain how market variations are not adequately characterized by symmetric, thin-tailed Gaussian distributions, that a World seen through these models will have many (unexpected!) “black swans” or “perfect storms.”

John Stewart illustrates the same point by lining up quotes from the talking heads in the segment A Nightmare on Wall Street. (Sorry for the link out, embedding Comedy Central Flash video in WordPress stumped me.)

Tagged: black swan, complex systems, fooled by randomness, investing

Dollar cost averaging: making hay while the sun shines or doubling down?

Dr. Skippy — Fri, 23 Apr 2010 20:17:58 +0000

Over at Scientific Clearing House, Carson posted a pretty proof of the how dollar cost averaging is an optimal investment strategy. Unfortunately, I think the conditions for this optimization aren’t practical because they are far too optimistic about one’s ability to choose to invest in appreciating assets.

In a long winded comment, I take delight in making the following perverse equivalence between an honest work-ethic idiom and foolish gambling practice,

Under these optimistic assumptions, DCA [dollar cost averaging] is equivalent to the idiom: make hay while the sun shines. Downward price fluctuations are viewed as opportunities to use the same resources to make even more of a good investment. Under less certainty, this is what gamblers call doubling down–when you are loosing, bet more so when you win, you make up your previous losses.

I have advocated DCA as a reasonable investment strategy to friends of mine, but I am increasingly skeptical about the assumptions. It is starting to seem more like doubling down and less like making hay.

Tagged: dollar cost averaging, investment, mathematics

Misunderstanding the long tail

Dr. Skippy — Tue, 01 Dec 2009 17:23:28 +0000

The Economist blows it! (I don’t get to say that very often.) On page 79 of the Nov 28 – Dec 4 2009 issue, there is an article (a world of hits) that misrepresents and misunderstands the ideas behind Anderson’s Long tail and proceeds for 2 more pages to build a story on the misunderstanding. This is not just useless reporting, writing and thinking, it is damaging and a sad illustration of mathematical and analytic carelessness.

I am sure Anderson and others will jump on the many errors here. I just posted this rant in the comments section of the online version of the article…feel much better now

The mistake seems to be not understanding that long tail statistics come from underlying dynamics. In particular, books and movies are small hits before they are big hits. The difference between the long tail and the big hits (possibly call it “tall head”) is how fast a book or movie becomes a hit and how big of a hit it becomes. It is difficult to propose a dynamic in which the bottom and middle of the distribution grow where the big hits don’t get bigger as well. Anderson never proposes such a fundamental shift in underlying dynamics.

There seems to be a complete disconnect between the statistics and the market dynamics producing the statistics in this article.

To say that the long tail ideas predict the demise of the hit is sloppy (or ignorant?). In fact, as technologies that enable access to the long tail become more efficient and capable, there may be an argument that the outcome will be that big hits get bigger. The tall head may grow taller since all tall head items start out with no following at all and grow from long tail-dom–the place where broad cheap access is changing our access most.

Hits will always be big (seems like I wouldn’t need to write a sentence that dumb on purpose!) and big hits will always make big money. The point of the long tail is that there are now opportunities to make significant money on less popular items (e.g. books and movies in the long tail.)

Tagged: long tail, mathematics, the economist

Recursion – Part 2

Dr. Skippy — Fri, 23 Oct 2009 17:49:22 +0000

I love this sign!

A sign mostly about itself.

About a year ago, I posted some self-referring python code toys. I always find these things amusing. I feel recursion, like Steven Wright’s humor, in my brain with a mental “clunk” and a sense of delight and awe–maybe synethetically? I also believe that recursion is deeply related to consciousness and human-style intelligence, so the experience seems important. Enjoy.

Hat tip: Hannah Kaiser

Tagged: recursion

Mathematics education: Calculus or statistics?

Dr. Skippy — Mon, 13 Jul 2009 17:15:21 +0000

The attracting force of all mathematics education should be shifted from calculus to probability and statistics. Advanced mathematics students in high school should be taking Pre-stats instead of PreCalc or APStats instead of APCalc. Arthur Benjamin nails it in this TED talk. This point is vital for competing successfully in the future. It is important for our business decision makers, community planners, disaster planners, economic and political strategists, and all consumers of democracy, investment products, real estate and career opportunities. It is key to understanding uncertainty and risk. The amount of data we have on our complex world is only an advantage is we know what to do with it, how it tells stories of causation and optimal decisions. I love teaching calculus and admire its deep relation to fundamental physical laws, but I want to live in a world shaped by the voices of many skilled statisticians.

Tagged: calculus, eduction, mathematics, probability, statistics

Color Illusion

Dr. Skippy — Fri, 26 Jun 2009 17:15:15 +0000

I have a friend who often admonishes people to hold their beliefs lightly. Our perceptions can be distorted by habit, framing, context, etc. I like to think that with concentration and awareness, I can see what is. And so, this color illusion drives me nuts! The problem? The “green” and “blue” spirals are actually the same color…

Colored spirals - not quite what it appears

Hat Tip: Bad Astronomy, TED Blog, James Cascio, (…nerds run in packs?)

Tagged: illusion

Synchronizing Fireflies Video from Tinkerlog

Dr. Skippy — Thu, 25 Jun 2009 21:38:41 +0000

I mentioned the collective effects foreshadowed by the Programmable LED paper and posted a related video in Coupling and Synchronization Video. Here is the result of some great work by tinkerlog to make it happen. He had to exchange the resistive sensor for a more senisitive photodiode and had circuit boards made for an Arduino version. Nice work.

64 Fireflies Part II from tinkerlog on Vimeo.

Tagged: synchronization physics