Last year I made a cocktail-napkin calculation of the payoff challenges of home energy monitoring. My question then was whether the granularity of measurement required to make energy decisions in the home was mismatched to the amount energy used, and therefore, the amount that could be saved. It seems others are arriving at this question at even a slightly higher level of granularity–the (smart) meter.
From earth2tech comes an article on California utilities struggling to get data to customers. The last paragraph brings up the challenge of payoff periods being out of line with available savings,
There are signs that the smart meter backlash is spreading beyond California —Duke Energy is being ordered by Indiana regulators to justify the costs of an 800,000 smart meter deployment in that state, and Dominion Virginia Power is delaying a $600 million smart meter rollout to do more testing, after state regulators questioned whether the meters will cost customers more in increased rates than they will help them save in reduced energy usage.
It seems natural to extend the basic idea from the Predictioneer’s Game introduced in the last post in two ways.
First, what happens if subsets of the group meet and some of those parties sway others or reach compromises in order to form coalitions?
Second, taking this idea further, can we investigate a variety of decision processes to find an optimal process for a desired outcome and design coalitions in order to reach a desired outcome? In terms of the example situation, this is the problem of finding coalitions that will drive a desired result from those we predicted last time.
Again, here is the prediction from the last post:
party : Pos Inf(Norm) Sal ----------------------------------------------- d2 : 15 0.03774 80 ctcust : 35 0.09434 5 ctsal : 35 0.03774 5 er : 50 0.01887 20 eng : 60 0.07547 99 me : 75 0.15094 99 d1 : 75 0.15094 99 adv : 100 0.15094 20 legal : 100 0.13208 95 inv : 100 0.15094 5 ----------------------------------------------- Position (weighted avg): 76.4 Position (balance of power): 71.7
TABLE 1
To answer the first question, we need to build a simple model that replaces a coalition with a single new entity. In the model, the choice was to add the Influence of the parties of a coalition. The new position was the weighted average of the positions as described before. For the Salience, we need to reflect the idea that the most interested party in a coalition will drive the others and use their influence to support the new positions but with some skewing toward their Salience. To model this simple, replace the new entity’s salience with 20% of the average Salience + 80% of the max Salience. Many other choices can be made and explored using the tools here.
To answer the second question, we need a way to generate all of the possible coalitions. This amounts to generating a set partitioning of all the parties. TABLE 2 shows a simple example with only 4 parties.
Parties : ['a', 'b', 'c', 'd']
<<< 15 Partitions >>> ----------------------------- [['a', 'b', 'c', 'd']] [['a', 'b', 'c'], ['d']] [['a', 'b', 'd'], ['c']] [['a', 'b'], ['c', 'd']] [['a', 'b'], ['c'], ['d']] [['a', 'c', 'd'], ['b']] [['a', 'c'], ['b', 'd']] [['a', 'c'], ['b'], ['d']] [['a', 'd'], ['b', 'c']] [['a'], ['b', 'c', 'd']] [['a'], ['b', 'c'], ['d']] [['a', 'd'], ['b'], ['c']] [['a'], ['b', 'd'], ['c']] [['a'], ['b'], ['c', 'd']] [['a'], ['b'], ['c'], ['d']]
TABLE 2
For 10 parties, there are 115,974 games to play out. (See Donald Knuth’s The Art of Computer Programming Vol 4 F3). You can download output of all 115,974 games. The interesting games are the extremes: the game that results in the maximum position and the one that results in the minimum position.
Game A maxPos : 83.22147 party : Pos Inf(Norm) Sal ---------------------------------------------------------------- eng : 60 0.07547 99 _d1+d2_ : 64 0.18868 95 _er+me_ : 74 0.16981 83 _adv+ctcust+ctsal+inv+legal_ : 97 0.56604 77 ----------------------------------------------------------------- Position (weighted avg): 83.2 Position (balance of power): 77.1
(Game # : 2089)
Game B minPos : 66.27527 party : Pos Inf(Norm) Sal ------------------------------------------------------------ _ctcust+ctsal+d2+inv_ : 32 0.32075 65 _d1+eng+me_ : 72 0.37736 99 _er+legal_ : 98 0.15094 80 adv : 100 0.15094 20 ------------------------------------------------------------ Position (weighted avg): 66.3 Position (balance of power): 51.2
(Game # : 47615)
TABLE 3
In Game A, a coalition of d1+d2 and er+me results in the most extreme high-number position. The final position realized was 100 and these two coalitions were instrumental in the outcome. To design for to most extreme high position, these are the coalitions to nurture.
In this case, however, d2, the biggest stakeholder in the low-number position, would have achieved more of their goals had they designed the decision process around the coalitions shown in Game B. By building a relatively strong coalition around ctcust+ctsal+d2+inv and a diffusive coalition around er+legal, a much lower position is achieved.
There is a second hint here too. If the decision process could be more focused on Balance of Power rather than a salience/influence weighted average, a position of around 50 might be reached. This might be done, for example, by getting all the parties to agree to a final vote with 1 vote per person, winning position takes all votes, then voting in rounds sequentially through the coalitions of Game B top to bottom.
Download the Python model modules: MultiPlayerGame.py MultiPlayerNegotiationsModel.py Partitioner.py
Bruce Bueno De Mesquita’s Predictioneer’s Game describes work by De Mesquita’s team of applied game theorists and his students to business and political negotiations and decision making. Predictioneer’s Game covers some of the very basic ides, but De Mesquita leaves nearly all of the details of his models a mystery.
I recently had the opportunity to observe a business negotiation between about a dozen parties that seemed ripe for PG analysis. Among the basic ideas explained in the book are the basic inputs used by De Mesquita’s models and how to make estimates of “game” outcomes. In a future post the model here will be expanded to model more of the details of interactions between the parties and sub-groups. As the book contains few hints to how De Mesquita does this, those models will be striking off on our own.
But first, back to the negotiations and simple estimates of the outcome. All of the parties below are actual people involved in the negotiation. Negotiations covered about 2 months and are completed with respect to these positions.
The object is to calculate two estimates of the outcome of the negotiations following Predictioneer’s Game. Here are the steps:
(1) Identify the stake-holding parties involved in the negotiations. Anyone who takes a definite position and has some stake in the outcome should be included. This model includes only the most obvious players.
Party [Position, Influence, Salience]
-----------------------------------------
data = {
'd2': [ 15, 20, 80],
'ctcust': [ 35, 50, 5],
'ctsal': [ 35, 20, 5],
'er': [ 50, 10, 20],
'eng': [ 60, 40, 99],
'd1': [ 75, 80, 99],
'me': [ 75, 80, 99],
'adv': [100, 80, 20],
'inv': [100, 80, 5],
'legal': [100, 70, 95]
}
TABLE 1
(3) Describe the relative influence of the parties. Below, the influence will be normalized to add up to 100%, so just choose a scale that is convenient to get the relative numbers right. In the example, I estimated the influence of each party at the same time as I estimated positions.
(4) Now it is time to the estimate cost/benefit to each party. Salience is a score starting with ambivalence=0 moving up to everything there is to gain or loose is staked on the outcome scoring 100.
Predictioneer’s Game describes students and analysts researching positions and salience for political problems through interviews, CIA analyst records, newspapers, eye-witness reports etc. These steps could take a great deal of work. But also some discipline to hear and interpret what people are trying to accomplish.
Time to estimate outcomes. The first estimate is the weighted average position given by
Plugging in the numbers above gives a position just above parties d1 and me at 76.4. See Table 2 below.
The second estimate of outcome is the point along the position line where the cumulative total influence is 50%. That is, there is as much influence pulling the final position to the left as to the right. In this case the the balance-of-power point is at position 71.7.
The results are summarized on the table below. Influence has been normalized to add up to 100%.
Party : Pos Inf(Norm) Sal ------------------------------------------------------------ d2 : 15 0.03774 80 ctcust : 35 0.09434 5 ctsal : 35 0.03774 5 er : 50 0.01887 20 eng : 60 0.07547 99 me : 75 0.15094 99 d1 : 75 0.15094 99 adv : 100 0.15094 20 legal : 100 0.13208 95 inv : 100 0.15094 5 ------------------------------------------------------------ Position (weighted avg): 76.37 Position (balance of power): 71.72 TABLE 2
Both of these predictions give a position that was discussed as a plausible compromise between the more extreme positions.
This is not how the negotiations worked out. The final outcome was at position 100. In fact, the middle ground evaporated about 7 days after I first made this calculation. This situation moved very quickly to position 100 with everyone from position 50 and above accepting the 100 was the only possible outcome and the majority of the influence below position 50 also insisting that, although undesirable, 100 was the only possible outcome.
I am fairly confident that any proposal from the influential parties in the positions below 50 that landed at or slightly above 50 would have lead to further negotiations and resolution. In this case, some of the parties decided not to fight for their positions directly. This means I overestimated the salience of their positions?
Last fall I happened across Body by science in the bookstore. I tend to see the weight loss and fitness industries as genetically above average humans telling genetically average humans that they aren’t doing it right. But these enthusiastic trainers don’t usually have much in the way of systematic research to back up their claims.
The first things I read when leafing through BBS was the research on the genes one needs to build a body like Arnold Schwarzenegger used to have. It turns out that we can currently identify 4. This means that there of some variations in genetic disposition to build large muscles with people having all four in the “on” position having a clear, measurable advantage when it comes to building muscle. This knowledge undermines much of the current body building industry–if you don’t have the genes, there is not workout regime, personal trainer, magical powdered drink or incantation that will make you grow big muscles. (Steroids appear to be the exception.) Given the constraints, however, most people can become stronger than they are now. Body by Science tries to answer how that is most effectively done.
After reading the sections on metabolism and muscle growth, it seemed clear that BBS was serious about recommendations based on evidence so I committed to try the workouts. I hate lifting weights. I learned the high-school “state of the art” 20 years ago, but never really caught the bug. The routine then was 3 sets with tapering reps 25, 20, 10 or something like that. Recommendations were to work out 3-5 times per week for 30-45 min and do lots of different exercises. You need to work out hard and stick with it.
Body by Science starts by recognizing that there are only two phases to muscle growth: first work a muscle very intensely, then rest long enough for the muscle to recover and grow.
“Work” in this case means one set of 9-11 repetitions done very slowly (10+s per rep) with muscle failure coming after about 90 seconds. Work large muscle groups and use machines (since working to complete failure.) The recommendation is to work only 5 muscle groups during the first 15 weeks.
This means the schedule is to workout for about 23 minutes per week.
Rest means 7-10 days between workouts–that’s 1 or fewer work outs per week. One obstacle to this program is that you don’t feel like you paid your dues. The big advantage is that the investment of time and effort is sustainable. If progress slows, add a day of rest.
Is the science sound? One anecdote is not evidence that it works for everyone, but it seems to be working for me. I just completed week 10. Results charted below.
The unanticipated challenge of this system is finding the weight that results in muscle failure at 90-100 seconds. This is because is because your strength changes quite a lot from week to week.
Percent change in strength
From the percent change chart, it is clear that there are 2 (maybe 3, see below) different growth rates for muscle groups. The Chest and leg muscles gain strength most rapidly, with arms and shoulders growing more slowly.
To give some reference here are the actual weights moved in the workouts. As you can see by the very steep rise at the beginning, I started with too light of weights and had to increase a lot the first week to reach failure in less than 12 reps.
Weights (machine) week by week.
One mistake I persisted in was with Pull Downs. I increased the weight too quickly in week three and didn’t realize how important thorough muscle fatigue is to strength building. The weight was too heavy and I was only able to do 4-5 reps. I thought I would improve enough during the week (based on my other exercises) to use the same weight the next week and reach failure in 9 or 10 reps. But strength didn’t improve nearly enough. In my last workout, I decided to reduce the weight down to 175 lbs in order to completely fatigue the muscle. The next few weeks should tell if this diagnosis is correct. If so, the red line should start taking off like the other smaller muscle groups.
On hearing about this experiment, a friend pointed me to an older book with essentially the same message. Check out The power of ten if you want a second opinion.
Born to Run was one of my favorite reads last year. Today Nature published some of the science hinted at in McDougall’s entertaining and inspiring book. Nature has gotten onto the part of the story about to toe vs. heel footfalls and running barefooted. This makes a lot of sense to me from the perspective of adaptation and mechanics. $200 running shoes aren’t the answer to the majority of running injuries. This shocking surprise comes after having listened to trained salesmen and running shoe ads for the last 30 years. Silly me.
Nice video showing forces on the foot and talking to researcher (apologies for the jump, doesn’t seem to be embeddable).
EO Wilson: “Games are the future in education. I envision visits to different ecosystems that the student could actually enter…with an instructor. They could be a rain forest, a tundra, or a Jurassic forest.” (hat tip GG). Games are the way humans did it for thousands of years (up to a few hundred ago); they are how we will do it in the future. To acquire the complex science, math, social, economic, etc. skills that a crowded, connected fast moving world requires, we will have to get to a superior mix of fun, curiosity and content with simultaneous mental, physical and emotional engagement integrated and pushed to the limit. The “golden age” of pounding the 3Rs into kids orderly and quietly sitting at rows of desks will fade away as the dark age of the beginning of ubiquitous education…
A conspiracy theory’s (CT) complexity is typically not minimal to explain the situation and therefore violates the simplicity principle behind Occam’s razor. CTs often require an unlikely coincidence of people, events and motivations. There are often systemic explanations that fit the events and facts more simply. This does not prove any particular CT false, but it says we may want to approach all CTs with skepticism toward the elaborate solution if just to ward off helplessness.
Here are some corollary ideas:
By adopting systemic explanations that capture a repeated pattern, we are farther ahead in dealing with the next challenge of design, perception or interaction than if we have the correct but exceptional explanation in any particular case. Call this the “secular” assumption or “uniformity” assumption. Patterns that repeat in some form give us options for response. Exceptional explanations take away our ability to react sanely.
It is easy to mistake familiar as equivalent to simple. This is true in some sense as there are mental and emotional costs in incorporating and using new tools, ideas and perspectives. I don’t mean emotionally simply or personally simple (convenient). Simple means this explanation accounts for more facts and events with fewer assumptions, especially assumptions in-particular (i.e. super-natural forces). This situation is analogous to algorithmic complexity or compression and coding.
Some failures to recognize a simpler-than-CT solution may be merely naive or narrow perception or laziness (failure to exploration a plurality of explanatory systems. See Kevin Kelly on Simplicity and Occam’s razor.)
CTs make us helpless by removing (i) learning from the past (ii) learning from the experiences of others (iii) removing confidence in our ability to apprehend a new situation without making the primary assumption that secret or esoteric knowledge will be needed to interpret and understand.
The code I introduced in this post a few days ago has been evolving. Below is the latest version. Additions include searching multiple paths for the properties file, explicit naming of properties and minimal exception handling. If you find this useful, please drop me a line in the comments.
# $Date: 2009-12-07 19:11:09 -0700 (Mon, 07 Dec 2009) $
# $Author: scott.hendrickson<at>buildingsi<dot>com $
#
##############################################
# Default resource files are named after class:
# class_name.properties
#
# If this class is the base class, then properties
# file is required.
import os, sys
import logging
class res:
def __init__(self, logger=None,
resourceName=None):
self.resourceValidFlag = False
self.logger = logger
self.resourceMap = {}
# current path, path of script
searchPaths = [os.getenv('PWD'),
sys.path[0]]
if resourceName is None:
filename = ''.join([self.__class__.__name__,
".properties"])
else:
filename = ''.join([resourceName,
".properties"])
propFile = None
for path in searchPaths:
tmp = os.path.normpath(''.join([path,
'/',
filename]))
if os.path.exists(tmp):
propFile = tmp
if propFile is not None:
if self.logger is not None:
self.logger.info("opening properties file %s" %
filename)
try:
f = open(propFile, 'rb')
for prop in f:
if prop[0] <> "#" and prop[0] <> '\n':
list = prop.split("=")
value = ''
for i in range(1,len(list)):
value += list[i] + "="
key = list[0]
value = value[:-1]
self.resourceMap[key] = value.strip("\n\r '\"").strip('\n\r')
if self.logger is not None:
self.logger.debug("property %s set to '%s'" %
(key,
self.resourceMap[key]))
self.resourceValidFlag = True
except IOError:
if self.logger is not None:
self.logger.info("unable to read resource file %s" %
propFile)
else:
sys.stderr.write("unable to read resource file %s\n" %
propFile)
else:
if self.logger is not None:
self.logger.info("unable to find resource file %s in %s" %
(filename,
str(searchPaths)))
else:
sys.stderr.write("unable to find resource file %s in %s" %
(filename,
str(searchPaths)))
def isValid(self):
return self.resourceValidFlag
def getString(self, key):
if key in self.resourceMap:
return self.resourceMap[key]
else:
if self.logger is not None:
self.logger.error("resource %s missing" %
key )
else:
sys.stderr.write("resource %s missing\n" %
key )
return None
Nearly 15 years ago, Kelly wrote Out of Control. This book still reads pretty well today; for its time, it seems uncanny in its synthesis and foresight. I am really excited about Kelly’s new book, What technology wants due out late next year. Below is a short production based on an and interview with Kelly about the Technium. For Kelly’s blog, see the blog roll on the homepage of this blog. Enjoy.
The Economist blows it! (I don’t get to say that very often.) On page 79 of the Nov 28 – Dec 4 2009 issue, there is an article (a world of hits) that misrepresents and misunderstands the ideas behind Anderson’s Long tail and proceeds for 2 more pages to build a story on the misunderstanding. This is not just useless reporting, writing and thinking, it is damaging and a sad illustration of mathematical and analytic carelessness.
I am sure Anderson and others will jump on the many errors here. I just posted this rant in the comments section of the online version of the article…feel much better now 
<rant>
The mistake seems to be not understanding that long tail statistics come from underlying dynamics. In particular, books and movies are small hits before they are big hits. The difference between the long tail and the big hits (possibly call it “tall head”) is how fast a book or movie becomes a hit and how big of a hit it becomes. It is difficult to propose a dynamic in which the bottom and middle of the distribution grow where the big hits don’t get bigger as well. Anderson never proposes such a fundamental shift in underlying dynamics.
There seems to be a complete disconnect between the statistics and the market dynamics producing the statistics in this article.
To say that the long tail ideas predict the demise of the hit is sloppy (or ignorant?). In fact, as technologies that enable access to the long tail become more efficient and capable, there may be an argument that the outcome will be that big hits get bigger. The tall head may grow taller since all tall head items start out with no following at all and grow from long tail-dom–the place where broad cheap access is changing our access most.
Hits will always be big (seems like I wouldn’t need to write a sentence that dumb on purpose!) and big hits will always make big money. The point of the long tail is that there are now opportunities to make significant money on less popular items (e.g. books and movies in the long tail.)
</rant>
