Python resources as Java-like properties files

2009 November 24
by DrSkippy27

BuildingSI’s application stack is Linux-Tomcat-MySQL-Java Servlet/Python/Wicket.  Wicket, Log4j, Tomcat etc. use of Java properties files to manage resources and Localized resources.  Because I am working in this framework all of the time, I decided to write a small base class for all of my Python objects that allows the Java-like properties file use for Python.

Of course, with pretty code formatting, managing string constants and varying configurations within your Python code is not as inconvenient as with a compiled language. The main advantage is uniformity.  During maintenance or message changing tasks, you merely edit all of the *.properties files at once.  A secondary advantage has been the ability to use identical code for staging and production by using platform/environment-dependent properties like,

"SOURCE.hostname="/cygdrive/c/..."

accessed by the Python code snippet,

hostname = os.environ['HOSTNAME']
SOURCE = self.getString("SOURCE." + hostname)

I haven’t worked out Localization analogous to Java resources, but it seems like a natural extension.

To use the code, just add the module res to your PYTHON_PATH and use it as a base class for each object (named like the module, if you want foo.py and foo.properties, for example).

# Resource files are named after class:
# class_name.properties
#
# If this class is the base class, then properties
# file is required.
# Logging is standard Python logging framework
import sys

class res:
    def __init__(self, logger=None):
        self.logger = logger
        self.resMap = {}
        filename = ''.join([sys.path[0],'/',self.__class__.__name__,".properties"])
        if self.logger is not None:
            self.logger.info("opening properties file %s"%filename)
        try:
            f = open(filename, 'rb')
            for prop in f:
                if prop[0] <> "#" and prop[0] <> '\n':
                    list = prop.split("=")
                    value = ''
                    for i in range(1,len(list)):
                        value += list[i] + "="
                    key = list[0]
                    value = value[:-1]
                    self.resMap[key] = value.strip("\n\r '\"").strip('\n\r')
                    if self.logger is not None:
                        self.logger.debug("property %s set to '%s'"%(key, self.resMap[key]))
        except IOError:
            if self.logger is not None:
                self.logger.info("unable to read resource file %s"%filename)
            else:
                sys.stderr.write("unable to read resource file %s\n"%filename)

    def getString(self, key):
        if key in self.resMap:
            return self.resMap[key]
        else:
            if self.logger is not None:
                self.logger.error("resource %s missing"%key )
            else:
                sys.stderr.write("resource %s missing\n"%key )
        return ""

Be sure to call res’s __init__ function for your class. Notice you can call it with a logger reference if you are using Python logging.

class test1(res.res):
    def __init__(self):
        res.res.__init__(self)

To access a property,

case1.getString("key1")

where the properties file looks just like it’s Java counterpart. If you try it out or make any extensions, let me know how it goes.

Building Systems Insight changes focus

2009 November 24
by DrSkippy27

Building Systems Insight has relaunched and has a new website explaining our change of focus away from short-term measurement and verifications projects for one-time upgrades such as lighting retrofits.

BSI’s new focus is on monitoring total building critical resource use.

Critical resource monitoring includes energy measurement, logging and data mining.  But in addition, BSI’s data warehouse and analytics are available for watching gas, water, waste water, temperature and nearly any other sensor that produces a stream of raw data.

Building Systems Insight has events and alerts in development as well as enhancements to the time-series analysis tools released in August.

Climate change – conspiracies are permission to deny the obvious

2009 October 29
by DrSkippy27
Pew research shows we dont beleive what we see.

Pew research shows we don't beleive what we see.

The idea that most of us don’t believe in climate change stuns and amazes me.  (I assume this applies to “global climate change” as well, and that people are not making a subtle and sophisticated distinction between “global warming” and “global climate change.”) Somehow global climate change has become a “conspiracy theory” for which expressing skepticism in public is acceptable or even intelligent.

One of the ways this has happened is that there are two types of questions that seem to be confused in much of the public discourse.  The question of “What do we see?” is a different kind of question and has a different kind of truth than “What can we infer from it?” and “What ought our response to be?”  The uncertainties are different and the type of public policy discourse around one verses the other ought to be quite different.

But for us to say we don’t believe in global climate change is simple to deny what we see.  There are many sources showing one outcome-melting ice. Here is one interesting example:

But there are many other ways to see it as well. Try to find some for yourself–that would be a responsible thing to do as part of a democracy in which we collaborate on policy decisions.

Why do we think the idea of global warming is merely a conspiracy theory? What do the promoters of the conspiracy theory gain? What is their motivation? How have we been lulled into waiting for scientists to settle a debate that isn’t going on anymore among the scientists.  The only debate seems to be among people who aren’t looking around and asking basic questions like “if a lot ice is melting, what must be the case regarding temperature?” and yet refuse to listen to experts who are.

Social proof and residential energy monitoring

2009 October 27

A short article in The Atlantic got my attention today.  I have been thinking about the projected allocation of spending on the Smart Grid over then next decade.  It turns out that the main costs will be for upgrading the electricity distribution grid to deal with a number of issues with the current infrastructure:

  • Evolved slowly, as needed, over the last 4-5 decades
  • Built to provide power from central generation facilities, not solar panels in New Jersey and the Southwest, or wind farms in North Dakota.
  • Not built to re-direct the flow of energy effectively over large areas.
  • No electrical energy storage
  • Nor was it built to support an efficient market of energy trading (not the infrastructure nor the organizations)
  • Not taking advantage of information technology for grid management

Although you wouldn’t know immediately based on the hype, only a small fraction (much less than 20%, the entire AMI and DR budget) of the spending will go to home energy monitoring and control [ref]. This spending makes sense based on the ratio of available savings from home energy monitoring/cost to deploy monitoring and control.

Robert Cialdini is implementing an interesting alternative to automated demand response in the home: publish comparative data on energy use to motivate changes in behavior.

Now Cialdini is applying that concept to energy consumption, with promising results. Positive Energy, a company that has drawn on his work (he’s the chief scientist), has created software that assesses energy usage by neighborhood. Results are sent to consumers on behalf of their local utility, praising you with a row of smiley faces (you’ve used 58 percent less electricity than your neighbors this month!) or damning you with none (you used 39 percent more electricity than your neighbors in the past 12 months, and it cost you $741 extra).

- From Greening With Envy, The Atlantic

The article goes on to explain the ideas behind “social proof” and tout the successes of the project.  This approach makes a lot of sense because:

  1. Passive monitoring is boring.  No one but the most devoted energy geek is going to sit and watch their energy use numbers roll by in order to respond to “energy events” after the first few weeks.
  2. Significant levels of automated demand response are irritating.  ”My washing machine stopped mid cycle because the energy company ran out of capacity?”
  3. Monitoring and control is difficult to justify when the payback period on installation and monitoring systems from electricity bill savings is 5-10 years.

The aggregation and simple publishing idea from Cialdini seems practical–compatible with human psychology and cost effective.

Recursion – Part 2

2009 October 23
tags:
by DrSkippy27

I love this sign!

A sign about itself.

A sign mostly about itself.

About a year ago, I posted some self-referring python code toys.  I always find these things amusing.  I feel recursion, like Steven Wright’s humor, in my brain with a mental “clunk” and a sense of delight and awe–maybe synethetically?  I also believe that recursion is deeply related to consciousness and human-style intelligence, so the experience seems important.  Enjoy.

Hat tip: Hannah Kaiser

Delicious Wordle tag cloud

2009 October 2
by DrSkippy27

The Wordle tag cloud widget provides a few moments of fun.  I noticed the shifting focus of my browsing activities over the last few months as I have been working on BuildingSI WattsGoingDown. While “energy” is growing, projects are getting less attention.

Delicious Tag Cloud - DrSkippy27

Delicious Tag Cloud - DrSkippy27

Corruption and democracy – Lessig Lecture

2009 September 10
by DrSkippy27

Lawrence Lessig has shown up here before because I think he is an important political and legal thinker. His latest book, Remix, was humorously illustrated on Colbert and Lessig provided an early explanation of Sarah Palin’s political “qualifications.”

In 2007, he announced he was moving from focusing his work on copyright to focusing on corruption. The last chapter of Remix hints at how his focus moved to corruption during the 10-tyear period he spent working on copyright.

The presentation embedded below is a little old and I am sure Lessig’s thinking has advanced since this was posted, but it is still a relevant introduction and a clear framing of why this question matters very much to the future of democracy. (It is not about Al Gore, even though that is the static screen capture shown for the video by Google Videos.)

Reframe the health care debate

2009 September 9
by DrSkippy27

President Obama is scheduled to talk to us about health care this evening. Anyone notice how much of the health care debate isn’t about health care? I hope some of the follow on conversations are more substantive than much of the discourse over the summer. The results of a democratic process depend on the quality of the conversation. But we are spending a lot of time and money talking about other stuff.  Aside from some of the more obvious quality issues with the health care debate (e.g. bringing guns and trying to talk about gun control at a health care debate, focusing on the fabrication of “death panels,” ideological extremists decrying communism,  and the fact that many of the most vocal opponents of any health care reform already use a single payer government system [usually medicare]), we don’t talk much about the salient health care reform issues.

Here are the topics I want to hear analyzed from all sides of the health care reform debate:

  • Economics of health care. This one is complex.  What are the trends of health care cost in terms of percentage of income and GDP?  How does this compare to 10 years ago? How will it look in 10 years? How much do we spend on health care vs. prevention vs. administration?  Who is paying whom for what?
  • Health insurance.  Closely related to the topic above, insurance is a statistical and information problem.  An insurance company lives and dies on population statistics and predictive mathematics.  When insurance companies have enough information, they will choose only healthy customers;  when consumers have enough information, they will only buy insurance when they are about to incur expenses.  In the world of improving information gathering, access and analysis, there will be minimal shared risk. If we believe some baseline of health care is valuable to socienty, we need to build it into the system explicitly.
  • Rationing.  (Again, broadly and economics problem.) Health care is rationed now.  People without insurance don’t get non-acute care. Insurance companies tell you what is covered and when. Insurance companies ration based on profit. Doctors make decisions to give and withhold care based on the personal ethics, pressures of time, cost, and legal considerations.  Health care will be rationed in the future. The question isn’t if, it is how and who and whether the process is transparent.
  • Medicine as art and science.  Doctors, understandably, want autonomy.  Insurance companies, understandably, want less expensive treatments and minimal responsibilities, and patients, when scared, want everything possible.  We need to improve our ability to synthesize treatment form the experience and intuition of doctors (art) and both statistical and causal understanding of how the body works (the science of outcomes we can cause).  And we need to take some care around the “innocent frauds” of expensive drugs and treatments that don’t improve health or quality of life.
  • Learning from other countries.  We seem to be pretty smug about our health care system.  Yet the rest of our economic and political peers in the World spend significantly less (20% – 50% less in the case of our favorite punching bags of Canada, Great Britain and France) and get similar outcomes (statistically) to ours from their health care systems.  We continue arrogantly dismissing these successes as if we have nothing to learn.  They have tried things that work and some that don’t. Ignoring this experience is foolish.
  • Agency.  The current system is expensive and inefficient not primarily because of “conspiracies” but due to the system structure. Each party is making rational decisions given the compensation and risks each party has to deal with.  This problem isn’t going to fixed by encouraging people to “be better.” They are already being socially responsible, rational agents in most cases.  We need to redesign the system so that decisions that are good for the country are also good for the individual agents in most of their daily work.
  • Inevitable change. The current trajectory of health care costs points to an unsustainable future.  Major reforms of parts of the system will happen–one way or another, now or 10 years from now. These problems can’t be solved by ideological soundbites yelled louder than the next guy’s.

We can improve on a design through better conversations.

Verification of energy savings

2009 September 8

One of Building Systems Insight WattsGoingDown’s clients has recently completely a lighting upgrade. There was a lot of discussion of the energy demand of the old lights, speculation about the energy demand of the proposed upgrade lights and how long it would take to break even on the upgrade.

WattsGoingDown BSI provides online access to real time energy use information for our clients and partners. In this case, verification is fairly simple but it illustrates the power of easily accessible, real time energy intelligence. Saturday morning, I grabbed as screen shot from BSI’s WattsGoingDown’s SaaS application.  It shows energy savings of 50%. The upgrade reduced lights-on power from 32 kilowatts to 16 kilowatts.  The upgrade happened over two different 2-day periods.  Looking at the plot, it isn’t hard to tell when these upgrades were going on…

WattsGoingDown shows clear energy savings from lighting retrofit.
WattsGoingDown Building Systems Insight demonstrates energy savings from lighting retrofit.

First Shopping Run With B.O.B. Yak

2009 August 28
by DrSkippy27
Townie and B.O.B Yak

Townie and B.O.B Yak

I recently found a nearly new B.O.B Yak on Craigslist.  Today, I made my first shopping trip with it. The trailer performed well and extending my townie’s capabilities to grocery shopping and running other errands is going to further increase apathy toward my car.

The trailer install amounts to changing the skewer on your back wheel to the “hitch” provided by B.O.B.  In my case, I had to shorten the skewer by about 1/4″ with a hacksaw, so my install took 15 minutes instead of five.  The hitch consists of short extensions on each end of the skewer that support the tongue rails, which are held in place by cotter pins. Attaching and releasing the trailer takes about 30 seconds.

The trailer is rated to 70 lbs.  I found that 5 gallons of water wasn’t much of a challenge for riding straight down the street, but when navigating sidewalks I turned too sharp and the trailer flopped over awkwardly.  I think if you put 70 lbs on it, the weight needs to be packed as low as possible on the trailer to lend stability.  And water is a little more challenging because it sloshes about a bit.

The bag is a nice addition as it kept all my other groceries safely contained in the trailer, but I suppose any appropriately sized duffel bag would do.

I am not sure the suspended version would work any better for me.  I have only paved roads between home, grocery stores, hardware stores, coffee, etc. and the Yak trails along comfortably on these surfaces with and without load.

The B.O.B Yak may become my favorite bicycle accessory since the handle bar cup holder.