Jeethu's Blog

We’d discussed about Uraga at the last BangPypers meeting. Uraga aims to be a package manager for Python, quite like CPAN for Perl and Gems for Ruby. We’d also discussed about Python Eggs, the developing standard for a Java Jar like package format for python. And now there’s the new EasyInstall, which roughly resembles the Uraga spec (as I understand it) except for the fact that its based on the Python Egg format, while Uraga is not. I can’t wait to get back home and sink my teeth into its code. I might learn a lot of stuff that can then be applied Uraga (Words out that we’re gonna have an Uraga coding sprint soon).

Some major blogwars happening on Tim Lesher’s blog Aftermarket Pipes: Unsung Heroes of Python: asynchat/asyncore. Being a neutral spectator (albeit with some Twisted experience in the past), since I’m not doing anything with Twisted anymore, I’d say there’s no point in arguing. Asyncore and Asynchat serve a different purpose than what Twisted does. When I tried twisted, I did that for then number of protocols which were available on it. I’m lazy. I have reinventing the wheel another reason I remember is Deferreds in Twisted. This is one feature which you really can’t miss once you get used to it. If I was writing something that should run on any out of the box Python installation, I’d use asyncore, but fortunately for me, I had a choice and I could leverage the protocol implementations already sitting in the twisted code base. The dependency is an issue, but it was’nt a concern for me. So the bottom line is variety is the spice of life. Thats what I love about Python. We have so many options, which most non Python drones simply don’t have. If I think asyncore is best for a project, I’ll use that or maybe I’ll use BaseHTTPServer or maybe Twisted.web. Its all a matter of making the right choice in the right situation.

Its raining like crazy here in the whole of Bangalore city. Its raining just about every evening. My neighbour told me that we had hail in our area on last Saturday (I’d missed it, since I’d been to the BangPypers meetup that evening).

Just read this thesis by Brett Cannon. Pretty interesting stuff. I got into OCaml predominantly for speed. And though speed is not directly associated with type inferencing, one of the strong points of OCaml is type inferencing. From a certain perspective, type inferencing can also give the compiler the a definite edge during optimizations. One fact that I realized after learning a bit of OCaml (I’m still learning it!) is that its as strongly statically typed as any language can get. I deliberately term this as strongly statically typed because Strong Typing is different from Static Typing, quite like in Python’s case. Python is Strongly Typed and Dynamically Typed Language, C is a Weakly Typed and Statically Typed Language (void pointers). Ocaml is even more statically typed that C or C++.

Ok, enough of Ocaml worshipping.

How can we apply this wisdom to Python ? I love Python for (among other reasons) its Dynamic typing. It takes the best advantage of being an interpreted language. If you know Python, you’ll know what I mean. I’m pro optional static typing for Python. If we can have optional static typing and tools like Psyco can take advantage of it (well, why Psyco, the very compiler can take advantage of this.)

Take for instance, this snippet:

>>> def func(x):
return x + 2
>>> import dis
>>> dis.dis(func)
2           0 LOAD_FAST                0 (x)
           3 LOAD_CONST               1 (2)
           6 BINARY_ADD
           7 RETURN_VALUE

I think even Brett’s Type Inferencing avatar of Python (with its restricted bytecode-compiler and interpreter modifications) can take advantage of this, by emiting an optimized INT_ADD instruction instead of the generic BINARY_ADD instructions (For the record, there’s no INT_ADD instruction in Brett’s list of optimized instructions).But, if there’s some type information here, like in the snip below:

(This is a hypothetical situation, does’nt work (yet!) in Python 2.4)

>>> def func(x:int):
return x + 2

And if the compiler does take advantage of this, it can effectively inline a call to the function. This example is way too trivial, but any static transformation can be optimized out (Quite like the optimizing compilers for static languages). Which could lead to us the 5% or more improvement that Brett was looking for in the first place.

I’ve blissfully ignorant about Lisp Compilers all these days. Lisp is a Dynamic Language and yet there have been optimizing Lisp compilers for decades. Well, we all knew they were around. But since almost the whole world is Imperative, I’d been under an impression that Lisp is primitive (And believe me I was compelled to think that way.). But all that started to change about 2 months ago, when I got into OCaml, and everyone in the Functional part of the programming world regard Lisp as the grand daddy of all High level languages. We might have a lot of lessons to learn from the Lisp camp, on compiling Dynamic Languages.

I found Andy Todd’s blog entry ‘Returning Rows as Dicts in MySQLdb‘ while browsing the Daily Python URL. This is something which just about every one of us has reinvented at some point of time. Here’s a class that I use very often to wrap PyGreSQL cursors. I’ve not tried it with any other DB interfaces, but does’nt use any DB compliant features, so it should work fine with any DB API 2.0 compliant DB interfaces.

class DictCursor(object):
 def __init__(self,cursor):
     object.__setattr__(self,'__cursor',cursor)
     object.__setattr__(self,'__dirty',True)
     object.__setattr__(self,'__description',[])

 def __getattribute__(self,x):
     try:
         return object.__getattribute__(self,x)
     except AttributeError:
         cursor = object.__getattribute__(self,'__cursor')
         return getattr(cursor,x)

 def __setattr__(self,x,v):
     setattr(self.__cursor,x,v)

 def fetchone(self):
     cursor = object.__getattribute__(self,'__cursor')
     dirty = object.__getattribute__(self,'__dirty')
     r = cursor.fetchone()
     if dirty is True:
         description = [x[0] for x in cursor.description]
         object.__setattr__(self,'__description',description)
         object.__setattr__(self,'__dirty',False)
     else:
         description = object.__getattribute__(self,'__description')
     if r is not None:
         return dict(zip(description,r))
     return r

 def fetchall(self):
     cursor = object.__getattribute__(self,'__cursor')
     dirty = object.__getattribute__(self,'__dirty')
     l = cursor.fetchall()
     if dirty is True:
         description = [x[0] for x in cursor.description]
         object.__setattr__(self,'__description',description)
         object.__setattr__(self,'__dirty',False)
     else:
         description = object.__getattribute__(self,'__description')
     for count,x in enumerate(l):
         l[count] = dict(zip(description,x))
     return l

 def execute(self,*args,**kwargs):
     object.__setattr__(self,'__dirty',True)
     cursor = object.__getattribute__(self,'__cursor')
     return cursor.execute(*args,**kwargs)

 def __iter__(self):
     return self

 def next(self):
     x = self.fetchone()
     if x is None:
         raise StopIteration
     return x

if __name__=='__main__':
    import pgdb
 connection = pgdb.connect(user="user",password="xyz",host="localhost",database="db")
 cursor = DictCursor(connection.cursor())
 cursor.execute("SELECT * FROM x LIMIT 10")
 for count,x in enumerate(cursor):
     print count,x