The sTate of Things - Python

As Promised to TriZPUG: EPDB

nospam@example.com (Joseph Tate) — Fri, 29 Jan 2010 00:36:37 -0600

So I did some digging around after giving my off-the-cuff lightning talk at TriZPUG tonight and it looks like some other ex-rpathers (Thanks Dugan and Gafton!) have forked epdb. There's also the the rPath tree synchronized from here but this tree is missing some of the latest changes. The dugan tree is "python setup.py installable" now, instead of using make, and some shortcut documentation has been created, so I don't have to make this post as long as I thought I was going to have to.

For those who didn't see my little demo, epdb is like pdb (the standard Python debugger), but it adds multi-line text input, history and tab completion, nested debugging from the debug prompt, shortcuts to introspecting code, and a very nice post mortem debugger. Last, but not least, it also contains a server and client for remote debugging. The docs are still pretty sparse, but hopefully more attention can help fix that. I'd also be happy to answer questions about it.

An Exercise in Python planet link recursion

nospam@example.com (Joseph Tate) — Thu, 31 Dec 2009 13:16:43 -0600

Thanks to Chris Calloway, TriZPUG has a planet now. I don't know why it's taken me so long to connect with this group of people (I've been working with Python for 5 years now), but it's a pretty cool group from what I've seen so far. Thanks for making me (a TurboGears guy) feel welcome among all you Zope/Plone/Django developers.

Better E-mail validation

nospam@example.com (Joseph Tate) — Wed, 02 Dec 2009 15:05:30 -0600

Due to several shortcomings of the stock formencode email validator, I forked it and extended the test suite. This fixes the two most glaring issues I know of, namely the inability to handle unicode strings (international domains), and several problems with input checking (e.g., allowing commas) where invalid e-mail addresses make it through.

I did not write most of the code, I just refined it and added tests to exercise it. Let me know if it's useful to you, and if you find problems with it.

Icon View Style Grid Layout?

nospam@example.com (Joseph Tate) — Sat, 26 Jul 2008 13:28:16 -0500

For a couple of projects now, I've wanted a grid layout engine that is similar to how desktops display lists of icons: nearly fixed width items, but varying slightly in height, displayed on a variable width page, so your layout could end up with 1 column or 8 depending on the width of the browser window. Tables are no good because they're always a fixed number of columns. Div elements using float works, so long as you make all the elements a fixed width, but they also have to be the same height, or you'll end up with gaps. I'm thinking it's going to have to be javascript driven including redrawing when the page size changes, and to manually size all items to the tallest item in the row, but I can't seem to find an example on the web anywhere (or my Google foo is weak).

Dear lazy web, can you point me in the right direction?

PyCon 2008

nospam@example.com (Joseph Tate) — Mon, 17 Mar 2008 09:57:40 -0500

I just returned from PyCon 2008 in Chicago, where I connected up again with a few people I met last year and met a few new people. Rather than write a travelogue, I'll just highlight a few of the main things that I learned/did.

It looks like some of the niceties of Zope2 are going to leave the silo under the Repose project. This means that the zope transaction manager (repoze.tm), retry engine (repoze.retry) and other zope features can be used by your WSGI application. Also it means that Zope can be hosted as a WSGI app.

I learned there are ways to extend _import_ that don't involve rewriting _import_. I don't know if I'll ever need to know this, but it might be useful in a plugin loader of some sort. There's a flow chart in Brett's SVN.

I should probably use more weak references in my Python code. Dr. Tim Couper gave a lucid overview of references weak and strong, and practical ways of dealing with circular references, including how to detect them using unittest. I hope he posts his slides soon.

Kevin Dangoor gave a slick demo of TG 2 and Dojo, and gave a quick introduction to Comet servers. It looks like dojo does pretty much everything that Mochikit does, but adds support for Javascript UI widgets. Comet looks really cool. More about that later.

Tahoe is a nifty looking distributed remote filesystem concept. It needs fuse drivers though before it's really a filesystem rather than a storage mechanism (like S3). It's all RESTy though, which makes some people happy.

There are metrics for measuring code complexity, and python has tools to do that. Basically every branch in a method increases complexity by one. Keep complexity down to 10 or less to make unit testing feasible. More than 10 and the number of unittests required to cover every branch starts to get unmanageable. Apparently there is a PyMetrics module for measuring code complexity. Here are the slides from a very interesting talk.

I held a BoF on i18n'izing web applications, and a few people helped me to brainstorm this problem. The general consensus was that you should do translation as close to the user as prudent; view certainly, controller is ok, but never in the model, if you have messages that are generated by some other process, or that gets cached to the database, the "data" should be stored separately from the operational message. Generally you should avoid sending this kind of data to the end user, and instead abstract it with messages of your own. Barry Warsaw came in to the BoF with a hard problem; usually when you're handling web templates, you filter out the non-element text and build a string table out of that. What about the cases however, where you have <em> or <b> tags? I think that if you had an XML parser that would extract the text elements, and if sub-elements are present lump them with the text. For example, if you had "This text <b>needs</b> to be translated", it would be collected as one string, but if you had "<div><span>Phrase 1</span><span>Phrase 2</span></div>" it would be two phrases.

There are a couple of cool tools that have stemmed from the PyPy project. The first offers a framework for distributed testing. Py.test does stack introspection so that simple "assert" statements can be used instead of the pseudo self.assertEquals().

I got together with some people to talk about Python Packaging. Here are some notes from the meeting, and there's been some additional discussion on the distutils-SIG mailing list. Jeff Rush did a tutorial on Thursday, and his slides and exercises are available. Hopefully something coherent comes out of this, but it looks like more of the same resistance to making application installation reasonable.

I talked briefly with Ivan Krstić and Noah Kantrowitz about python plugin frameworks. Hopefully we'll be able to collaborate some on a generic system for application extensions.

Probably the coolest thing I learned about was a couple of protocols, both implemented by orbited. Orbited's client libraries allow for a user to connect to a orbited server to provide a push mechanism for sending messages to a web browser. The Orbited team was shooting for a 0.4 release after the PyCon Sprints this week. Check out their IRC client demo.

PyCryptoPP was suggested to me as a decent Python based Crypto library, since it's simply python wrappers for Crypto++. PyOpenssl has a new maintainer, and should be getting some much needed attention.

I also enjoyed Raymond Hettinger's talk on "Core Python Containers". It was very helpful in understanding what list, dict, deque and set do behind the scenes, and how to use them most efficiently.

Overall, I had a great experience at PyCon 2008. The venue was big enough for all of us (there were more than 1000 registrants), and there was more than enough room for Open Space/BoF talks.

Chicago style pizza is definitely different from anything I have eaten before. The jury is still out as to whether it's worth the hype.

PyCon 2007

nospam@example.com (Joseph Tate) — Mon, 26 Feb 2007 15:08:39 -0600

Last night I returned home after 3 days at PyCon 2007 in Dallas Texas. I think my brain is full. It was great to meet people to whom I've talked on IRC or email, or whose software I've used. It was also fun to hang out with Mark Ramm, Robert Brewer, Ian Bicking, Ben Bangert, Rob Orsini and Chad Whitacre at Robert and Chad's suite on Friday night. I haven't laughed so hard in a long time.

As Robert mentioned in his blog post, there was some mind melding, er, meeting on the part of Zope, Pylons and CherryPy as well as others. I'll look at the way that sockets are handled and contribute some code. I'm looking forward to this collaboration. In addition to what Robert mentioned another thing we decided was to come up with several "stories" for deployment and help guide users down the right path for them when deploying a Python web application.

For the rest of the conference, most of the keynote and full length talks were good, the lightning plenary talks were great, having several rooms to hold Birdts of a Feather (BoF) or non-scheduled talks provided much enlightenment, owning an 802.11A wireless chipset was a difference maker in connectivity and bandwidth, and lots of Pythonistas have beards (some of which are out of control)

I noted several things that I thought merited additional attention:

trac -- I didn't know it could be used with mercurial instead of svn

pyjamas -- generate javascript from python code

Idea (some kind of testing tool, can't find a link)

IPython -- Interactive Python including a replacement for pdb

Jack Slocum's YahooUI extensions/datagrid

Fitnesse -- A framework for collecting functional and integration tests

Zope's TestRunner -- Has something that could be used for memory/object leak detection

Monit -- might be used for the same

Star Schema -- Data reporting in memory instead of in db

Packaging eggs instead of building new setuptools targets for linux packaging systems.

zdaemon, supervisor2 and other libraries that may make daemonizing a python program trivial.

Lots of cool testing tools: figleaf, twill, scotch

Hopefully I can make it again next year.

TurboGears 1.0b1 VMware and Xen images available

nospam@example.com (Joseph Tate) — Wed, 01 Nov 2006 08:28:38 -0600

I've been able to package all of TurboGears 1.0b1 with Conary. And with the new release of rBuilder Online, Xen domU images can be generated just as easily as VMware images. You can download images for Xen, VMware, Qemu, or Parallels here. Please let me know if you are interested in install CDs, and I'll generate them to add to the release.

Turbogears help wanted

nospam@example.com (Joseph Tate) — Fri, 15 Sep 2006 20:22:02 -0500

My employer, rPath, is hiring web developers to work on the rPath Appliance Agent team. rAA of course is built on TurboGears. See this page for more details.

rAA in the News

nospam@example.com (Joseph Tate) — Tue, 29 Aug 2006 10:41:04 -0500

So my last entry, oh so many moons ago, mentioned that I was working on a new project. Now that that project has been released, I can talk about it, but better yet, how about an article about rPath Linux, what we're doing, and the rPath Appliance Agent (or rAA) which I helped design and build.

A little bit about rAA itself; it provides a pluggable framework for system configuration tasks, like setting up networking, setting the root password and checking for and applying software updates. There are some screenshots of rAA in action linked from the eWeek article above. rAA is written in python and built around TurboGears, a web application framework for RAD and MVC web development using Python.

rAA is Open Source Software, released under the Reciprocal Public License. A conary package group that can be added to a rBuilder Project to insert rAA into your own custom software appliance is available from the rAA Project.

Some basic documentation for rAA is available here. It's built with rPath Linux as an assumed installation platform, but several appliances have rAA built-in, and have VMware images available so you can try it out without installing a new Linux distribution.