Skip to content

A zope.component introduction – Part 2

zope.component

zope.component is a reusable python package that usually comes with zope.interface and zope.event. We can split this package into two categories:

  1. Components: adapters, utilities, named adapters, multi-adapters and others
  2. Registries: global component registry, local component registry.

What we need to remember from all this is that there are registries that store adapters. The idea is to use these registries through out our application. Think of it as a global dictionary with values: adapters and keys: interfaces – because it’s exactly what it is. Without further ado we’ll jump directly to a simple example, but instead of adapters will use an utility (an adapter that adapts nothing).

A small utility

Let’s see what we did here:

  1. We declared an interface and an utility class Hello.
  2. We got the global registry and registered our utility for the ISpeach interface.
  3. In the end we used our utility first by querying the register and then just used it as a Hello object.

The benefits

The nice thing about registering a utility like this is that you can change its location in the project, you can refactor it and give it a different name, you can override it with another utility at will – and most importantly is that the last 2 lines never change! Here is why:

  • All we really care about is the interface with which we are making the query – the implementation can change in the future.
  • The code is decoupled (no direct imports to the implementation) – just the interface is known to us.

A direct result to this is:  more control for the developer which can replace the implementation at will without worrying that he/she needs to refactor a lot of code. Looks like a lot of code just to register one simple utility isn’t it? I couldn’t agree more, but remember that we are talking here about big applications with tons of utilities and adapters and events and event handlers. In the end everything is a structured chaos. It’s structured because as long as we have a registry that has all the components we have an overview of our application and all its components.

But my application is already far in it’s 3rd year of development. I don’t have everything in one big component registry!

The good news is that you don’t need to have everything as components – I strongly advice against it actually. You can use this just for the parts you feel that your application will benefit the most. You can gradually use it where it makes sense.

Now let’s return to our apples and oranges example, but make it use adapters.

Oranges and Apples – again

It’s really not that different from our previous example with apples and oranges except the whole interfaces and registration stuff. The benefit however is that this is a lot more usable in the long run for the reasons I explained above. And the difference from the utility example is that it actually adapts something – IApple in our case. The key thing to remember is that all the registry queries are done using interfaces.

In the next part I will try to draw the big picture, what else can components do besides adapters and utilities and what I personally don’t like about their implementation – discussing possible alternatives.

Categories: coding, Uncategorized.

Tags: , , , ,

A zope.component introduction – Part 1

Note: If you are already familiar with the adapter concept and you just want to learn about zope.component you can skip to part 2.

The problem

Recently I gave a talk about zope component architecture at a company in Paris where I started consulting on zope, plone, django and python in general. In these posts, aided by code examples, I will try to show what  zope components are capable of, what are the common use-cases and some of my personal views about the subject.

First lets identify the problem that we are trying to solve using a component architecture:

Applications tend to get big. Especially those applications that have a lot of users and have years of development. Users want different features/changes in different stages of the development of the product. Hence the key to a successful and long lasting application is to comply with the user needs as fast as possible with as little technical dept as possible. Finding a balance between the two is not easy.

Speed of development is important especially in the beginning of a project. Your clients want that software today and you can’t waste time on thinking a lot about long term benefits if your bank account is empty. Those benefits are innexistent if you don’t finish in time so the shortest path usually gives you the best chances of success. This is the correct way to think about it. However when the project is getting bigger you may want to start thinking in different terms. Questions like these are usually good onces to ask:

  • How much time does it take to create a feature?
  • Refactoring is hard, but can it be made faster?
  • Is it easy to extend existing features?
  • How different parts of your application talk to each other?
  • How about debugging?

These questions matter both to managers and coders. Even if the managers and developer are very good at what they do usually what happens is that layer after layer gets added on top of the existing product until it becomes a big pile of crap – new developers are screwed because they can’t figure out what is happening in the system and the time to train them is very expensive. New features take enormous amounts of time to develop and your technical dept gets bigger and bigger with every new modification.

Very little can be done about this once you’re big. There are no easy solutions and old applications sometimes just whither and die. Tough. But maybe components can help.

Prelude

Python is a great language. It is easy to learn and there are many different frameworks that can get you started really fast. What I found very important about python is that it’s a good prototyping language – some complain that it’s too good. It’s no accident that it’s the language of choice for many successful startups (Dropbox, Disqus, Quora, etc.).

So once you start delivering and get some sort of income you try to improve the product to make your users happy. You provide missing features, improve stuff. More code, more complexity, more tests, more features, more developers and so on and we get back to our problem.

Zope components architecture promises to alleviate some of the problems of big applications by using adapters as a way to organize code. If this is true for your case you will have to decide for yourself.

What’s an adapter?

Wikipedia:

In computer programming, the adapter pattern (often referred to as the wrapper pattern or simply a wrapper) is a design pattern that translates one interface for a class into a compatible interface. An adapter allows classes to work together that normally could not because of incompatible interfaces, by providing its interface to clients while using the original interface.

In other words, an adapter (usually a function or a class) allows two interfaces to talk to each other. Consider this example:

Say we have interfaces iA and iB, receptively  A and B their implementations. At the beginning they don’t know about each other. But then you decide that you want to make them talk to each other. Let’s say A wants to list all Bs. At the same time you don’t want to change A or B because they are central to your application and a lot of stuff uses both A and B and changing them can be both ugly and difficult. This sucks!

The good news is that in some cases this can be avoided by using a third party that acts as a intermediary (the adapter) between iA and iB and provides functionality that did not exist before. Not only that, but it allows you to keep your A and B unchanged.

Let’s forget about interfaces and adapters for a while and let’s consider this simple scenario of apples and oranges in a basket:

It’s pretty clear that Apple doesn’t have slices and the last line will fail with an AttributeError. 

So how do we make apple have slices? There are many possible solutions. But let’s impose these 2 restrictions (I will explain why later):

  1. We are not allowed to modify the Apple class
  2. We are not allowed to subclass the Apple class.

A possible solution is to create a wrapper class around the Apple class that will give us the functionality we need.

So let’s see what we did here.

  1. We had an Apple class which didn’t have any slices
  2. We created an AppleWrapper which takes an Apple object and provides a slices function which can be used later with that apple object.
  3. As far as get_slices is concerned it just needs to call a function called slices on basket’s items.
  4. We also didn’t subclass the Apple class and most importantly Apple class remains untouched.
  5. AppleWrapper is what we call an adapter and the apple object is the adaptee.

Why it is so important not to subclass?

For a longer explanation of why subclassing is considered harmful search for “inheritance is evil”. The short explanation is that we don’t really need to inherit all Apple’s properties and methods – we just need to make it have slices and that’s it.

Why just not modify the Apple class to give it slices?

Because we don’t really need the Apple to have slices all the time. Just the time when it’s in a basket. This way the Apple class will stay a clean and simple – and as a result easily extendable in the future.

Ok. Now that we understood how an adapter works we can switch to a more complicated example and this time we are going to use zope components.

Go to part 2 in which I will explain how to use zope.component.

Categories: coding.

Tags: , , , ,

Go pathfinding

Remember:

I found out why it’s a good idea to implement common algorithms in a language you wish to learn.

First because you don’t use advanced language features. Simple constructs should be sufficient to implement an efficient common algorithm. You don’t have to memorize a lot of new concepts which can be confusing and can throw you off your path. Sometimes even quit learning that language! You become more accustomed with low level stuff before getting into complicated standard library features.

Second advantage is that you immediately start doing something practical. I mean who doesn’t need A* or Dijkstra? Everybody?!

Practical:

So I needed a good path-finding algorithm such as A*. I wanted it to be implemented in golang. I wasn’t  able to find any standalone implementation of pathfinding algorithms in golang. Naturally I implemented one. So here it is:

https://github.com/humanfromearth/gopathfinding

Lessons learned:

  • I found that if you can use slices instead of maps you should always use slices.
  • I found that you don’t need to use make all the time. You can use append which does the allocation for you.
  • I found that pointers can be really cool.
  • I found that again golang is a robust language and gives great hints when hitting an error.
  • Did I say I love the test framework?

Update:

This is the answer to Patrick’s comment:
Why should I use slices instead of maps when possible and why append is sometimes better than make?
First of all you can’t use the built-in append function on maps – just slices. So adding new values to maps or extending maps are not as easy as for slices. ‘append’ is very powerful. Here is why:
Maybe the above it’s not the best example, but you can see that we need to specify a length to the make function to allocate and initialize s11.  Comparing that to append: you don’t need to know the exact size of your slice to append stuff. This is better for a because  you don’t have to deal with indexes and can make your code a little bit cleaner. Also append does the allocation on runtime.  Doing lots of allocation calls is not very efficient – keep that in mind.
Because slices have append to support them it makes them more powerful than maps in some scenarios. My conclusion again: use slices instead of maps if you don’t really need maps – it’s easier.

Categories: coding.

Tags: , , ,

Make your own Wikipedia dictionaries for Kindle

One day I was siting on my balcony, reading an article I downloaded on my kindle using instapaper and stumbled upon this term: Stone-Wales defect. I had no idea what this defect was since I’m don’t know much about carbon nanotubes. Then I realized that I needed to leave my tea and the conform I got used to and get up and go to my computer to lookup what this term was all about.

I’m lazy. I didn’t want to get up and search the term on my desktop so I opened up that terrible kindle browser and tried to search that term on Wikipedia which was really slow and uncomfortable and the wireless was down – you got my point. So it struck me that I could probably create some python script to generate a kindle dictionary. So I did.

This article will be about how I accomplished this. Also hopefully it will give you an insight on how to tune this tool to make your own kindle dictionaries.

Note: If you’re not interested in the implementation details and just want to generate a dictionary for kindle you can just get the source: https://github.com/humanfromearth/wikimobi and use it. (usage details in README).

Wikipedia is big

It’s so big that all the abstracts of all the articles from wikipedia would be a 3GB mobi file. 3GB is also the storage size on a Kindle 3. That’s no good. So I needed a way to filter out  the articles I don’t need. I needed physics mostly. I wanted to create a physics dictionary! Eureka! So all I needed to do was to query the physics related articles and get their articles.

At first I tried to use the sql dumps from http://en.wikipedia.org/wiki/Wikipedia:Database_download but that failed because sql was really slow to load into a mysql database. Fortunately I found an alternative: http://downloads.dbpedia.org

These guys had archives of wikipedia in a N-Triples format which is more convening to use and a lot faster that sql (at least in my case).

Cool. So I needed a few archives.

This one contains english abstracts:

http://downloads.dbpedia.org/3.6/en/short_abstracts_en.nt.bz2 (300mb)

This one contains the article -> categories relations (so we can find the articles from a specific category):
http://downloads.dbpedia.org/3.6/en/article_categories_en.nt.bz2 (115mb)

Contains relations between categories and subcategories:
http://downloads.dbpedia.org/3.6/en/skos_categories_en.nt.bz2 (18mb)

Great! We have the data. It’s time to filter out that data and generate a .mobi file.

Again.. Problems.. Kindle really sucks at this because it uses the proprietary .mobi format which doesn’t have good open converters which I could use. So I used the mobigen.exe provided by http://www.mobipocket.com/dev/ that allows you to create mobi dictionaries from an .opf file. Ok so now I need and opf file? That sucks. Fortunately this guy: Klokan Petr Přidal (www.klokan.cz) created this script: tab2opf.py. Which converts tab separated files into opf dictionaries.

Ok so in the end I needed this:

  1. Create a tab separted file: [term]\t[definition] -> out.tab
  2. Convert out.tab -> out.opf
  3. Convert out.opf -> dicitionary.mobi

Easy right? So after writing the stuff I described above we use the tool (wikimobi.py):

python wikimobi.py -nt path_to_nt_files_dir/ -o output_file -c Category -l levels

For physics it would something like:
python wikimobi.py -nt path_to_nt_files_dir/ -o physics -c Physics -l 3

The level options goes down 3 levels in the category hierarchy so we don’t end up in Philosophy:

Extended Mind

If you have an idea on how to perform the filtering of articles based on their categories so we don’t get a lot of unrelated stuff I’d be interested to hear it.

Have fun!

Update:

On stehk’s request I’ve generated a few dictionaries:

  1. Physics: http://blog.hiresasha.net/wp-content/uploads/dictionaries/physics.mobi
  2. Chemistry: http://blog.hiresasha.net/wp-content/uploads/dictionaries/chemistry.mobi
  3. Mathematics: http://blog.hiresasha.net/wp-content/uploads/dictionaries/math.mobi
  4. Biology: http://blog.hiresasha.net/wp-content/uploads/dictionaries/biology.mobi
  5. Philosophy: http://blog.hiresasha.net/wp-content/uploads/dictionaries/philosophy.mobi
Note: The dictionaries are usable but there are still some unicode and metadata problems which I wasn’t able to solve. Hopefully someone can help me to solve those.

Categories: coding.

Tags: , , , ,

Go and Python

I see a lot of talk nowadays about Golang. It’s fresh, it’s shiny, not too many features, documented, statically typed, garbage collected, parallel, python zen inspired sweet candy.

I’m a python addict, I admit it. Looking at poor people from 3rd world of computer languages with their cluttered syntaxes and semicolons and other useless unintelligible syntactic voodoo makes me feel really good. Too good actually. When I need to do something in some other language I feel sick sometimes. I’m serious. I have a familiar feeling with the process called: eating spinach. I can eat it, don’t get me wrong here, but I don’t enjoy it. Just moving my maxilla to get the job done. My brain got comfortable with python. My synapses are now irreversibly fused towards python and python ways of doing stuff.

So what about Go? Well.. it turns out Go is remarkably similar to python in many ways. In my mind it’s like a python for the system-programming/statically-typed/compiled/crazy-fast world. For me this language is good news. Because sometimes python is too slow and I don’t want to get dirty with C. I want a language that actually has a usable standard library similar to python’s standard library that allows me to get low level enought without sacrificing my time and patience. I have a specific example I played with recently which I intend to bring forward for comparison of these two amazing languages.

I needed a Boyer-Moore-Horspool search algorithm which I stole from this recipe here. My example is a little bit modified so that it does not stop on the first search result, but continues until is done counting the number of occurences of a ‘word’ in a string:

Attention! I don’t want to compare the speed of the two languages. It would be stupid. Go is faster. What I want to do instead is to compare the code complexity/readability for both. I believe that clean-slow code is bettter than ugly-fast code. I want to see if my code is still readable in Go and if it’s close to the python example. Maybe even better for this particular example. Who knows?

The above example is not equivalent to python’s: str.count because it does more checking. As you can see there are a few optimisations here and there, but overall the code looks pretty clean. This code has a big dissadvantage. It’s called speed. Compared to the find function in cpython (which is really fast – I think it uses the same algorithm: bmh) it’s really slow. I tested this in pypy as well, it is very slow. Like 10 times slower then CPython for this example. Is this normal? Am I wrong?

Anyway.. here is the same written in Go:

Now I don’t know about you, but this looks good to me. I mean just visually compare the python example with this one. I looks similar right?

This example is bad. It may even say: “Go is better”.. It’s not true. This example surelly does not demostrate all the benefits that you get from Go or Python.. it’s a simple example.. one of many.. and should be treated as such.

Now here are a few rants about Go:

  • Api stability. Go guys please don’t change the API’s so often, it makes people nervous. It makes serious comercial development problematic. I know it’s a new language and a lot of things are changing which is normal, but keep in mind that people like me who need to get their shit done fast will think twice before building a real world product without a api ‘stability’.
  • Debugger – I hear there is one in development right now. No language is worth much without a usable debugger. I’m sorry.. I’m just too lazy to look at my code to figure out what’s the problem. I want a usable debugger right now.
  • Too little zen of python in some third party libraries. import this. Please…

I will say that Go certainly sparked my interest. It’s a cool language. It’s a keeper. I will try to find a project to actually do something practical with it.

Until then gooo!

A more complete example of Go (with tests): https://github.com/humanfromearth/goooo

Categories: coding.

Tags: , ,

An introduction

I was thinking of starting a blog about my coding experiences many times. Almost all times when I get this idea my usual reply is: “What? There are so many out there I’ll hardly make a contribution!”. Turns out I was wrong in about this. I thought about writing for others rather than myself.

That changed and here is why.

There are many good reasons to start a blog: to show of the cool stuff you’re doing, to remember stuff, to get better at explaining complicated things to a critical audience and probably others. By writing about a specific experience you tend to organize your thoughts in a sistematic way which will be dumped in a nice and clean blog post that can be digested by some people that never have time to read your posts.

All this ‘wisdom’ maybe common knowledge for others, but for me this is new.

Most of my posts will be related to Python coding encounters the new Go programming language adventures and C. I’m a web coder so most of the things about these languages and libraries related to them will be revolving around the web technologies.

So here is to a new adventure.

Categories: about.