Arc does not behave like an open-source project

that’s it. for example:

• what (open-source or free-software) license does the code use?
• where is the version-controlled repository for the project?
• where is the bug-tracker?

The answer to all this is probably that the project is too young, and all this is still in flux, and it will be clarified when the language becomes more ‘final’. I think it should be the opposite way. Paul Graham should have specified the license from the start. They should also open up the version-control-system they use to develop Arc, and if they don’t use one, they should.

If i got something wrong, please tell me. I’d love to be corrected on these.

p.s: if you’d like to try out a lisp that’s usable currently, there are a lot of them. here are some to start with: mzScheme from the scheme family and SBCL from the common-lisp family. Or, if you would prefer to try out something different, something new, try Clojure. It runs on the JVM, so you also have access to all java libraries.

but there is an issue with it seems many do not realize:

the “database” backend of google-app-engine (i will call it BigTable in the following text) is not a relational (read “SQL”) store, and it will never be. for example, it does not support SQL JOINs. but it’s worse than that. because of it’s architecture, JOINS will never be fast there. BigTable is essentially a collection of spreadsheet-tables, where you can do some basic searches, that’s all. oh, and transactions.

for this reason, there probably never will be a BigTable django-ORM wrapper. of course technically it’s possible to implement in python all the missing features, but it’s performance characteristics will not be the same as of a relational-database. so you will not be able to simply take your mysql/postgresql-optimized application, and deploy on it, and all is fine. you will have to restructure your application.

and if you have to restructure your app anyway, why do you need the django-ORM? you can as well write google-app-engine-specific code.

(on the other hand, maybe there could be a more stupid django-orm, that does not assume a relational-db-backend, and it could work with the various non-relational databases like BigTable or hBase or other column-oriented databases…)

the basic idea when writing BigTable code is that read-operations will happen much more often than write-operations. so do more at write-time, and less at read-time. denormalize tables.

for example, take a simple forum-application. it stores discussions. a discussion has comments. now let’s see how we could implement 2 basic features: “add comment” and “list discussion-names with comment-count”.

SQL:

• “add comment”: store a new comment-entry, which contains a link (a foreign-key) to it’s discussion
• “list discussion-names with comment-count”: do an SQL query like: SELECT discussion.name,count(1) from discussion LEFT OUTER JOIN comment GROUP BY comment.discussion_id; (let’s not discuss right now if it’s inner or outer join etc. it’s quite late at night here, so maybe it’s not 100% correct. but it should be enough to demonstrate the situation)

BigTable (one possible solution):

• “add comment”: store a new comment entry, which contains a link to it’s discussion. also, count the number of comments for this discussion, and store this value in the discussion-table
• “list discussion-names with comment-count”: select * from discussion

of course the whole denormalize-your-database-if-you-want-performance mantra is nothing new. if i remember correctly, Flickr also does this. but still, for most developers, (also for me), it’s just painful to give up our nice, clean, normalized db-tables.

p.s: please note, that all the info is not based on my own performance-benchmarks. it’s more a summary of what i’ve read in the the google-appengine documentation and the google-appengine mailing list.

let’s say you want to create a web-app which:

• shows a form to the user, where he can enter a number and submit it
• then you show him a second form, where the user can enter a second number and submit it
• then you show the user a page which displays the sum of those 2 submitted numbers in Seaside, you can implement it in something like this: (pseudocode)

first_num = get_number()
second_num = get_number()
display(first_num + second_num)

my language of choice is python, so i was curious if it’s possible to achieve something similar in that language. seside uses continuations to achieve this, so i turned to python generators. they allowed me to write this:

def sum_two_numbers():
    first_num = yield InputPage("enter first number:")
    second_num = yield InputPage("enter second number:")
    sum = int(first_num) + int(second_num)
    yield TextPage("the sum isresult is: %s" % sum)

the full code is below, but first some notes about it:

• it requires python 2.5 or higher, because i use the “advanced” version of yield, which appeared in python2.5
• this is only proof-of-concept code, and i know that it uses global-variables, ugly names, get-where-it-should-do-post etc.
• i know Seaside does much much more (backbutton-support etc.)

the code:

#!/usr/bin/env python2.5
from wsgiref.simple_server import make_server
from cgi import parse_qs
 
 
class TextPage(object):
    def __init__(self,text):
        self.text = text
 
    def get_response_content(self):
        return '<html><body>%s</body></html>' % self.text
 
    def get_request_data(self,environ):
        return None
 
class InputPage(TextPage):
    def get_response_content(self):
        return """<html><body>%s
            <form method="get" action=".">
            <input type="text" name="data" value=""/>
            </form>
            </body></html>""" % self.text
 
    def get_request_data(self, environ):
        get_req_dict = parse_qs(environ['QUERY_STRING'])
        return get_req_dict['data'][0]
 
 
def sum_two_numbers():
    first_num = yield InputPage("enter first number:")
    second_num = yield InputPage("enter second number:")
    sum = int(first_num) + int(second_num)
    yield TextPage("the sum isresult is: %s" % sum)
 
s = sum_two_numbers()
cur_item = None
 
def simple_app(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-type','text/html')]
    start_response(status, response_headers)
 
    global cur_item
 
    if not cur_item:
        cur_item = s.next()
    else:
        cur_item = s.send(cur_item.get_request_data(environ))
 
    return [cur_item.get_response_content()]
 
 
httpd = make_server('', 8000, simple_app)
httpd.serve_forever()

i did a quick google-check about this topic (“python web framework continuations”), and only found Nevow with Wolf, which uses a less-elegant approach, probably because when it was implemented, python2.5 was not yet available. if anyone knows about other python web-frameworks that use generators, leave me a comment.

i am very interested in distributed version control systems, so i follow the mailing lists of several of them (bzr, git, hg)…

currently my favourite is mercurial btw…

but this post is not about why i chose mercurial.

also not about why distributed version controls are much better than centralised ones (svn, cvs, etc.) (probably will write about this a post at a later point, but not now)

this is about why git is not suitable for me.

intro:

in short, git was created when the kernel developers needed a version control system after the BitKeeper problems, so Linus wrote Git.

it’s a very powerful, and extremely fast version control system. i think, if i would have to decide purely on the features, probably Git would be the winner.

the problem:

but the problem with git is the ..hmmm.. mentality?

it was written (primarily) for the kernel developers originally, and somehow what’s user-friendly for them is not always user-friendly for me.

let’s take a recent example, which i think nicely characterizes the issue:

( the discussion on the git mailing-list )

when you use git, anytime you can do “git diff” which shows you what changed since the last commit, and “git stat” which shows you which files changed since the last commit (this is not an exact definition, but it’s more&less like that).

someone found out, that if he just re-saves a file in the repository without changing it (or in unix-speak: “touch myfile”), then:

1. “git diff” will mention the file, but show an empty diff
2. “git status” will not list the given file
3. if he does “git diff” again (after the “git status”), then it will not even mention the file

so in short, “git diff” behaves differently before “git status” and after “git status”.

when he asked on the mailing list, he was informed that this is not a bug, because for performance reasons “git status” does some things to the repository, which cause that the touched-file does not show up at the second “git diff”.

when he argued that “git status” should be a read-only operation (so that doing it should not affect the behavior of the other commands), he was told that “git status” is not a read-only operation, because in reality it does some internal things in the repository, and also that if this would be read-only, then some things might become slower, etc.

then he argued that while it can be true, that “git status” is not read-only from the implementation point of view, it still should be read-only from the user’s point of view

(for example imagine a simple form of caching of anything.. when you first request some data, they are retrieved, cached and given to you, but when you request it next time, they arrive from the cache. so from the implementation point of view, the retrieval-operation is not read-only, because it writes to the cache, but from the user’s point of view it’s a read-only operation (because to the world outside of the implementation, it behaves as read-only))

but somehow his argument did not get through…

so currently i am not sure if the problem is that:

• the git-developers are not able to understand this distinction between being read-only from the implementation point of view and being read-only from the user’s point of view

or

• the git-developers think that this property of “git status” is not important enough to maybe sacrifice a little performance for it

but whichever is the reason, it just proves for me that git is suitable not for me.

i agree with most of his arguments, but the generics one… well…

i wanted to post a comment on his blog, but it kept asking me to login when i wanted to post a comment, and i couldn’t find a way to register, so i am posting my “reply” here:

where he argues about java/c++ generics/templates, he says that the java ones are basically better. while it might be true, that there are certain enhancements in the way java does templates, we must not forget, that there are certain very useful things one can do with templates, but cannot with generics.

example

#include <iostream>
#include <string>
 
using namespace std;
 
class Clazz1
{
    public:
    string get_hello_message();
};
 
string Clazz1::get_hello_message()
{
    return string("Clazz1.hello");
}
 
 
class Clazz2
{
    public:
    string get_hello_message();
};
 
string Clazz2::get_hello_message()
{
    return string("Clazz2.hello");
}
 
 
 
template <class T>
T say_hello(T obj)
{
    cout < < obj.get_hello_message() << endl;
}
 
int main()
{
Clazz1 c1;
Clazz2 c2;
say_hello(c1);
say_hello(c2);
 
return 0;
}

in this C++ program i define 2 classes, both have a method called get_hello_message, but the two classes are not related to each other. then i define a function, that takes an object of any type (and calls it’s get_hello_message method.

try this in java :)