Page 10 of 17
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 07:05
by hughperkins
aegis wrote:hibernate?
https://www.hibernate.org/
It makes storing / retrieving things to/from databases really easy in C# and Java. Basically, you just handle lists and objects in C#/Java, and hibernate handles storing them in the db if they change, and retrieving them from the db when you want it to.
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 07:19
by hughperkins
To be honest, since in python, data seems to be handed around as dictionaries and lists anyway, it's half done, for the retrieval side, since you can get such a list of dictionaries in two lines:
Code: Select all
dictcursor.execute("select * from maps")
maps = dictcursor.fetchall()
... then the question is how to change some stuff in maps, and inject the stuff that changed back into the db
... or, more simply, and probably 99% of uses, be able to give a single dictionary (=single 'object') to something, and get that to be pumped automatically into a table.
...also, the question of how to handle joins
PS we have headless spring now. Do you fancy setting up a botrunner or two?
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 07:31
by hughperkins
Perhaps something like:
Code: Select all
def doinsert( tablename, map ):
global dictcursor
sqlinsert = "insert into " + tablename + " ("
isfirstkey = True
for key in newmap.keys():
if not isfirstkey:
sqlinsert = sqlinsert + " , "
sqlinsert = sqlinsert + key
isfirstkey = False
sqlinsert = sqlinsert + ") values ("
isfirstkey = True
for value in newmap.values():
if not isfirstkey:
sqlinsert = sqlinsert + " , "
sqlinsert = sqlinsert + " %s "
isfirstkey = False
sqlinsert = sqlinsert + ")"
#print sqlinsert
dictcursor.execute( sqlinsert, newmap.values() )
Usage:
Code: Select all
newmap = { 'map_name': 'foo1', 'map_archivechecksum': 12343513, 'map_id': 11, 'map_url': '' }
doinsert("maps", newmap )
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 07:33
by hughperkins
Addendum: we'd also need some way of identifying which fields are the keys so we can choose automatically between inserting and updating.
I don't imagine I'm the first person to want to do this. There's probably something out there somewhere, but I didn't find it yet.
Or maybe I'm looking for the wrong thing, and there's another way of doing this, which is at least as easy, but not quite the same...
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 07:58
by hughperkins
prototype for simple hibernate for python:
http://pastebin.com/f2dfc92da
usage example:
Code: Select all
print pytodb.doselect("maps")
newmap = { 'map_name': 'foo1', 'map_archivechecksum': 12343513, 'map_id': 11, 'map_url': '' }
pytodb.doinsert("maps", newmap )
print pytodb.doselect("maps")
newmap2 = { 'map_name': 'new map name', 'map_archivechecksum': 135135, 'map_id': 11, 'map_url': '' }
pytodb.doinsertorupdate("maps", ['map_id'], newmap2 )
print pytodb.doselect("maps")
pytodb.dodelete( "maps", ['map_id'], newmap )
print pytodb.doselect("maps")
Whether that's easier to use than coding the sql by hand is another question
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 10:21
by imbaczek
sqlalchemy.
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 14:27
by hughperkins
imbaczek wrote:sqlalchemy.
Sweet! I'll check that out. The website gives a great first impression, looks professional, so I'm looking forward to checking that out in more detail.
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 15:40
by hughperkins
Well, that really does seem to behave exactly like hibernate, so I guess what I was looking for was: an object-relational-mapper, and hibernate happens to be one specific implementation of one.
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 17:04
by aegis
yeah, I used sqlalchemy in uberserver and I'm a fan.
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 17:23
by hughperkins
aegis wrote:yeah, I used sqlalchemy in uberserver and I'm a fan.
Make sure to tell me that *after* I've spent ages figuring it out
Re: AI Ladder, 0.4
Posted: 15 Oct 2009, 20:09
by hughperkins
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 20:52
by hughperkins
Right, so sqlalchemy looks like exactly what I was looking for.
There's a new branch sqlalchemy in git where I've started moving to sqlalchemy.
http://github.com/hughperkins/ailadder/tree/sqlalchemy
It seems to work great. Some of the hardest pages are migrated, and look a lot simpler, eg viewrequests.py and viewresults.py.
sqlalchemy needs at least python 2.4 but it also makes it easy to use sqlite and so on for those who want.
Re: AI Ladder, 0.3
Posted: 15 Oct 2009, 21:58
by imbaczek
now you just need to get rid of those fugly prints; use a template language like jinja or mako.
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 00:48
by AF
Gets a lot further but, error:
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 03:20
by aegis
looks like normal internet error
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 04:15
by hughperkins
aegis wrote:looks like normal internet error
So you reckon: just put a try...except around it?
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 04:17
by aegis
sure.
python has an awesome ability to recover from mostly anything more mild than a segfault, don't waste it
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 04:17
by hughperkins
imbaczek wrote:now you just need to get rid of those fugly prints; use a template language like jinja or mako.
Ah, sweet! Which do you recommend? Or I just pick one at random and run with that?
I think I will finish migrating to sqlalchemy first though...
You know, it would have been nice to know about this stuff *before* actually coding 80% of the site
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 04:24
by hughperkins
AF wrote:Gets a lot further but, error:
error.JPG
Hi AF. The verdict seems to be it's a transient error, and the code just needs not to crash during the transient errors.
This new version:
http://manageddreams.com/ailadder/downl ... .1.tar.bz2
... has the getserverrequest surrounded by a try ... except block, so may resolve the issue you are seeing.
Re: AI Ladder, 0.3
Posted: 16 Oct 2009, 04:30
by hughperkins
Guys,
Any hints on the installation process for the website?
I'm thinking, for sqlalchemy, there are a few possibilities:
- ask people to install it themselves, check it is installed
- deploy sqlalchemy with the ailadder setup package, set it up automatically as part of the setup if it's not already installed
- deploy sqlalchemy with the ailadder setup package, and also virtualenv (
http://pypi.python.org/packages/source/ ... 3.4.tar.gz) set it up automatically as part of the setup, to a virtualenv, if it's not already installed
- something else... ?
Thoughts?