Re: [Gnumed-devel] choice of web frameworks

[Karsten Hilbert]

On Tue, Jul 13, 2010 at 12:28:24PM +0200, Hilbert, Sebastian wrote:

> > There is no connection limit unless the user sets one in the
> > PG configuration.
> > 
> > PG defaults to some number like:
> > 
> >     max_connections = 100                   # (change requires restart)
> >     # Note:  Increasing max_connections costs ~400 bytes of shared memory 
> > per
> > 
> > GNUmed uses exactly as many connections as it needs. Given
> > the PG default you could run 30 clients full tilt without
> > needing to change a thing (if the machine is beefed up to
> > the task).
> 
> I was wondering what part of the PG/psycopg2/gmPG2/GNUmed client equation is 
> responsible for the "connection pool exhausted" message referenced earlier.

The "connection pool exhausted" comes from psycopg2's
pool.py:

        """Get a free connection and assign it to 'key' if not None."""
        if self.closed: raise PoolError("connection pool is closed")
        if key is None: key = self._getkey()

        if self._used.has_key(key):
            return self._used[key]

        if self._pool:
            self._used[key] = conn = self._pool.pop()
            self._rused[id(conn)] = key
            return conn
        else:
            if len(self._used) == self.maxconn:
                raise PoolError("connection pool exausted")
            return self._connect(key)

In GNUmed we set maxconn to 2 just for sanity:

def get_connection(dsn=None, readonly=True, encoding=None, verbose=False, 
pooled=True):
        """Get a new connection.

        This assumes the locale system has been initialzied
        unless an encoding is specified.
        """
        # FIXME: support pooled on RW, too
        # FIXME: for now, support the default DSN only
        if pooled and readonly and (dsn is None):
                global __ro_conn_pool
                if __ro_conn_pool is None:
                        __ro_conn_pool = cConnectionPool (
                                minconn = 1,
                                maxconn = 2,
                                dsn = dsn,
                                verbose = verbose
                        )
                conn = __ro_conn_pool.getconn()
        else:
                conn = get_raw_connection(dsn=dsn, verbose=verbose, 
readonly=False)


This already shows some hacks how to quickly fake your way
around it:

- use read-write connections only
- always use an explicit DSN
- set maxconn to some obscene value

I would just try upping maxconn a bit to find out whether it
is just lacking a few connections. If that doesn't fix it
I'd look for a more fundamental, conceptual problem (perhaps
the web client/framework assumes it can close() the
connection it gets from the pool ?).

Karsten
-- 
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346