Thread: [Python-sybase-misc] handling multiple threads

I am trying to develop a python webapp using pysybase that is pretty basic. I want to be able to create SQL QC reports that our users can update on demand as they edit the database.

However, I am having problems getting this to work in a multi-threaded environment like cherrypy. If I execute one long query, no new connections can be made, in fact all other threads come to a halt until that one query returns. Is this something that is not supported by this driver?

Here is a sample test that I've been using:

def sy_query(query):
        db = Sybase.connect(dbServer,dbUser,dbPass,dbName)
        cur = db.cursor()
        cur.execute(query)
t1 = Thread(target=sy_query,args=("select * from blah",))
t2 = Thread(target=sy_query,args=("select something from blah2",))
t1.start()
t2.start()
t1.join()
t2.join()


As soon as thread 1 calls "status = ct_results(self->cmd, &result);" inside of cmd.c nothing happens in either the main function nor in thread 2.
Immediately after that line finishes, everything moves along.

Any suggestions? I can't have a webapp that only works for one user at a time.

-Kevin

The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

On 06/19/13 15:26, Kevin Stone wrote:
> I am trying to develop a python webapp using pysybase that is pretty 
> basic. I want to be able to create SQL QC reports that our users can 
> update on demand as they edit the database.
>
> However, I am having problems getting this to work in a multi-threaded 
> environment like cherrypy. If I execute one long query, no new 
> connections can be made, in fact all other threads come to a halt 
> until that one query returns. Is this something that is not supported 
> by this driver?
>
> Here is a sample test that I've been using:
>
> def sy_query(query):
>         db = Sybase.connect(dbServer,dbUser,dbPass,dbName)
>         cur = db.cursor()
>         cur.execute(query)
> t1 = Thread(target=sy_query,args=("select * from blah",))
> t2 = Thread(target=sy_query,args=("select something from blah2",))
> t1.start()
> t2.start()
> t1.join()
> t2.join()
>
>
> As soon as thread 1 calls "status = ct_results(self->cmd, &result);" 
> inside of cmd.c nothing happens in either the main function nor in 
> thread 2.
> Immediately after that line finishes, everything moves along.
>
> Any suggestions? I can't have a webapp that only works for one user at 
> a time.
>
> -Kevin

It is the intent that everything done in C calls to Sybase would release 
the GIL
and can run in different threads concurrently.
However, this code hasn't had a lot of maintenance recently, and so may
suffer from being out of sync with modern python versions.

So if you're queries take a long time (like  a minute) then
we'd need to determine what your specific configuration is.
What version of Sybase are you linked to, what platform are you on,
and what version of python are you running?

Thanks,

Robert Boehne

The problem is not that queries take a long time. The problem is that queries are not being executed in parallel. All it takes is for one user to run a query (it does not have to be a long one, maybe a few seconds), and the entire app will be down until that query finishes.

I am using python 2.7 on linux with freeTDS against sybase 15.

-Kevin

________________________________________
From: Robert Boehne [ro...@tr...]
Sent: Thursday, June 20, 2013 5:25 PM
To: pyt...@li...
Subject: Re: [Python-sybase-misc] handling multiple threads

On 06/19/13 15:26, Kevin Stone wrote:
> I am trying to develop a python webapp using pysybase that is pretty
> basic. I want to be able to create SQL QC reports that our users can
> update on demand as they edit the database.
>
> However, I am having problems getting this to work in a multi-threaded
> environment like cherrypy. If I execute one long query, no new
> connections can be made, in fact all other threads come to a halt
> until that one query returns. Is this something that is not supported
> by this driver?
>
> Here is a sample test that I've been using:
>
> def sy_query(query):
>         db = Sybase.connect(dbServer,dbUser,dbPass,dbName)
>         cur = db.cursor()
>         cur.execute(query)
> t1 = Thread(target=sy_query,args=("select * from blah",))
> t2 = Thread(target=sy_query,args=("select something from blah2",))
> t1.start()
> t2.start()
> t1.join()
> t2.join()
>
>
> As soon as thread 1 calls "status = ct_results(self->cmd, &result);"
> inside of cmd.c nothing happens in either the main function nor in
> thread 2.
> Immediately after that line finishes, everything moves along.
>
> Any suggestions? I can't have a webapp that only works for one user at
> a time.
>
> -Kevin

It is the intent that everything done in C calls to Sybase would release
the GIL
and can run in different threads concurrently.
However, this code hasn't had a lot of maintenance recently, and so may
suffer from being out of sync with modern python versions.

So if you're queries take a long time (like  a minute) then
we'd need to determine what your specific configuration is.
What version of Sybase are you linked to, what platform are you on,
and what version of python are you running?

Thanks,

Robert Boehne



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Python-sybase-misc mailing list
Pyt...@li...
https://lists.sourceforge.net/lists/listinfo/python-sybase-misc

The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

On Thu, Jun 20, 2013 at 4:59 PM, Kevin Stone <Kev...@ja...> wrote:
> The problem is not that queries take a long time. The problem is that queries are not being executed in parallel. All it takes is for one user to run a query (it does not have to be a long one, maybe a few seconds), and the entire app will be down until that query finishes.
>
> I am using python 2.7 on linux with freeTDS against sybase 15.

Rob and I discussed this a little bit off-line (we happen to work at
the same Sybase-using company).  A quick glance last night suggested
to me that the python-sybase code does its own GIL releasing and
acquiring, not relying on the macros Python provides for this purpose.
 It's possible that over time the GIL-handling code in Python has
changed, and the code in python-sybase hasn't kept up.

Just to be confirm though, your other threads are threads are not even
into the Sybase part of the system, correct?  They aren't blocking
because the first query has locked resources in the server?

I'll try to take a look more closely at GIL-handling code tomorrow.

Skip