Recent changes to bugs

#29 varchar column > 255 characters truncated

2016-11-03T06:48:56.009000Z

Above patch to Sybase.py works for reading long data from the DB but not uploading long data. See patch #13 for a more complete solution.

#41 bcp fails for long string

2016-11-03T06:47:28.288000Z

See Patch #13 which should fix this issue.

#29 varchar column > 255 characters truncated

2016-10-18T23:37:00.606000Z

This has an easy fix in Python-level code (assuming you use at least version 12.5 of ctlib):

Index: Sybase.py

--- Sybase.py (revision 6648)
+++ Sybase.py (working copy)
@@ -298,7 +298,7 @@

# Setup global library context
-status, _ctx = cs_ctx_alloc()
+status, _ctx = cs_ctx_alloc(CS_VERSION_125)
if status != CS_SUCCEED:
raise InternalError('cs_ctx_alloc failed')
#_ctx.debug = 0

#29 varchar column > 255 characters truncated

2015-11-27T10:32:30.810000Z

Just came across the same issue.
Here's how I fixed it locally. Instead of hardcoding a specific version I just put the current one. Definition found in $SYBASE/$SYBASE_OCS/include/cspublic.h
So this line works for me [file: sybasect.c, function: sybasect_cs_ctx_alloc]:
173: int version = CS_CURRENT_VERSION;

Invalid microsecond conversion using DateTimeAsPython

2013-10-08T16:06:59.095000Z

When using DateTimeAsPython for my output map, I found that the microsecond value in datetime.datetime was 1000 times the value stored in the database. here's a patch:

--- Sybase.py   (revision xxx)
+++ Sybase.py   (working copy)
@@ -197,10 +197,10 @@
     DateTimeAsPython = {
         CS_DATETIME_TYPE: lambda val: datetime.datetime(val.year, val.month + 1, val.day,
                                                         val.hour, val.minute,
-                                                        val.second, val.msecond * 1000),
+                                                        val.second, val.msecond),
         CS_DATETIME4_TYPE: lambda val: datetime.datetime(val.year, val.month + 1, val.day,
                                                          val.hour, val.minute,
-                                                         val.second, val.msecond * 1000) }
+                                                         val.second, val.msecond) }
     if _have_cs_date_type:
         DateTimeAsPython.update({
             CS_DATE_TYPE: lambda val: datetime.date(val.year, val.month + 1, val.day) })

Additional note: DateTimeAsPython also raises an error if the datetime field is NULL, probably not the desired behavior.

Select Query error while fetching 5M records using cursor size=1000

2013-08-27T14:10:44.105000Z

I am getting following error while retrieving 5M records using cursor size=1000.
module: python_sybase-0.40
OS: redhat 5 64bit
sybase server version: 15.2
python version: 2.7.2

Traceback (most recent call last):
File "gen_data.py", line 99, in
for entry in connObj.result_iterator():
File "gen_data.py", line 65, in result_iterator
results = self.cursor.fetchmany(arraysize)
File "/python/lib/python2.7/site-packages/python_sybase-0.40-py2.7-linux-x86_64.egg/Sybase.py", line 783, in fetchmany
self._row_result()
File "/python/lib/python2.7/site-packages/python_sybase-0.40-py2.7-linux-x86_64.egg/Sybase.py", line 952, in _row_result
count = self._fetch_rows(self._bufs, logical_result)
File "/python/lib/python2.7/site-packages/python_sybase-0.40-py2.7-linux-x86_64.egg/Sybase.py", line 858, in _fetch_rows
status, rows_read = self._cmd.ct_fetch()
File "/python/lib/python2.7/site-packages/python_sybase-0.40-py2.7-linux-x86_64.egg/Sybase.py", line 265, in _clientmsg_cb
raise DatabaseError(msg)
Sybase.DatabaseError: Layer: 5, Origin: 3
ct_fetch(): network packet layer: internal net library error: Net-Library operation terminated due to disconnect
Exception Sybase.DatabaseError: DatabaseError('Layer: 1, Origin: 1\nct_cursor(): user api layer: external error: This routine cannot be called because the command structure is in an undefined state.',) in ignored
Exception Sybase.DatabaseError: DatabaseError('Layer: 5, Origin: 3\nct_cancel(): network packet layer: internal net library error: Net-Library operation terminated due to disconnect',) in ignored

bcp fails for long string

2012-06-12T02:10:12Z

Hello experts,

I tried to perform bcp for the table with long string.
But it failed with the following error.

===>
Traceback (most recent call last):
File "./test.py", line 12, in <module>
bcp.rowxfer(data)
File "build/bdist.linux-i686/egg/Sybase.py", line 1331, in rowxfer
TypeError: string too long for buffer
<===

I confirmed the error both with python sybase version 0.39 and 0.40 (relased on 2012/06/11).
In the message, I attached a example script for the bcp, and a table definition of the table.

I also tried with a pre sql command "set textsize 65536", but It doese not help.

I appreciate your help to fix this problem.

Takahiro Matsumoto
SPring-8/Japan

* example script for the bcp
--->
import Sybase

con = Sybase.connect(server,user,passwd,database,bulkcopy=1,auto_commit=1)
con.execute("set textsize 65536")

bcp = con.bulkcopy("LI_BPM_INF")

data = [14478988, '6,278499,1336542893,835434,9002,32,900201:-0.183868,900202:-0.174713,900203:-0.180816,900204:-0.175018,900205:-0.084823,900206:-0.193834,900207:-0.158925,900208:5.206670,900209:-0.225982,900210:-0.229950,900211:-0.228424,900212:-0.227203,900213:-0.149371,900214:0.168893,900215:-0.038464,900216:5.292470,900217:0.259857,900218:0.058746,900219:0.031280,900220:0.173187,900221:0.392141,900222:0.306122,900223:0.171315,900224:5.592490,900225:0.926055,900226:0.094757,900227:0.306243,900228:0.116119,900229:1.158630,900230:0.829900,900231:1.594330,900232:5.719180,278499,1336542893,898750,9003,80,900301:0.859527,900302:0.727996,900303:0.732879,900304:0.962677,900305:0.365398,900306:-0.231927,900307:-0.237131,900308:6.054610,900309:0.126800,900310:0.096893,900311:0.143585,900312:0.174408,900313:-0.031849,900314:0.017995,900315:-0.052093,900316:5.515550,900317:-0.079498,900318:-0.118866,900319:0.010834,900320:0.091705,900321:-0.046850,900322:0.058648,900323:-0.002574,900324:5.409000,900325:-0.374298,900326:-0.278778,900327:8.140100,900328:-0.149078,900329:-35.189999,900330:-35.343399,900331:34.676800,900332:11.032900,900333:-0.383758,900334:-0.075531,900335:-0.202789,900336:-0.187530,900337:-0.273444,900338:0.047234,900339:-0.226614,900340:5.197910,900341:-0.411529,900342:-0.190277,900343:-0.169220,900344:-0.225982,900345:-0.262568,900346:-0.003853,900347:-0.209829,900348:5.144450,900349:-0.410919,900350:-0.290985,900351:-0.516510,900352:-0.500946,900353:-0.133514,900354:0.549354,900355:-0.482768,900356:4.936800,900357:-0.440216,900358:-0.578460,900359:-0.703277,900360:-0.444488,900361:0.527011,900362:0.181474,900363:-0.244405,900364:4.832940,900365:-0.521087,900366:-0.589752,900367:-0.677642,900368:-0.456085,900369:0.402894,900370:-0.015435,900371:-0.380922,900372:4.747070,900373:-0.587921,900374:-0.251922,900375:-0.660858,900376:-0.165863,900377:0.014197,900378:0.506781,900379:-1.420020,900380:4.943980,278499,1336542893,889696,9004,48,900401:-1.200400,900402:-1.078030,900403:-1.096340,900404:-1.125030,900405:-0.152537,900406:-0.022578,900407:-0.441863,900408:4.228640,900409:-0.576934,900410:-0.551300,900411:-0.544586,900412:-0.645294,900413:-0.227300,900414:-0.053380,900415:0.218422,900416:5.018500,900417:-0.883331,900418:-0.743560,900419:-0.737457,900420:-0.649871,900421:0.115544,900422:-0.255609,900423:-0.273120,900424:4.999600,900425:-0.735015,900426:-0.635833,900427:-0.746307,900428:-0.499420,900429:0.344715,900430:0.061771,900431:-0.368334,900432:5.061980,900433:-0.264740,900434:-0.481414,900435:-0.637054,900436:-0.248565,900437:0.856570,900438:-0.034503,900439:-0.082828,900440:5.339250,900441:-0.074005,900442:0.068207,900443:-0.040436,900444:-0.111236,900445:-0.393045,900446:0.307678,900447:0.099320,900448:5.437590,278499,1336542893,913569,9005,96,900501:-0.026092,900502:-0.324249,900503:-0.662384,900504:-0.084991,900505:1.332360,900506:0.446868,900507:-0.293990,900508:5.416990,900509:-1.208340,900510:-0.885772,900511:-1.131740,900512:-1.433560,900513:-0.713219,900514:0.652453,900515:0.180684,900516:4.422060,900517:-1.474150,900518:-1.569970,900519:-1.670380,900520:-1.822960,900521:-0.045453,900522:0.451088,900523:0.343343,900524:4.438530,900525:-1.546170,900526:-1.671600,900527:-1.656340,900528:-1.544030,900529:0.167237,900530:-0.045221,900531:0.184364,900532:4.364410,900533:-7.382050,900534:-7.530060,900535:-7.586210,900536:-7.474510,900537:0.516106,900538:0.344557,900539:0.095863,900540:-3.699540,900541:-1.461630,900542:-0.903167,900543:-1.049950,900544:-1.436920,900545:-1.177780,900546:0.217093,900547:-0.637293,900548:3.107920,900549:8.196860,900550:-0.145416,900551:-0.167388,900552:-0.118560,900553:14.511700,900554:39.331200,900555:23.726200,900556:6.488330,900557:0.380401,900558:0.237884,900559:0.370636,900560:0.426178,900561:0.329246,900562:-0.435025,900563:-0.545026,900564:3.491610,900565:-5.176230,900566:-5.164640,900567:-5.186610,900568:-5.087430,900569:-1.062690,900570:0.409785,900571:-0.650489,900572:-0.827818,900573:-5.320280,900574:-4.814600,900575:-5.253140,900576:-5.180510,900577:-0.286145,900578:1.305010,900579:-1.754740,900580:-1.171920,900581:-5.137170,900582:-5.305630,900583:-4.609520,900584:-4.708400,900585:-0.179265,900586:-0.437503,900587:0.445507,900588:-0.572958,900589:-5.217740,900590:-5.323940,900591:-4.965660,900592:-5.195160,900593:-0.658842,900594:-0.900420,900595:0.683489,900596:-0.872636,278499,1336542893,876871,9006,64,900601:2.122650,900602:-0.347137,900603:0.970916,900604:1.034390,900605:3.397380,900606:-0.719938,900607:5.626320,900608:3.745930,900609:1.716760,900610:1.028900,900611:0.021820,900612:1.457060,900613:2.834520,900614:3.665390,900615:-1.892530,900616:3.719690,900617:-5.475610,900618:-5.057830,900619:-5.139610,900620:-4.647360,900621:0.010406,900622:-1.944470,900623:-1.660400,900624:-1.766620,900625:-5.299830,900626:-5.157620,900627:-5.054470,900628:-4.315330,900629:1.129220,900630:-5.579560,900631:-4.626490,900632:-1.839510,900633:-8.515160,900634:-8.670800,900635:-8.649130,900636:-8.530420,900637:0.290835,900638:-0.403259,900639:-0.554375,900640:-2.787630,900641:-7.047270,900642:-7.108610,900643:-7.217560,900644:-9.999840,900645:-57.160400,900646:58.778198,900647:57.738899,900648:-12.303300,900649:-7.306360,900650:-7.604520,900651:-7.416530,900652:-7.577970,900653:0.482920,900654:0.233433,900655:1.252440,900656:-1.525820,900657:-5.381620,900658:-4.575950,900659:-4.958340,900660:-4.640650,900661:-1.594950,900662:-1.567520,900663:0.472036,900664:0.393103,278499,1336542893,876267,9007,56,900701:0.161590,900702:-0.726776,900703:-3.120570,900704:1.062770,900705:6.497410,900706:5.708490,900707:-7.359410,900708:1.895900,900709:-0.131988,900710:-0.772552,900711:-2.180320,900712:0.939788,900713:4.847690,900714:1.679110,900715:-5.559110,900716:2.072430,900717:0.634613,900718:-0.365753,900719:-0.139312,900720:1.111900,900721:3.024060,900722:-2.725770,900723:-0.438676,900724:2.394390,900725:1.183010,900726:1.306910,900727:1.313930,900728:1.210480,900729:-0.377435,900730:-0.127329,900731:-0.103735,900732:2.787370,900733:1.314240,900734:-1.306000,900735:-1.283110,900736:0.810699,900737:6.524040,900738:1.402270,900739:1.005850,900740:1.545360,900741:0.463714,900742:0.488739,900743:0.362396,900744:0.494842,900745:0.044534,900746:-0.001435,900747:-0.358687,900748:1.841470,900749:-0.282440,900750:-0.247955,900751:-0.056000,900752:-0.194549,900753:0.000376,900754:-0.036469,900755:0.019638,900756:1.815470,', '2012-05-09 14:54:53']

bcp.rowxfer(data) # <--- error happend here

bcp.batch()
bcp.done()
<---

* table definition
---->

1> sp__help LI_BPM_INF
2> go
table name insert trigger update trigger delete trigger
-------------------- ------------------ ------------------ ------------------
LI_BPM_INF .................. .................. ..................

Column name Type I Null Dflt Rule Table Num
-------------------- ------------- - ---- ---- ---- -------------------- ------
seq_no int 0 No LI_BPM_INF 1
c2 text 0 No LI_BPM_INF 2
time datetime 0 No LI_BPM_INF 3

**** Index Information ****

Name c u i a s List of Index Keys
------------------------------ - - - - - -----------------------------------
LI_BPM_INF.seq_no Y Y seq_no
LI_BPM_INF.time time
LI_BPM_INF.tLI_BPM_INF N.A.

<----

update for building with freetds

2012-02-14T19:55:59Z

I followed the documentation at this URL for building python-sybase with FreeTDS:

http://python-sybase.sourceforge.net/sybase/node5.html

These instructions did not work as-is. The following is an explanation of what I did to get it to work:

The system constraints I dealt with:

CentOS 5.6 (64-bit: x86-64)
Python 2.7.2 (64-bit)
FreeTDS 0.91 (64-bit)
python-sybase 0.39

Here is what worked:

1. patch setup.py to respect 64-bit:

@@ -120,7 +120,10 @@
sys.exit(1)

syb_incdir = os.path.join(sybase, 'include')
-syb_libdir = os.path.join(sybase, 'lib')
+if have64bit:
+ syb_libdir = os.path.join(sybase, 'lib64')
+else:
+ syb_libdir = os.path.join(sybase, 'lib')
for dir in (syb_incdir, syb_libdir):
if not os.access(dir, os.F_OK):

2. build the extension with the FreeTDS libs:

export SYBASE=/usr
export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
python setup.py build_ext -D WANT_THREADS,HAVE_FREETDS -U WANT_BULKCOPY -l ct,sybdb
python setup.py install

Also, due to an open bug in distutils (see http://bugs.python.org/issue1326113\), I had to patch lib/python2.7/distutils/command/build_ext.py to get the -l flag to handle multiple libraries:

@@ -160,8 +160,7 @@
if plat_py_include != py_include:
self.include_dirs.append(plat_py_include)

- if isinstance(self.libraries, str):
- self.libraries = [self.libraries]
+ self.ensure_string_list('libraries')

api_exists() in setup.py is broken

2012-02-13T20:05:35Z

The regex is bad. It basically catches everything. Here's the diff for that line to a version that works correctly:

- if re.search(r'CS_PUBLIC|CS_RETCODE %s' % func, text):
+ if re.search(r'(CS_PUBLIC|CS_RETCODE) %s' % func, text):

Thanks!

Mac OSX Import Error

2011-10-17T06:28:26Z

I'm trying to install python-sybase on OSX Lion (FreeTDS 0.82). These are the steps I took:

$ sudo python setup.py build_ext -D HAVE_FREETDS -U WANT_BULKCOPY (Successful)
$ python setup.py install (Successful)

But as soon as I try and import the lib, this is want I get. Please could someone help.

Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Sybase
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_sybase-0.40pre1-py2.7-macosx-10.6-intel.egg/sybasect.py:3: UserWarning: Module sybasect was already imported from /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/python_sybase-0.40pre1-py2.7-macosx-10.6-intel.egg/sybasect.pyc, but /Users/jonathan/Downloads/python-sybase-0.40pre1 is being added to sys.path
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "Sybase.py", line 15, in <module>
from sybasect import *
File "build/bdist.macosx-10.6-intel/egg/sybasect.py", line 7, in <module>
File "build/bdist.macosx-10.6-intel/egg/sybasect.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/jonathan/.python-eggs/python_sybase-0.40pre1-py2.7-macosx-10.6-intel.egg-tmp/sybasect.so, 2): Symbol not found: _blk_bind
Referenced from: /Users/jonathan/.python-eggs/python_sybase-0.40pre1-py2.7-macosx-10.6-intel.egg-tmp/sybasect.so
Expected in: flat namespace
in /Users/jonathan/.python-eggs/python_sybase-0.40pre1-py2.7-macosx-10.6-intel.egg-tmp/sybasect.so