python – inserting 8k + character strings into azure sql data warehouse using pyodbc

As the title states, im trying to insert an 8k+ string into azure sql data warehouse using pyodbc in a python notebook. I’ve seen a few answers related to changing the driver, and ive updated to latest driver which supposedly supports this, still no luck.

Heres the code i’m using

import pyodbc
connectionString = "Driver={ODBC Driver 18 for SQL Server};Server=myServerInfo;Database=myDB;Uid=user;Pwd={pw};Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;LongAsMax=1;"
cnxn = pyodbc.connect(connectionString)

cnxn.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
cnxn.setencoding(encoding='utf-8')

cursor = cnxn.cursor()

sql = "insert into myTable (id, veryLongText) values (?, ?)" 
params = (1, 'a' * 8001) 

cursor.execute(sql, params) 

this gives the error

ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]104220;Cannot find data type 'text'. (100000) (SQLExecDirectW)")

if i truncate the string to 4k characters, i dont have any issues, but unfort i need all the data as is for the project.

I’ve tried casting the long string to varchar(max) to see if i can force the driver to use varchar(max) instead of text

sql = "insert into myTable (id, veryLongText) values (?, cast(? as varchar(max))" 

which gives the error:

ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Insert values statement can contain only constant literal values or variable references. (104334) (SQLExecDirectW)')

I’ve also tried using pyodbc.Binary to cast the string into a varchar(max) datatype (chatgpts reommendation lol) but i get this error:

ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Unsupported data type error. Statement references a data type that is unsupported in Parallel Data Warehouse, or there is an expression that yields an unsupported data type. Modify the statement and re-execute it. (104051) (SQLExecDirectW)')

can anyone shed some light on how i can fix this?

Read more here: Source link